This is the second episode of our exciting saga regarding the complete rewrite of our site in Astro. If you haven't had the chance to read the first episode, we highly recommend you do so, as it lays the groundwork and context for what we're diving into today!
Chapter 1: Why we switched to Astro (and why it might interest you)
Chapter 2: Astro, GraphQL and DatoCMS Cache Tags: how we built the killer combo
Chapter 3: Astro, Sitemaps, SEO, and Best Practices
Chapter 4: Cooking
In this episode, we will talk about one of the most critical and fundamental topics that are essential for a DatoCMS-powered Astro website: how to effectively work with GraphQL.
A simple pattern to obtain the data
The vast majority of our site's pages primarily execute GraphQL queries to DatoCMS and convert them into Astro pages. Due to the frequency of this process, it was essential to establish the simplest possible pattern — ideally, a single function call. This is what we came up with (source code):
import { executeQuery } from '~/lib/datocms/executeQuery';import { notFoundResponse } from '~/lib/notFoundResponse';import { query } from './_graphql';
const variables = { slug: Astro.params.slug! };const { productUpdate } = await executeQuery(Astro, query, { variables });
if (!productUpdate) { return notFoundResponse();}
---
<Layout> {JSON.stringify(productUpdate)}</Layout>The executeQuery function does more than just executing the query and returning the result:
Checks if the domain of the request is
wwworwww-draft, and sets theX-Include-Draftsheader accordingly to return draft content or not;Sets the
X-Cache-Tagsheader totrueto obtain the cache tags associated with the GraphQL response;Copies the cache tags obtained from DatoCMS into the
Surrogate-Keyheader of the Astro response (source code). An important consideration is that each Astro component involved in rendering a page may trigger a different GraphQL query, meaning theSurrogate-Keyheader must be a union of all cache tags returned by individualexecuteQueryinvocations.
The global Astro carries all the necessary context for these purposes (i.e. the request and response objects), so it is the only additional argument to pass, besides the GraphQL query and its variables. Fantastic!
Strict Mode for the DatoCMS CDA
When working with TypeScript, remember to always enable Strict Mode when sending GraphQL requests to DatoCMS.
By adding the X-Exclude-Invalid header, you basically filter out invalid records from GraphQL responses, AND enable more precise and reliable TypeScript types that guarantee non-null, correctly validated data fields across your models.
(In our case, the header is set by default by our executeQuery() function)
Caching on Fastly
Let's analyze the headers that our executeQuery() sets on Astro's responses. If the domain of the request is www, the headers will be:
Surrogate-Key: <LIST OF CACHE TAGS>Surrogate-Control: max-age=31536000, stale-while-revalidate=60, stale-if-error=86400The Surrogate-Control header instructs Fastly to cache the response for one year. In addition to the classic max-age, we also use the stale-while-revalidate and stale-if-error directives, which are very powerful:
stale-while-revalidate=60means that for up to one minute after the cache expires, Fastly can immediately serve the stale content while asynchronously fetching a fresh version in the background. This means that visitors do not pay the price of a slower response even in the case of invalidated cache.The
stale-if-error=86400allows Fastly to continue serving the cached content for up to 24 hours if the origin server is unavailable or returns an error, ensuring better availability and user experience during potential server issues.
On the www-draft domain, we do not have Fastly in front, so we are simply concerned with exposing the cache tags for debugging purposes, and instructing browsers to never cache the server responses — we always want the latest data:
Debug-Surrogate-Key: <LIST OF CACHE TAGS>Cache-control: privateInvalidating Fastly's cache
Since our pages will be cached by Fastly for a year, it’s time to focus on invalidating them!
I absolutely love this part every time I work with DatoCMS: setting up a webhook from the interface, writing just 20 lines of code, and accomplishing what previously would have taken weeks of optimization and debugging to invalidate pages correctly! 😅
import type { APIRoute } from 'astro';import { json } from '../_utils';import { FASTLY_KEY, FASTLY_SERVICE_ID } from 'astro:env/server';import ky from 'ky';
export const POST: APIRoute = async ({ request }) => { const data = await request.json();
// DatoCMS sends us the tags to be invalidated via webhook const cacheTags = data.entity.attributes.tags;
const response = await ky.post(`https://api.fastly.com/service/${FASTLY_SERVICE_ID}/purge`, { headers: { 'fastly-key': FASTLY_KEY, // Required for stale-while-revalidate to work! 'fastly-soft-purge': '1', 'content-type': 'application/json', }, json: { surrogate_keys: keys }, }).json();
return json({ cacheTags, response });};Man, I'm so proud of our Cache Tags. 🥰
GraphQL and TypeScript
One of the objectives of the rewrite was to achieve complete TypeScript coverage. For this purpose, we chose to use gql.tada, an incredible library capable of deriving the types for your GraphQL queries on the fly.
We have chosen to organize our routes this way and to declare the GraphQL queries in a _graphql.ts file:
The following is an example of a query. Once passed as an argument to our executeQuery(), the result will be fully typed!
import { ProductUpdateFragment } from '~/components/product-updates/ProductUpdate/graphql';import { TagFragment } from '~/lib/datocms/commonFragments';import { graphql } from '~/lib/datocms/graphql';
export const query = graphql( /* GraphQL */ ` query ProductUpdate($slug: String!) { productUpdate: changelogEntry(filter: { slug: { eq: $slug } }) { _seoMetaTags { ...TagFragment } ...ProductUpdateFragment } } `, [TagFragment, ProductUpdateFragment],);Fragment composition
One of the less talked about strengths of GraphQL, when working in "componentized frameworks" like Astro, React, Vue or Svelte, lies in fragment composition and hierarchical schema design. The gql.tada documentation does a great job of explaining this pattern and its benefits.
Basically, each component defines its own data requirements using a GraphQL fragment stored in a graphql.ts file next to the component itself:
When a parent component includes child components, it aggregates their GraphQL fragments. Here's how the process unfolds for a <Parent /> component:
The parent imports both the child component (e.g.,
<QuestionAnswer />) and its corresponding GraphQL fragment (e.g.QuestionAnswerFragment).It then creates its own fragment by declaring its specific data requirements and incorporating the fragments of its child components.
This fragment composition continues up the component tree until it reaches the top-level Astro page, which will then execute a single, comprehensive combined macro-query.
It's an incredibly powerful pattern: it creates a modular, clean, maintainable way of managing component data requirements in a type-safe, composable manner, enforcing the Single Responsibility Principle and Separation of Concerns. Regardless of how many pages use your component, future changes will require modifications in only one place in your code.
If there's only one takeaway from this article, it's this: use fragment composition!
Simplifying GraphQL Pagination
Sometimes website sections require displaying a large volume of content. A perfect example is our Wall of Love ♥️ , where the goal is to create a wow effect by showcasing numerous quotes, without necessarily expecting visitors to read each one carefully. In such cases, pagination becomes crucial for retrieving entire collections of records.
GraphQL pagination is notoriously challenging — it's rarely as straightforward as developers would like, and especially its logic, unlike REST APIs, is much more difficult to extract and reuse across different parts of your application.
However, it's not impossible! Recently, we've added an ingenious method to our GraphQL client @datocms/cda-client that simplifies this entire process called executeQueryWithAutoPagination:
import { executeQueryWithAutoPagination } from "@datocms/cda-client";
const { allQuotes } = await executeQueryWithAutoPagination(` query WallOfLove { allQuotes(first: 5000) { author quote } }`);What should jump out at you from this piece of code is that we're fetching 5,000 records in a single call — which seems impossible, given that at the moment, pages with DatoCMS are limited to 100 elements!
Well, executeQueryWithAutoPagination automatically analyzes the query and dynamically rewrites it behind the scenes, like this:
query WallOfLove { splitted_0_allQuotes(first: 100, skip: 0) { author quote } splitted_1_allQuotes(first: 100, skip: 100) { author quote } splitted_2_allQuotes(first: 100, skip: 200) { author quote } # ... and so on}Once executed, the results are seamlessly collected and recomposed, making the entire pagination process transparent to the developer, and without requiring multiple calls. It's like magic! ✨
Wrap-up
We hope that sharing our site's data fetching strategies gives you some inspiration for a modern, efficient approach to web development, even if you're using different frameworks than Astro:
we’ve implemented robust content delivery with Fastly caching, and at the same time streamlined route query complexity to a single function call, so that no one can make unintentional errors;
we've built a modular and mantainable GraphQL design with component-level fragments and bottom-up fragment composition.
everything is supported by TypeScript, so it becomes immediately clear if any GraphQL query is incorrect (or becomes incorrect in the future due to a schema change).
Stay tuned for the next episode of our saga, where we'll dive deep into other areas of our website! 👋🏼