✨Discover storytelling in the AI age with Pixar's Matthew Luhn at Sanity Connect, May 8th—register now

Discussion on how to load dynamic content at load time in Gatsby, including URL structure and SEO concerns.

118 replies
Last updated: Mar 19, 2021
Hello everyone 🙂
Using Gatsby, I need to load in content at
load time not build time which I agree goes against the idea of Gatsby being static content but the client has dropped this on me at the final hour and I don’t have much choice.
The current URL structure is like this:
<http://www.example.com/category/the-article-slug|www.example.com/category/the-article-slug>
but if you visit that, you get a 404 for
category/the-article-slug.js
— ideally here it would somehow load
article.js
and pass it the
/category/
and
/the-article-slug
to fetch with the sanity client
My other option was to change the URL structure like this:
<http://www.example.com/article?category=category&slug=the-article-slug|www.example.com/article?category=category&slug=the-article-slug>
but this loses some of its clean appeal and isn’t ideal
I posted this on Stack Overflow but the reply I got seemed to think it’s impossible

https://stackoverflow.com/questions/66699757/dynamic-pages-at-load-time-not-build-time
Mar 19, 2021, 9:54 AM
Would you not be able to use gatsby incremental builds?
Mar 19, 2021, 9:57 AM
You obviously loose the benefits of gatsby, but means some of your content could be dynamic whilst others rendered at build time.
Mar 19, 2021, 10:00 AM
If you look at the live site currently, it’s all done at build time: www.dscvrd.co but they’re saying the build times are too slow (I optimised the hell out of it and have them down to 1m30s but that’s still not fast enough) so it has to be ‘instant’ now
Mar 19, 2021, 10:01 AM
I’m going to keep a lot of the data at build time, but the articles and article content has to be at load time
Mar 19, 2021, 10:01 AM
Yeah so with gatsby you’d prefix those pages with a path and then use reach router to handle all those urls.
Mar 19, 2021, 10:02 AM
Is your client worried about seo?
Mar 19, 2021, 10:02 AM
I suppose they would be, yeah
Mar 19, 2021, 10:03 AM
I guess that’s going to be the other issue isn’t it — doing it this way is gonna mean these pages don’t ‘exist’ and therefore won’t be indexed?
Mar 19, 2021, 10:04 AM
They will be, google crawls using a mobile js enabled browser, but it does mean they’ll get hit by the google web vitals tax soon.
Mar 19, 2021, 10:04 AM
Mar 19, 2021, 10:05 AM
☝️ saved me a search, thanks
Mar 19, 2021, 10:05 AM
wait a sec.
Mar 19, 2021, 10:05 AM
I guess as long as they’re being indexed, that should be enough for me
Mar 19, 2021, 10:05 AM
What your client should be aware of is that lighthouse scores matter https://web.dev/measure/
Mar 19, 2021, 10:05 AM
On the fullscreen nav do you think there’s a way of stopping it jumping to the yellow on hover of nav items?
Mar 19, 2021, 10:07 AM
They’re coming from a WordPress/database powered website so these are more like issues I’ve created for them by switching to Gatsby 😕
Mar 19, 2021, 10:08 AM
How do you mean?
Mar 19, 2021, 10:08 AM
When i hover over those the yellow flashes up, but if it faded between that would be really nice.
Mar 19, 2021, 10:09 AM
Anyway back to your client.
Mar 19, 2021, 10:09 AM
1m30 is a really good build time.
Mar 19, 2021, 10:09 AM
Ha yeah I could add an animation to it for sure 🙂
Mar 19, 2021, 10:10 AM
I think so, but I suppose they’re going to add more and more pages, eventually that number will increase
Mar 19, 2021, 10:10 AM
I feel like the only way to do this is with the
article?cat=category?slug=example-slug
approach
Mar 19, 2021, 10:11 AM
No. You can do
/articles/slug
Mar 19, 2021, 10:11 AM
or
category/slug
Mar 19, 2021, 10:11 AM
Without getting a 404?
Mar 19, 2021, 10:11 AM
Yes.
Mar 19, 2021, 10:11 AM
You setup a page called category then use reach router to deal with the slug.
Mar 19, 2021, 10:12 AM
Did you take a look at the gatsby link?
Mar 19, 2021, 10:12 AM
I’m just reading it now
Mar 19, 2021, 10:12 AM
I missed it the first time
Mar 19, 2021, 10:12 AM
Also with gatsby incremental builds. it stores a cache so only builds the pages you need to update each time.
Mar 19, 2021, 10:12 AM
How are you building the site?
Mar 19, 2021, 10:12 AM
ci?
Mar 19, 2021, 10:13 AM
I have that, I upgraded to gatsby 3 recently
Mar 19, 2021, 10:13 AM
Netlify yeah
Mar 19, 2021, 10:13 AM
Yeah netlify supports that.
Mar 19, 2021, 10:13 AM
So it’s likely your 1m30 build is actually your baseline? where nothing is really changing.
Mar 19, 2021, 10:13 AM
Then they’ll be a slight overhead for each page that changes per build?
Mar 19, 2021, 10:13 AM
That adds up
Mar 19, 2021, 10:14 AM
What i’d do is a quick proof of concept for an article using reach-router.
Mar 19, 2021, 10:14 AM
Then profile it in lighthouse.
Mar 19, 2021, 10:14 AM
That’ll give you a sense of the value to the user for a longer build time.
Mar 19, 2021, 10:14 AM
and for your client improved seo and google rank.
Mar 19, 2021, 10:15 AM
Did you design it too?
Mar 19, 2021, 10:15 AM
Yeah it’d be nice to put both things forward. They were really p--’d off about the “long build time”, my fault as I guess I should have been more forthcoming about the approach but hey, we’re learning
Mar 19, 2021, 10:16 AM
Yeah I designed it too
Mar 19, 2021, 10:16 AM
A developer and designer. Good job.
Mar 19, 2021, 10:16 AM
Thanks Raffi 🙂
Mar 19, 2021, 10:16 AM
Coming from wordpress what was their cache times?
Mar 19, 2021, 10:16 AM
I don’t think they’d have that information
Mar 19, 2021, 10:16 AM
that was probably 1 minute or more.
Mar 19, 2021, 10:16 AM
Cache time would be the time from when a user visits the site, to revisiting the site and seeing fresh content from the database?
Mar 19, 2021, 10:17 AM
Well in wordpress they’ve probably got a cache somewhere.
Mar 19, 2021, 10:17 AM
on the server or cdn.
Mar 19, 2021, 10:18 AM
Did the gatsby link give enough info?
Mar 19, 2021, 10:18 AM
I haven’t used
@reach/router
before
Mar 19, 2021, 10:19 AM
So just getting to grips
Mar 19, 2021, 10:19 AM
Yeah this is what I’m reading 🙂
Mar 19, 2021, 10:20 AM
I’ve created a
pages/category.js
file and I would guess I need to put the router code here?
Mar 19, 2021, 10:22 AM
The page will be your path.
Mar 19, 2021, 10:22 AM
then:

import React from "react"
import { Router } from "@reach/router"
import Layout from "../components/Layout"
import Article from "../components/Article"
import Listing from "../components/Listing"
const App = () => {
  return (
    <Layout>
      <Router basepath="/category">
        <Article path="/:slug" />
        <Listing path="/" />
      </Router>
    </Layout>
  )
}
export default App
Mar 19, 2021, 10:22 AM
and in your article component you can use: https://reactrouter.com/web/example/url-params
Mar 19, 2021, 10:23 AM
and this would be in
category.js
right? ☝️
Mar 19, 2021, 10:23 AM
If you want it to match
/category
Mar 19, 2021, 10:23 AM
then gatsby builds a placeholder page.
Mar 19, 2021, 10:23 AM
So what you are really doing is building a single page app in your gatsby built site.
Mar 19, 2021, 10:24 AM
each base path needs it’s own page / router.
Mar 19, 2021, 10:24 AM
What urls structures do you want to support?
Mar 19, 2021, 10:25 AM
Well from the live site, you can visit the category directly which gives you a list of articles within this category: https://dscvrd.co/noise
Mar 19, 2021, 10:27 AM
ah ok.
Mar 19, 2021, 10:27 AM
and then from that you can go to an article: https://dscvrd.co/noise/slowthai-misunderstood-or-making-trouble
Mar 19, 2021, 10:27 AM
you need it for wildcard base paths.
Mar 19, 2021, 10:27 AM
The category pages are created at build time which is fine, they rarely if ever add new categories
Mar 19, 2021, 10:27 AM
Ok. What you’ll need to do is a bit annoying then.
Mar 19, 2021, 10:28 AM
You need 1 page per category.
Mar 19, 2021, 10:29 AM
So you’ll probably want to create a template for a page then dynamically add the pages you need with a script before the gatsby build.
Mar 19, 2021, 10:29 AM
Ha yeah I thought as much, so
noise.js
format.js
etc?
Mar 19, 2021, 10:29 AM
Otherwise your root page will need to be dynamic too.
Mar 19, 2021, 10:30 AM
or could I not use my
category.js
TEMPLATE file for this?
Mar 19, 2021, 10:30 AM
Yeah you can do that.
Mar 19, 2021, 10:31 AM
your page context would add the basepath for the router.
Mar 19, 2021, 10:32 AM
I’m doing this which seems to work nicely:

const Category = ({ path }) => (
  <>
    <Router basepath={path}>
      <ArticlePage path="/:slug" />
      <CategoryListings path="/" />
    </Router>
  </>
);
Mar 19, 2021, 10:58 AM
Although, actually, the article pages return a 404
Mar 19, 2021, 10:59 AM
looking for
src/pages/noise/slowly-slowly-isolation-radio-station.js
Mar 19, 2021, 10:59 AM
Is this in the build step or the server?
Mar 19, 2021, 11:17 AM
Server. If I visit
/noise/
it picks up the category route
Mar 19, 2021, 11:17 AM
but visiting
/noise/some-slug
it 404's
Mar 19, 2021, 11:17 AM
  <Router basepath={path}>
    <ArticlePage path="/:slug" />
    <CategoryListings path="/" />
  </Router>
Mar 19, 2021, 11:18 AM
No 😓
Mar 19, 2021, 11:18 AM
That tells gatsby router to forward the path/* to the same page.
Mar 19, 2021, 11:18 AM
I guess I would swap
app
with
category
here?
Mar 19, 2021, 11:20 AM
yes.
Mar 19, 2021, 11:20 AM
oh sorry.
Mar 19, 2021, 11:20 AM
you want to do it per category.
Mar 19, 2021, 11:21 AM
So
noise
,
format
, etc?
Mar 19, 2021, 11:21 AM
// Implement the Gatsby API "onCreatePage". This is
// called after every page is created.
exports.onCreatePage = async ({ page, actions }) => {
  const { createPage } = actions
  // Loop through categories and do the following per category
  if (page.path.match(/^\/{CATEGORY_SLUG}/)) {
    // page.matchPath is a special key that's used for matching pages
    // with corresponding routes only on the client.
    page.matchPath = `/${CATEGORY_SLUG}/*`
    // Update the page.
    createPage(page)
  }
}
Mar 19, 2021, 11:23 AM
Where does CATEGORY_SLUG come from?
Mar 19, 2021, 11:24 AM
it’s psuedo code.
Mar 19, 2021, 11:24 AM
However you got the listing of categories for the page creation.
Mar 19, 2021, 11:24 AM
Ah
Mar 19, 2021, 11:24 AM
That’s handled in another function so might be tricky to pull it across. Maybe it’s easier to just include the category names in the regex as they’re pretty static
Mar 19, 2021, 11:26 AM
^\/(drop|format|kulture|noise|shred)\/$
Mar 19, 2021, 11:27 AM
just do that.
Mar 19, 2021, 11:27 AM
🙂
Mar 19, 2021, 11:27 AM
Feels hacky but this whole approach isn’t ideal
Mar 19, 2021, 11:28 AM
By the way thank you for being so helpful today, I’ve been pretty stressed out about this and you’ve really helped me out
Mar 19, 2021, 11:28 AM
Well if you wanted it dynamic you’d do a sanity query to pull the categories and do it that way.
Mar 19, 2021, 11:32 AM
Obviously still wouldn’t be
instant
Mar 19, 2021, 11:32 AM
True!
Mar 19, 2021, 11:33 AM
Nearly there with this approach 👌 if I can get the pages creating, I can pull all the data in tonight
Mar 19, 2021, 11:33 AM
Hope it all works out.
Mar 19, 2021, 12:12 PM
Thanks! I have the slug pulling in now, just need to run the groq query to fetch the data and we’re done 😄
Mar 19, 2021, 12:15 PM

Sanity– build remarkable experiences at scale

Sanity is a modern headless CMS that treats content as data to power your digital business. Free to get started, and pay-as-you-go on all plans.

Was this answer helpful?