Go Behind the Experience to see how Tecovas brings the West to life with Sanity 🤠 July 18th

Discussion about caching issues and image optimization in Gatsby and Netlify site, with a mention of search functionality.

96 replies
Last updated: Apr 8, 2021
Hi everyone. I have a sanity + gatsby + netlify site and the client is constantly complaining that when they update the site, the content doesn’t appear. If I go to the site and hit refesh the content appears. How can I ensure that the content is always fresh? This caching stuff confuses the hell out of me
Apr 8, 2021, 4:49 PM
are you setting custom headers on Netlify?
Apr 8, 2021, 4:55 PM
Nah it’s all default
Apr 8, 2021, 4:56 PM
What is your site again? Perhaps it’s something weird with the header
Apr 8, 2021, 4:57 PM
Apr 8, 2021, 4:58 PM
mmm headers looks fine
Apr 8, 2021, 5:01 PM
do you have gatsby-plugin-offline installed?
Apr 8, 2021, 5:02 PM
I did at one point but I scrapped it in the end for this reason
Apr 8, 2021, 5:02 PM
Are you using the CDN or got that disabled?
Apr 8, 2021, 5:03 PM
It’s been a non stop complaint from them, not an issue I’ve had before with the same set up, I don’t know if their expectations of the site updating instantly are unrealistic
Apr 8, 2021, 5:03 PM
Disabled
user L
Apr 8, 2021, 5:04 PM
gatsby-plugin-offline install workers IIRC — you might have to do something extra to remove installed workers, not just removing the plugin
Apr 8, 2021, 5:04 PM
Might be right. I still have the worker installed from when I helped you out a while back.
Apr 8, 2021, 5:06 PM
When I go to application / service workers, I can see your old gatsby-plugin-offline worker that was installed the first time we talk (3/17/2021)
Apr 8, 2021, 5:06 PM
Do you think that could be causing the cache issues then? Like I say, when I visit the page myself I have to refresh the site to see new content
Apr 8, 2021, 5:27 PM
I’ll try de-registering it and see if that works! 🤞
Apr 8, 2021, 5:27 PM
Client from hell this one haha
Apr 8, 2021, 5:28 PM
In chrome you can disable the service worker to check.
Apr 8, 2021, 5:28 PM
There's a bypass toggle.
Apr 8, 2021, 5:28 PM
I think its worth a try. IIRC gatsby offline plugin default setting is to retrieve data from cache first, then network requests. Not sure if that had changes. Since you had already uninstalled the plugin, might as well remove it completely.
Apr 8, 2021, 5:28 PM
Client from hell definitely haha they seem very picky
Apr 8, 2021, 5:29 PM
might also want this
gatsby-plugin-remove-serviceworker
Apr 8, 2021, 6:11 PM
netlify’s default caching setup is very good (must-validate or whatever) and has eliminated most all of our caching madness
Apr 8, 2021, 6:11 PM
Yeah I did some reading into it
user M
and found that to be the case, which made my head hurt even more that the client was still having issues
Apr 8, 2021, 6:17 PM
Hoping what
user L
and
user G
suggested will solve it! Just back from walking the dog so I’ll try it now
Apr 8, 2021, 6:17 PM
user L
could you check again to see if it’s unregistered for you too? https://dscvrd.co/ It’s gone for me
Apr 8, 2021, 6:28 PM
Service worker is gone
Apr 8, 2021, 6:33 PM
it’s very slow
Apr 8, 2021, 6:35 PM
Thanks Raffi. I just added a demo article and it literally updated immediately, so I’m hoping that’s sorted now
Apr 8, 2021, 6:35 PM
The images, they were going like 4x over their bandwidth on Sanity which was charging me, so I tried offloading this to another service
Apr 8, 2021, 6:36 PM
But yeah I need to look at this as they’re crazy slow
Apr 8, 2021, 6:36 PM
nice job. I think these service workers are one of the most dangerous “easy to enable” features of Gatsby.
Apr 8, 2021, 6:36 PM
we did it because… why not? Oh, that’s why.
Apr 8, 2021, 6:37 PM
100% chris
Apr 8, 2021, 6:37 PM
Ah ok. Then you couldn’t use the auto format either.
Apr 8, 2021, 6:37 PM
If there’s any way I can reduce the bandwidth you know, I’m all ears as I need to sort this haha
Apr 8, 2021, 6:37 PM
are you using the sanity image stuff that requests smaller files with different compression settings? I’m selfishly interested as we recently had an editor use a multi MB PNG for a share image and services wouldn’t use them because it went through untouched
Apr 8, 2021, 6:38 PM
I was originally using gatsby image with gatsby-source-sanity which was giving me all the sizes I needed, but I had to refactor the site to grab the data at load time instead of build time so it was faster for the client publishing articles so I had to look at other ways of handling the images
Apr 8, 2021, 6:40 PM
280kb
Apr 8, 2021, 6:40 PM
user L
can I append that to the weserv.nl CDN or would you suggest just serving them straight from sanity?
Apr 8, 2021, 6:41 PM
I sort of assumed images came from sanity directly with their resizing CDN, but the couple sanity-image plugins do a lot of this behind the curtain
Apr 8, 2021, 6:41 PM
You can’t use a proxy as it’ll server webp in most browsers so will break non webp browsers.
Apr 8, 2021, 6:41 PM
we use this, the developer is in this Slack and is super helpful:
gatsby-plugin-sanity-image
Apr 8, 2021, 6:42 PM
user S
had to rebuild the functionality as an spa within gatsby because his client didn’t want to wait for the rebuilds. I think.
Apr 8, 2021, 6:42 PM
Yeah that’s right… it was a total nightmare haha
Apr 8, 2021, 6:43 PM
I’ve built a lot of sites in Gatsby this past year and never had any issues like this, even got the build time down to like 1m 30s
Apr 8, 2021, 6:44 PM
I’ve come close to giving up the freelance life because of this project 😂
Apr 8, 2021, 6:45 PM
Which functionality did you rebuild as a spa
user S
?
Apr 8, 2021, 6:45 PM
I’m not sure what spa stands for exactly, but I guess it means grabbing data at load time? It was the articles themselves, so the index page, the category page and the article data itself
Apr 8, 2021, 6:46 PM
Single page app.
Apr 8, 2021, 6:47 PM
Things like the header banner, the ‘stories’, the ads, etc, they all come at build time still but as they’re not as time sensitive it was a good trade-off
Apr 8, 2021, 6:47 PM
If you followed through with the reach router approach then that's what I'd call it.
Apr 8, 2021, 6:47 PM
Yeah I did, the reach router thing worked really nicely in the end
Apr 8, 2021, 6:47 PM
Wow, thats a bummer
Apr 8, 2021, 6:48 PM
It's unfortunate that your client didn't understand the concept or benefits of your original approach.
Apr 8, 2021, 6:48 PM
Together with offline and image stuff you've probably throw away 75% benefits of Gatsby hahaha
Apr 8, 2021, 6:48 PM
Yeah exactly my thoughts
Apr 8, 2021, 6:49 PM
Wow, glad you got it working. Is it too late for gatsby cloud?
Apr 8, 2021, 6:49 PM
I’ve learned to make sure I document how gatsby/etc actually work before committing to building anything haha
Apr 8, 2021, 6:49 PM
Interestingly I was talking with a colleague today and he's got a whole page in a proposal about the change in mentality for using a headless CMS and the build process.
Apr 8, 2021, 6:49 PM
I think so
user M
Apr 8, 2021, 6:49 PM
It’s a massive deal
user L
, non-technical clients who have come from wordpress really don’t understand it
Apr 8, 2021, 6:50 PM
Yeah. Sounds like they were used to a traditional approach & had a hard time adjusting
Apr 8, 2021, 6:50 PM
Yeah remember you saying in our long thread the other week.
Apr 8, 2021, 6:50 PM
Site still looks great.
Apr 8, 2021, 6:50 PM
So the outcome to a user apart from image load speed is really good.
Apr 8, 2021, 6:50 PM
Thank you 👏 it means a lot as I’m really at my wits end with it
Apr 8, 2021, 6:50 PM
separate thing, we need some highly dynamic routes due to rapid data changes, but we still need the opengraph tags generated at request time (for crawlers that probably aren’t rendering the JS). Is there a solution to this that essentially amounts to SSR at request time?
Apr 8, 2021, 6:50 PM
liek I’m in your boat now wishing I jsut had a plain express app or something
Apr 8, 2021, 6:51 PM
Thats solution is called nextjs I think
Apr 8, 2021, 6:51 PM
didn’t gatsby bring out something similar to that recently?
Apr 8, 2021, 6:51 PM
Agree w Raffi, your site looked great dude. Did your client give you as much of a difficult time when you were designing it?
Apr 8, 2021, 6:53 PM
well we’d have this but I don’t think it solves for rendering the meta at request time. I’ve even considered a hacked up function that swaps in the few meta tags and hope I can get it to hydrate gatsby correctly If you’re a regular browser https://www.gatsbyjs.com/docs/data-fetching/#fetching-data-at-client-side-runtime
Apr 8, 2021, 6:54 PM
user G
Generally they were onboard the whole way, in fact they were really happy with them. However they dropped it on me yesterday that they need a search function, but it was never in the designs they signed off, but they said they mentioned it in their first email which they did and I didn’t see it… so it’s like this awkward situation about whether I build it as part of the original cost or not 😕
Apr 8, 2021, 6:55 PM
Something that helps is a sitemap when signing off scope with small clients.
Apr 8, 2021, 6:58 PM
Yeah something I’m learning. I started freelancing just over a year ago and generally clients have been really great, but the odd few have made me a sceptic. Looking at written agreements, signed by both parties before any work starts now
Apr 8, 2021, 6:59 PM
Welp lucky you sanity just release a few groq goodness that would really help w implementing search
Apr 8, 2021, 6:59 PM
Hope you sort that pricing thing out
Apr 8, 2021, 7:00 PM
Thanks
user G
it’s all a big learning curve
Apr 8, 2021, 7:00 PM
I watched the sanity seminar about that actually but not sure where I would start looking to get some ideas towards building a search… can you link me to anything? Last favour haha
Apr 8, 2021, 7:01 PM
Good search is hard. You'll get into the arguement about whats good relevancy for listing the results.
Apr 8, 2021, 7:03 PM
Although in this you can probably do text search with a topic filtering perhaps.
Apr 8, 2021, 7:03 PM
Or just settle on text search.
Apr 8, 2021, 7:03 PM
Did the client signoff designs?
Apr 8, 2021, 7:03 PM
It's pretty evident that search is mising and they should have picked up when the site launched.
Apr 8, 2021, 7:04 PM
I posted an example of searching with the new groq functions here https://twitter.com/DerekNguyen10/status/1375329869752455169?s=20
Apr 8, 2021, 7:05 PM
You can use
boost()
to add weight to more important parameters
Apr 8, 2021, 7:06 PM
They did sign off the designs
user L
that’s my side of the “argument” — I agreed I’d do the search but it would have to be in my own time (ie. evenings) and now I’m just getting pressured about it non-stop
Apr 8, 2021, 7:11 PM
Thanks
user G
I’ll take a look at this
Apr 8, 2021, 7:11 PM
They better pay extra for it as well!
Apr 8, 2021, 7:11 PM
😂 they ought to be!
Apr 8, 2021, 7:11 PM
Deliver search, but don't think too much about it then. Best of luck.
Apr 8, 2021, 7:14 PM
Yeah, my thoughts exactly. Keep it as simple as it can be 🙂 thanks for the help everyone
user L
user G
user M
Apr 8, 2021, 7:21 PM

Sanity– build remarkable experiences at scale

Sanity is a modern headless CMS that treats content as data to power your digital business. Free to get started, and pay-as-you-go on all plans.

Was this answer helpful?