Issue with API rate limits causing errors in build script

53 replies

Last updated: Mar 9, 2022

It seems we hit one of the api rate limits per IP when one of our developers was migrating data using the API and now he seems to be blocked: https://www.sanity.io/docs/technical-limits
How can we enable his IP again? How long does the block stays?

Feb 18, 2022, 12:51 PM

Hi User. Sorry for not getting back to you earlier. Is your teammate still being blocked?

Feb 18, 2022, 2:39 PM

Hello

user E

, yep I'm still getting this error (changed the project id for

foo

) :

web:build: Error occurred prerendering page "/de/about". Read more: <https://nextjs.org/docs/messages/prerender-error>
web:build: Error: getaddrinfo ENOTFOUND <http://foo.apicdn.sanity.io|foo.apicdn.sanity.io>
web:build:     at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:71:26)

Feb 18, 2022, 4:57 PM

It seems like User is being blacklisted from this slack too 😅

Feb 18, 2022, 4:59 PM

it's strange because it happening randomly, the build script render like 300 pages then I got this error and after that the script render more 200 pages or so without any error.

Feb 18, 2022, 5:01 PM

it's strange because it happening randomly, the build script render like 300 pages then I got this error and after that the script render more 200 pages or so without any error.

Feb 18, 2022, 5:01 PM

some times I get like 5 same error for different pages

Feb 18, 2022, 5:02 PM

Hi both! Sorry User, did we miss a different thread that you posted? We try our best to keep on top of new messages, but sometimes it can take a little while for us to get to them.
Do you have an estimation of how many API calls you are sending? Is it just one per page?

Feb 18, 2022, 5:15 PM

I just posted in this thread, User told about slack blacklist because my user appears blank for some reason lol

Feb 18, 2022, 5:21 PM

well in this case it's one query per page (build script) and we have 469 pages

Feb 18, 2022, 5:21 PM

but I think the main problem is because I ran some times a migration script that probably did something around 1k requests each time that I ran it

Feb 18, 2022, 5:24 PM

Thank you for providing the further details. The Slack block list is news to me!
Can you confirm your build script is issuing requests to the API CDN? It looks like it is based on your error message, but I just wanted to double check.

Feb 18, 2022, 5:26 PM

I think so, because of the subdomain in the error, but we are not calling directly requests to the API CDN we are using the property

useCdn: true

in our client

Feb 18, 2022, 5:29 PM

I just set the

useCdn

false

now for test and now I got this error running the build again:

web:build: Error occurred prerendering page "/fr/404". Read more: <https://nextjs.org/docs/messages/prerender-error>
web:build: Error: getaddrinfo ENOTFOUND <http://foo.api.sanity.io|foo.api.sanity.io>
web:build:     at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:71:26)

Feb 18, 2022, 5:33 PM

what is strange for me is:I'm not getting the error for all requests, for example 200 pages are rendered (200 queries runs with success) then I get the error and after that I have more queries that runs with success as well

Feb 18, 2022, 5:36 PM

and seems like I'm the only one with this problem lol

Feb 18, 2022, 5:37 PM

I just checked in with the team. I don't think this is due to rate limiting. Our API CDN doesn't have a rate limit in place, and any requests to our APIs that are rate limited will fail with a

429 - Too many requests

status.
What might be happening is that you're firing off too many requests simultaneously for node to handle. Node doesn't cache DNS, so it has to resolve for every request. I think this would explain why your issue is sporadic.

Would you be able to try throttling your build script? `p-throttle`
is a great tool for doing this .

Feb 18, 2022, 5:39 PM

Let me try do that, I'm just thinking why just me? lol even the build running in the Vercel don't get this error

Feb 18, 2022, 5:45 PM

I'm using the config for the throttle package:

const throttle = pThrottle({
	limit: 1,
	interval: 1000
});

the first time that I ran it build successfully, but I triggered again the build using the same thrittle config and got the error again after 117 queries

Feb 18, 2022, 5:53 PM

increasing the interval for test

Feb 18, 2022, 5:54 PM

const throttle = pThrottle({
	limit: 1,
	interval: 5000
});

I'm still getting the error

Error: getaddrinfo ENOTFOUND <http://zzz.apicdn.sanity.io|zzz.apicdn.sanity.io>

Feb 18, 2022, 5:56 PM

I'm thinking if it couldn't be an issue with the DNS in my region maybe

Feb 18, 2022, 5:59 PM

https://www.whatsmydns.net/#A/apicdn.sanity.io

Feb 18, 2022, 6:20 PM

I'm getting different results every time I refresh the DNS Checker page

Feb 18, 2022, 6:36 PM

I'm getting different results every time I refresh the DNS Checker page

Feb 18, 2022, 6:36 PM

Thank you for trying the throttling! Have you tried an alternative DNS resolver?

Cloudflare provides `1.1.1.1` , and Google provides `8.8.8.8` .
It would be interesting to run your build script while one of these alternative resolution services are active for your system.

Feb 21, 2022, 10:51 AM

let me try that

Feb 21, 2022, 11:21 AM

let me try that

Feb 21, 2022, 11:21 AM

I'm Still getting the error 😞

Feb 21, 2022, 11:42 AM

I think this is unlikely to be the issue, but can you try flushing your DNS cache by running this command in the terminal?

sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder

Feb 21, 2022, 12:38 PM

Same error after run the command

Feb 21, 2022, 1:56 PM

Same error after run the command

Feb 21, 2022, 1:56 PM

Could it be something with my network provider maybe?

Feb 21, 2022, 2:01 PM

I'm in moving process now so in some days I will test with another provider and let you know

Feb 21, 2022, 2:26 PM

I'm having similar problems. It's a nuxt site with the @nuxtjs/sanity plugin.

yarn dev

works fine but

yarn generate

emits spurious errors like

getaddrinfo ENOTFOUND <http://xyz.apicdn.sanity.io|xyz.apicdn.sanity.io>

read ECONNRESET

I'm not really sure when this started, possibly in the last couple of days

Feb 22, 2022, 2:12 PM

same here, it started some days ago

Feb 22, 2022, 5:31 PM

same here, it started some days ago

Feb 22, 2022, 5:31 PM

Now in my new place the error stopped, seems like it was something related to the network provider

Feb 28, 2022, 1:05 PM

Thanks for letting us know, Userian! Really glad this is working for you now. I'd love to get to the bottom of this one, but it's quite tricky to debug remotely.

Feb 28, 2022, 4:03 PM

Yeah totally understand that User, Thank you for your support

Feb 28, 2022, 4:06 PM

Having the same issue now. Why does the DNS change propagation that often?

Feb 28, 2022, 6:31 PM

now I got the same error again

Feb 28, 2022, 6:40 PM

😞

Feb 28, 2022, 6:40 PM

Really sorry to hear this issue is back 😔. What happens if you run a

traceroute

for the failed request domain from your terminal?

Mar 1, 2022, 4:18 PM

❯ traceroute <http://apicdn.sanity.io|apicdn.sanity.io>
traceroute to apicdn.sanity.onl (34.102.168.221), 64 hops max, 52 byte packets
 1  192.168.3.1 (192.168.3.1)  3.825 ms  2.256 ms  2.193 ms
 2  172.31.255.255 (172.31.255.255)  4.801 ms  5.117 ms  4.388 ms
 3  201.16.64.12 (201.16.64.12)  5.185 ms  5.530 ms  5.512 ms
 4  * * *
 5  * * *
 6  * * *
 7  <http://ae5-0.core02.spo1.commcorp.net.br|ae5-0.core02.spo1.commcorp.net.br> (201.16.1.93)  46.940 ms  17.810 ms  15.728 ms
 8  142.250.162.184 (142.250.162.184)  17.183 ms  16.140 ms  16.512 ms
 9  108.170.227.19 (108.170.227.19)  15.305 ms  15.032 ms
    108.170.226.225 (108.170.226.225)  14.828 ms
10  142.251.67.85 <tel:(142.251.67.85|(142.251.67.85>)  14.919 ms
    209.85.251.5 (209.85.251.5)  15.156 ms  14.931 ms
11  <http://221.168.102.34.bc.googleusercontent.com|221.168.102.34.bc.googleusercontent.com> (34.102.168.221)  14.245 ms  14.670 ms  14.597 ms
williantamagi in ~ took 45s

Mar 1, 2022, 4:22 PM

Hmm. Thanks for trying that. Would it be convenient to add a retry mechanism to your requests? `p-retry` would probably be the easiest way to do this .
If there is an intermittent connectivity issue between Sanity and yourself, this would help make your build script more fault tolerant.

Mar 1, 2022, 4:27 PM

Also: User and me are located in two different continents. It's not possible that this is related to sanity and ourselves. I think it has to do with weird dns propagation. If you see the screenshot I shared, many places were red.

Mar 1, 2022, 4:33 PM

I can try that

Mar 3, 2022, 5:19 PM

We found the problem, it's related to the Promise.all that was using in our project with 5 requests to the sanity and it was executing in every generated page, the fast fix was remove the Promise.all and replace to the sequential request using the

await request1 ... await request2 ....

approach.For reference:
https://github.com/vercel/next.js/issues/12494

Mar 8, 2022, 6:19 PM

I'm so glad you got to the bottom of this, and thank you for sharing the solution! That is certainly an interesting one.
I wonder whether allowing
some concurrency would be a better solution than running the requests serially. Either way, this is incredibly useful debugging. I'm sorry for the pain you had to go through to find the answer!

Mar 9, 2022, 11:26 AM

No problem at all User, we are trying to solve this thing of request a lot of time the fetches, I think when we fix that maybe we can use the Promise.all again

Mar 9, 2022, 4:50 PM

No problem at all User, we are trying to solve this thing of request a lot of time this fetches, I think when we fix that maybe we can use the Promise.all again

Mar 9, 2022, 4:50 PM

Sanity– build remarkable experiences at scale

Sanity is a modern headless CMS that treats content as data to power your digital business. Free to get started, and pay-as-you-go on all plans.

Get started for free Explore the demo

Platform

Features

Use cases

Integrations

Learn

Build and share

Frameworks

Discover

Case studies

Popular guides

Issue with API rate limits causing errors in build script

Sanity– build remarkable experiences at scale

Was this answer helpful?