Skip to content
Get help on Slack

Troubleshooting copying datasets with GitHub Actions in Sanity.io

19 replies
Last updated: Oct 24, 2023
Copying datasets with GitHub Actions not working...I'm following this guide: https://www.sanity.io/guides/multi-environment-deployments to create an GitHub workflow action that copies the production dataset to a "staging" dataset. Because I'm using the embedded studio, I need to modify this code a bit. Here is the code from my action:

name: Copy Sanity Production Dataset
on:
  push:
    branches:
      - "feature/**"
      - "!production"
      - "!main"
      - "!staging"
jobs:
  prepare-dataset:
    runs-on: ubuntu-latest
    env:
      SANITY_TOKEN: ${{ secrets.SANITY_TOKEN }}
      SANITY_PROJECT: ${{ secrets.SANITY_PROJECT }}
    steps:
      - name: Double-check branch name
        run: |
          if [ ${GITHUB_REF##*/} = "production" ]; then exit 1; fi
          if [ ${GITHUB_REF##*/} = "main" ]; then exit 1; fi
      - name: Clone dataset from production
        run: |
          curl --request PUT \
          --url <https://api.sanity.io/v2021-06-07/projects/$SANITY_PROJECT/datasets/production/copy> \
          --header 'Authorization: Bearer '$SANITY_TOKEN \
          --header 'Content-Type: application/json' \
          --data '{
          "targetDataset": "staging",
          "skipHistory": true
          }'
And when I pushed a new "feature/whatever" branch, the action ran "successfully", but when I log in to my feature branch studio that is using the staging dataset, it does not have the data from production. One thing I'm unsure about is the part in the guide that says, "'we can create a token under the “API” section of the management console. It should have the “*manage datasets*” permission.'" When I created a token, I don't see any options that include "manage datasets". I've tried it with both "viewer" and "editor" permissions, but either way production is not getting copied to staging. What am I missing? It doesn't need Deploy Studio privileges does it? I'm not deploying this studio; it's embedded in my Remix app.
Jun 19, 2023, 4:27 PM
Is this on a Business or Enterprise plan? This endpoint is using Cloud Clone , which requires at least a Business plan:

/v2021-06-07/projects/$SANITY_PROJECT/datasets/production/copy
Jun 19, 2023, 4:31 PM
hey
Jun 19, 2023, 4:33 PM
can look at my question please
Jun 19, 2023, 4:33 PM
Jun 19, 2023, 4:33 PM
user D
Please don’t do that.
Jun 19, 2023, 4:34 PM
Ahh, well that answers it. It's on a free plan. The guide mentions that, but it kind of gets lost in translation, which is why I didn't think of that. Can I run something like 
npx sanity dataset copy production staging
inside this action? I'm not too familiar with GitHub Actions yet.
Jun 19, 2023, 4:46 PM
You’d be limited to using export/import, so whatever process you used would need to be able to handle the size of your dataset.
Jun 19, 2023, 4:48 PM
Depending on your needs, ignoring assets may help control the size of the export enough to keep everything in RAM (I don’t know what GitHub Actions offer in terms of RAM or persistent space, if any).
Jun 19, 2023, 4:52 PM
Gotcha. Thank you!
Jun 19, 2023, 4:53 PM
Okay, so I'm running this command in my GitHub action (* indicates its reading the secret):
SANITY_EXPORT_DATASET=*** npx -y sanity@latest dataset export --skip-history production exports/production.tar.gz
But I'm getting this error: 
You must login first - run "sanity login"
. Obviously, I can't login like that, which is what I thought the token was for. Am I not setting the env var correctly like this?
Jun 19, 2023, 8:25 PM
What do you mean by “*** indicates its reading the secret”? Are you passing in 
SANITY_AUTH_TOKEN
?
Jun 19, 2023, 8:39 PM
Yeah, I was just saying that because I copied that output from the terminal and GitHub was just concealing my secret, so I wanted to be clear I wasn't literally passing in *, but I guess I wasn't as clear as I thought 😄. What are token with the "DEPLOY STUDIO (TOKEN ONLY)" permissions for? Is that just for deploying the studio or is that also for when you need to do what I'm trying to do without manually logging in?
Jun 19, 2023, 9:26 PM
Okay, thank you. What I meant was that the env variable you mentioned was SANITY_EXPORT_DATASET, but that’s where you put the three asterisks. Just wanted to make sure the right variable was being used.
Jun 19, 2023, 9:32 PM
I don’t believe any of the user tokens can be used to deploy a Studio—even administrator. The deploy studio robot token needs to be used instead.
Jun 19, 2023, 9:33 PM
Oh, well now that you mention it, I wasn't passing in SANITY_AUTH_TOKEN. Is that some kind of built in token variable name that the Sanity CLI is expecting? Or would I need to create that under my API settings?
Jun 19, 2023, 9:54 PM
Yes, it’s built-in. There’s a short write-up here that touches a bit on when and why it’s needed.
Jun 19, 2023, 10:04 PM
Ah ha. Okay, I got the export/import working now with GitHub workflows. Thank you!
Jun 19, 2023, 10:23 PM
I will need to be mindful, like you said, of the available resources on those workflow machines, but I'll cross that bridge when we get there I guess.
Jun 19, 2023, 10:25 PM
Is there any new and updated Sanity documentation for the Git Actions to do automated backups? Appears to be interopability issues with the CLI/GIT/Node versions?
Anyone currently able to use in production capacity ?
Oct 24, 2023, 3:29 PM

Sanity– build remarkable experiences at scale

Sanity is a modern headless CMS that treats content as data to power your digital business. Free to get started, and pay-as-you-go on all plans.

Get started for freeExplore the demo

Was this answer helpful?