Effects and runtimes

Why the engine queues effects instead of running them, and where the runtime lives: the verbs your code calls, and the drainer that delivers queued work.

A workflow regularly needs to act on the world beyond its own fields: send an email, kick off a build, call an API. Workflows splits that work in two. The engine records what should happen, and a runtime you supply makes it happen.

The engine never runs that outside work itself. Effects are declared on actions, the only construct that carries work. When the action fires, whether a caller fired it or its when condition turned true, the engine queues the effect and records what needs doing. A separate runtime then picks it up, runs it, and reports back.

The engine’s job is evaluating rules and moving instances, and that work has to be repeatable: the same instance and inputs always produce the same decision. Sending an email is not repeatable; it can fail, be slow, or happen twice. Queuing keeps the engine deterministic and testable, and puts the messy part where failure and retries belong.

Queuing writes a row into the instance’s pendingEffects. The row carries the effect’s name and its resolved params: an effect’s bindings are GROQ over the workflow’s state, and the engine evaluates them to concrete JSON at the moment it queues the effect. So a handler receives plain values, and never has to read the workflow back. The stored definition names the effect and never references code. A runtime registers a handler against that name, one handler per name.

Nothing runs in the background watching your content and pushing workflows along. @sanity/workflow-engine is a library your process imports: it evaluates rules and commits state when your code calls it, and does nothing at all in between. The runtime is whatever you build around it.

What you call is a small set of verbs. None of them runs unless something calls it.

An action that declares a when condition is a trigger, not a caller-fired action. The engine fires it itself inside whichever call first observes the condition true, so it is never fired through fireAction. A child workflow finishing needs no tick either: the child’s own commit stamps its completion onto the parent and re-evaluates the parent in the same call.

// your runtime calls the engine; the engine never calls itself

// someone or something acted; only an action without `when` is caller-fired:
await engine.fireAction({instanceId, activity: 'perform-review', action: 'approve'})

// nudge after a change the engine can't see; any trigger now true fires in the cascade:
await engine.tick({instanceId})

// a pure read for a UI, no writes:
const view = await engine.evaluate({instanceId})

Every one of those callers is a runtime. A reactive session calls the engine from a UI, a drainer calls it from a server, a scheduled function calls it on the clock. The engine is the rules they all run. The Workflows introduction compares the surfaces you can build a runtime on.

Time passing does not advance an instance until your runtime calls tick. In Workflows 0.32.0, the result's nextEvaluationAt can identify when another tick is useful.

This worker accepts a configured engine and an existing instance ID. It waits until the reported time or checks again after one minute, whichever comes first:

import type {Engine} from '@sanity/workflow-engine'

export async function tickOnSchedule(engine: Engine, instanceId: string) {
  while (true) {
    const result = await engine.tick({instanceId})
    if (result.instance.completedAt) return

    const untilNext = result.nextEvaluationAt
      ? Date.parse(result.nextEvaluationAt) - Date.now()
      : 60_000
    const delay = Math.max(1, Math.min(untilNext, 60_000))
    await new Promise<void>((resolve) => setTimeout(resolve, delay))
  }
}

Call await tickOnSchedule(engine, instanceId) from a long-running worker. It stops when the instance completes or is canceled. A failed tick rejects the call; your worker must handle the failure.

The one-minute check also picks up data changes and time conditions with no reported boundary. A missing timestamp does not mean an active instance is finished. This timer lasts only while the process runs; for a hosted schedule, see Scheduled Functions.

Because an effect is queued rather than run inline, the work it represents need not happen in the process that fired the action, or even on the same machine. The engine records the effect as pending and moves on. Something else picks it up later.

That something is a drainer. It claims a pending effect under a lease, invokes the registered handler, and reports the outcome back so the workflow can cascade. One engine.drainEffects({instanceId}) call keeps going until nothing claimable is left, which includes effects queued by the cascade its own completions triggered. It returns four buckets: drained, failed, skipped for entries with no registered handler, and lost for entries another party completed while this drainer was still dispatching.

Calling code queues work through the engine, which records the pending effect in the Content Lake. The effect runtime owns delivery: it drains and claims the effect before the engine invokes the registered handler and commits the result.

This example assumes a configured client, its workflowResource, and your own sendEmail and createAccount functions. Register the handlers under effects.handlers:

const engine = createEngine({
  client,
  workflowResource,
  tag: 'prod',
  effects: {
    handlers: {
      'notify-reviewer': async (params) => {
        const {id} = await sendEmail(params.to, params.body)
        return {outputs: {messageId: id}} // flows back as $effects['notify-reviewer'].messageId (declared on the effect)
      },
      // a handler's completion can also RETURN ops (the field.* subset), applied to
      // the instance in the completion commit (the effect's "state half"):
      'provision-account': async (params) => {
        const account = await createAccount(params.email)
        return {
          ops: [
            {
              type: 'field.set',
              target: {scope: 'workflow', field: 'accountRef'},
              value: {type: 'literal', value: account.ref},
            },
          ],
        }
      },
    },
  },
})

// a server, cron, or Sanity Function drains whatever is pending:
await engine.drainEffects({instanceId})

Set effects.leaseMs in milliseconds; the default is 300,000 (five minutes). After expiry, another drainer can claim the effect and run its handler again. Choose a duration longer than your slowest handler and make handlers safe to repeat. To release expired claims without dispatching them, call sweepStaleClaims.

A handler lives wherever the work belongs. Each service registers the handlers it owns in effects.handlers. An effect without a registered handler throws MissingHandlerError by default. Set effects.missingHandler to 'skip' to leave it pending for another drainer. Call verifyDeployedDefinitions() at startup to check deployed effect names against this engine’s handlers.

An outcome can be reported from a process that never claimed the effect. A webhook, a cron job, or a function that finishes minutes later can call completeEffect with the entry’s effectKey. A completer on a retrying transport should pass an idempotencyKey, so a redelivery replays instead of failing.

For a complete deployment with a Document Function that drains newly queued effects and a Scheduled Function that ticks existing instances, see Run Workflows with Sanity Functions.

Delivery of a queued effect is at-least-once: a handler may run more than once for the same effect. A dispatch can die after its side effect but before the completion commits, or a lease can lapse mid-dispatch and be taken over by another drain. Both leave the original work done and the entry still pending.

Write handlers that tolerate this. Before irreversible work, check effectHistory for a row keyed by the run’s ctx.effectKey, and derive identifiers you send to the external system from that same key so the receiving system can dedupe a repeat.

Completion is first-writer-wins. When two dispatches of one effect both finish, the first completion to commit is the one recorded. The other one reports the entry in its lost bucket and writes nothing.

When a handler throws, the engine removes the entry from pendingEffects, writes a failed row to effectHistory, and reevaluates automatic transitions using the failed result. The failure is final; the engine does not automatically queue the effect again.

A failed run is a settled run, so a transition gated on defined($effectStatus['<effect>']) fires on failure as well as success. Route the failure somewhere deliberate. Nothing retries the effect unless a stage the instance re-enters queues it again.

A handler can report progress or commit field updates before it returns its final result.

Use ctx.setProgress(...) for progress. Use ctx.commitOps(...) for other field operations.

Always await each report. Otherwise, un-awaited field operations can outrun backpressure and overflow the bounded report queue. Awaiting also surfaces a failed report where it was issued. Give every commitOps call a unique idempotency key.

'generate-assets': async (_params, ctx) => {
  await ctx.setProgress('generationProgress', 25)
  await ctx.commitOps({
    idempotencyKey: `${ctx.effectKey}:asset-ready`,
    ops: [
      {
        type: 'field.set',
        target: {scope: 'workflow', field: 'assetStatus'},
        value: {type: 'literal', value: 'ready'},
      },
    ],
  })
  await ctx.setProgress('generationProgress', 100)
}

An effect can hand values back when it finishes, and later steps can read them. An effect that creates a record elsewhere can return the new ID, and a downstream condition or another effect can use it. The effect declares what it can hand back, as typed outputs on its declaration. A completion that returns anything undeclared is rejected, so unexpected data never reaches the instance.

Two condition variables read a completed run, and they answer different questions. $effects['<effect>'].<output> is the value: the outputs of that effect’s latest completed run, anywhere in the instance’s history. $effectStatus['<effect>'] is the settle state, 'done' or 'failed', and it counts only runs queued during the current stage visit, so it is absent until this visit’s run settles. Gate a transition on $effectStatus. A transition gated on $effects can fire on a value left behind by an earlier visit to the same stage.

import {defineEffect, defineWorkflow} from '@sanity/workflow-engine/define'

export const announce = defineWorkflow({
  name: 'announce',
  title: 'Announce',
  initialStage: 'announcing',
  fields: [
    {type: 'string', name: 'reviewerEmail', initialValue: {type: 'literal', value: 'desk@example.com'}},
  ],
  stages: [
    {
      name: 'announcing',
      title: 'Announcing',
      activities: [
        {
          name: 'notify',
          actions: [
            {
              name: 'send',
              when: 'true',
              status: 'done',
              effects: [
                defineEffect({
                  name: 'notify-reviewer',
                  bindings: {to: '$fields.reviewerEmail'},
                  // The allowlist. A completion returning any other key is rejected.
                  outputs: [{name: 'messageId', type: 'string'}],
                }),
              ],
            },
          ],
        },
      ],
      // Waits for this visit's run to settle. Reads the status, not the value.
      transitions: [{name: 'sent', to: 'announced', when: "$effectStatus['notify-reviewer'] == 'done'"}],
    },
    {
      name: 'announced',
      title: 'Announced',
      activities: [
        {
          name: 'record',
          actions: [
            {
              name: 'file-receipt',
              when: 'true',
              status: 'done',
              effects: [
                defineEffect({
                  name: 'archive-receipt',
                  // Reads the typed output the earlier handler returned.
                  bindings: {messageId: "$effects['notify-reviewer'].messageId"},
                }),
              ],
            },
          ],
        },
      ],
      transitions: [{name: 'archived', to: 'archived', when: "$effectStatus['archive-receipt'] == 'done'"}],
    },
    {name: 'archived', title: 'Archived'},
  ],
})

Effect bindings are conditions too, evaluated over the same scope with a narrower set of variables. Conditions lists what binds where.

Every field operation returned by an effect handler must set target.scope explicitly to workflow or stage. An effect completion has no authoring site the engine could infer the scope from. A missing scope throws EffectOpsInvalidError, commits nothing, and leaves the effect pending so the corrected outcome can be reported again.

A handler receives a concrete sibling of the createEngine client, bound to the workflow resource with its full surface intact, including namespaces such as client.agent.action. clientFor(ref) does the same for another resource. Untagged requests through either client use workflow.effect by default, and on a concrete client an explicit request tag is nested beneath that prefix. The engine never changes the client’s API version.

The declarations and runtime methods in this section are the public contracts for effects. Handler reporting uses the same field-operation vocabulary described in Operations.

This is an effect request in an action’s effects array, commonly authored with defineEffect(...). Its name connects the declaration to a registered runtime handler.

These methods are available as the second argument to every registered effect handler.

• ctx.setProgress(target, value)(): Promise<void>

  Sets an absolute 0–100 progress value on a workflow- or stage-scoped progress field. Pending updates to the same target coalesce.

• ctx.commitOps({ops, idempotencyKey})(): Promise<void>

  Commits other field operations while the handler is running. Each report requires its own idempotency key.

These methods live on the engine and runtime surfaces that claim, run, and complete queued effects. They are not properties of the effect declaration.

• drainEffects({instanceId})(): Promise

  Claims eligible pending effects, invokes registered handlers, and reports drained, failed, skipped, and lost entries.

• completeEffect({effectKey, status, …})(): Promise

  Reports a done or failed outcome. A done completion may include declared outputs and field operations; completion is first-writer-wins.

• commitEffectOps({effectKey, claimToken, ops, idempotencyKey, …})(): Promise

  Low-level mid-dispatch field report. Requires the exact unexpired claim token and renews the lease after a transactional commit. Effect handlers normally use ctx.commitOps or ctx.setProgress.

• findPendingEffects({instanceId, claimed?, names?})(): Promise<PendingEffect[]>

  Lists pending effects, optionally filtered by claim presence or effect name. A present claim may already be expired.

• sweepStaleClaims({client, tag, instanceId})(): Promise<SweepStaleClaimsResult>

  Releases expired claims without dispatching their effects. A standalone export, not an engine method.