Effects and runtimes

Why the engine queues side effects instead of running them, and where the runtime lives: the verbs your processes call to move workflows forward.

Workflows regularly need to act on the world beyond their own fields: send an email, kick off a build, call an API. Editorial Workflows splits that work in two: the engine records what should happen, and a runtime you supply makes it happen.

The engine never runs that outside work itself. Effects are declared on actions, the only construct that carries work: when the action fires, whether a caller fired it or its when condition turned true, the engine queues the effect and records what it wants done, and a separate runtime picks it up, runs it, and reports back.

The engine’s job is evaluating rules and moving instances, and that work has to be repeatable: the same instance and inputs always produce the same decision. Sending an email is not repeatable; it can fail, be slow, or happen twice. Queuing keeps the engine deterministic and testable, and puts the messy part where failure and retries belong.

An effect can hand values back when it finishes, and later steps can read them. An effect that creates a record elsewhere can return the new id, and a downstream condition or another effect can use it. The effect declares what it can hand back, as typed outputs on its declaration; a completion returning anything undeclared is rejected, so off-contract data never reaches the instance.

There isn’t one. The engine is a library, not a service: nothing runs in the background watching your content and pushing workflows along. The engine only does something when your code calls it, so you supply the runtime.

What you call is a small set of verbs. fireAction is the “someone or something acted” signal for actions that wait on a caller, whether that is an editor approving or a webhook arriving; an action with a when condition is a trigger the engine fires itself, inside whichever call first observes the condition true, so it is never fired through fireAction. tick says “something changed that I might care about, look again”, for the kind of change the engine doesn’t hear about directly: the subject was edited, a deadline passed. It re-evaluates, fires any triggers that are now true, and advances the instance as far as it can. A child workflow finishing needs no tick; the child’s own commit propagates the completion to its parent. evaluate is a pure read that projects what an actor can do right now and why not the rest, which is what a UI renders. None of these runs unless something calls it.

// your runtime calls the engine; the engine never calls itself

// someone or something acted; only an action without `when` is caller-fired:
await engine.fireAction({instanceId, activity: 'perform-review', action: 'approve'})

// nudge after a change the engine can't see; any trigger now true fires in the cascade:
await engine.tick({instanceId})

// a pure read for a UI, no writes:
const view = await engine.evaluate({instanceId})

Because there is no service, nothing moves on a timer by itself. A transition that should fire once a deadline passes needs something to call tick after the clock crosses it, whether that is a cron job, a scheduled Sanity Function, or a queue worker. The cascade that advances an instance runs inside the call you made, not in a loop somewhere off-stage.

This is why “the runtime” wears so many hats across these docs. The reactive session calls the engine from a UI, the effect drainer calls it from a server, a scheduled function calls it on the clock. They are all the runtime, and the engine is the rules they run.

Because an effect is queued rather than run inline, the work it represents need not happen in the process that fired the action, or even on the same machine. The engine records the effect as pending and moves on. Something else picks it up later.

That something is a drainer. It claims a pending effect under a lease, invokes the registered handler, and reports the outcome back so the workflow can cascade. Delivery is at-least-once: a claim whose lease lapses is taken over by the next drain, so a handler may run twice. Before irreversible work, check effectHistory for the run’s ctx.effectKey, and derive external-system ids from it so the receiving system can dedupe. The handler lives wherever the work belongs. One service sends email, another rebuilds the site, a Sanity Function handles a third kind, and each registers only the handlers it owns. Completion can be reported from anywhere too, by a webhook, a cron job, or a function that finishes minutes later.

Calling code queues work through the engine, which records the pending effect in the Content Lake. The effect runtime owns delivery: it drains and claims the effect before the engine invokes the registered handler and commits the result.

You register handlers by name when you build the engine, and a separate runtime drains whatever is queued. The outputs a handler returns must be allowlisted as typed outputs on the effect’s declaration, for example outputs: [{name: 'messageId', type: 'string'}] on notify-reviewer; a completion that returns an undeclared key is rejected with EffectOutputsInvalidError.

const engine = createEngine({
  client,
  workflowResource,
  tag: 'prod',
  effectHandlers: {
    'notify-reviewer': async (params) => {
      const {id} = await sendEmail(params.to, params.body)
      return {outputs: {messageId: id}} // flows back as $effects['notify-reviewer'].messageId (declared on the effect)
    },
    // a handler's completion can also RETURN ops (the field.* subset), applied to
    // the instance in the completion commit (the effect's "state half"):
    'provision-account': async (params) => {
      const account = await createAccount(params.email)
      return {
        ops: [
          {
            type: 'field.set',
            target: {scope: 'workflow', field: 'accountRef'},
            value: {type: 'literal', value: account.ref},
          },
        ],
      }
    },
  },
})

// a server, cron, or Sanity Function drains whatever is pending:
await engine.drainEffects({instanceId})

A handler can report progress or commit field updates before it returns its final result.

Use ctx.setProgress(...) for progress. Use ctx.commitOps(...) for other field operations.

Always await each report. Otherwise, un-awaited field operations can outrun backpressure and overflow the bounded report queue. Awaiting also surfaces a failed report where it was issued. Give every commitOps call a unique idempotency key.

'generate-assets': async (_params, ctx) => {
  await ctx.setProgress('generationProgress', 25)
  await ctx.commitOps({
    idempotencyKey: `${ctx.effectKey}:asset-ready`,
    ops: [
      {
        type: 'field.set',
        target: {scope: 'workflow', field: 'assetStatus'},
        value: {type: 'literal', value: 'ready'},
      },
    ],
  })
  await ctx.setProgress('generationProgress', 100)
}

A handler receives a derived client for the workflow resource, while clientFor preserves the concrete client API for another resource. Untagged requests through either client use workflow.effect by default; an explicit request tag is composed beneath that prefix. The engine does not replace the client’s API version.

Every field operation returned by an effect handler must set target.scope explicitly to workflow or stage. An effect completion has no authoring activity site from which the engine could infer scope. A missing scope throws EffectOpsInvalidError, commits nothing, and leaves the effect pending so the corrected outcome can be reported again.

The declarations and runtime methods below are the public contracts for effects. Handler reporting uses the same field-operation vocabulary described in Operations.

This is an effect request in an action’s effects array, commonly authored with defineEffect(...). Its name connects the declaration to a registered runtime handler.

These methods are available as the second argument to every registered effect handler.

• ctx.setProgress(target, value)(): Promise<void>

  Sets an absolute 0–100 progress value on a workflow- or stage-scoped progress field. Pending updates to the same target coalesce.

• ctx.commitOps({ops, idempotencyKey})(): Promise<void>

  Commits other field operations while the handler is running. Each report requires its own idempotency key.

These methods live on the engine and runtime surfaces that claim, run, and complete queued effects; they are not properties of the effect declaration.

• drainEffects({instanceId})(): Promise

  Claims eligible pending effects, invokes registered handlers, and reports drained, failed, skipped, and lost entries.

• completeEffect({effectKey, status, …})(): Promise

  Reports a done or failed outcome. A done completion may include declared outputs and field operations; completion is first-writer-wins.

• commitEffectOps({effectKey, claimToken, ops, idempotencyKey, …})(): Promise

  Low-level mid-dispatch field report. Requires the exact unexpired claim token and renews the lease after a transactional commit. Effect handlers normally use ctx.commitOps or ctx.setProgress.

• findPendingEffects({instanceId, claimed?, names?})(): Promise<PendingEffect[]>

  Lists pending effects, optionally filtered by claim presence or effect name. A present claim may already be expired.

• sweepStaleClaims(…)(): Promise

  Releases expired claims without dispatching their effects.