Run lifecycle

Cloudflare Workflows coordinates each managed run, while PlanetScale stores the authoritative run, attempt, signal, approval, and state-commit facts. Workflow state carries opaque identities and capability handles, not secret plaintext or artifact bodies.

The Workflow instance ID is the run public ID. Every run also retains the Workflow definition and application build SHA selected at creation. The run review returns those values rather than inferring a version from the currently deployed Worker.

Source provenance

For a run created from GitHub, the authenticated run review follows the organization-scoped run, input snapshot, committed configuration, ready source ingestion, and configuration artifact records. It displays the provider, full commit SHA, working directory, source-ingestion public ID and ready status, source Workflow ID, GitHub delivery ID, configuration-version public ID, archive SHA-256, and configuration-artifact public ID.

This provenance comes from the persisted relational chain, not source content embedded in the input snapshot. A run created without source ingestion reports no source provenance. The review never includes R2 object keys, ciphertext metadata, upload capabilities, archive bytes, installation tokens, webhook secrets, or repository content.

Main path

dispatch_pending
  → queued
  → planning
  → plan_complete
  → policy_evaluating
  → policy_passed
  → awaiting_approval
  → applying
  → succeeded

plan_only and speculative runs move from policy_passed directly to succeeded. A plan_and_apply run whose finished plan has no changes also terminates there: its approval and apply stages are marked skipped, its pending apply record is closed, and no no-op apply is dispatched.

The plan_complete transaction also reserves the deterministic policy-check resource with status running. That pending resource is visible atomically with the plan, so native OpenTofu clients never have to race policy-evaluation row creation. Its input digest remains null only until the evaluator binds the exact sanitized plan input; completed evaluations always have a valid digest and complete result evidence.

For apply runs, the review reports the apply record independently from state publication. A populated resulting_state_version_id is resolved to its exact state public ID, checksum, serial, and creation time. A null pointer is reported as not_recorded; a non-null pointer that cannot be resolved is reported as unavailable. Neither condition is converted into a successful publication claim.

Durable statuses

Terminal or intervention statuses are:

• plan_credentials_required
• plan_failed
• policy_failed
• rejected
• expired
• canceled
• apply_failed
• apply_start_uncertain
• apply_interrupted
• needs_recovery

An unexpected state transition is rejected instead of silently skipping a control.

If no decision commits before the persisted approval_expires_at instant, the Workflow durably requests database arbitration after its user-action wait throws. PlanetScale either returns a persisted pre-deadline action, reports that the wait is still pending, or atomically moves awaiting_approval to terminal expired with reason APPROVAL_EXPIRED, closes the approval stage, and releases the workspace writer reservation. A pre-deadline approval continues even when Workflow event delivery was delayed. A still-pending result preserves the original wait failure rather than labeling it expiry. The expired Workflow runs normal cleanup and never prepares apply. Late approve and reject requests return 409 APPROVAL_EXPIRED; operators must create a new run.

Planning

The runner bootstraps from an immutable manifest and streams encrypted configuration, lock-file, and predecessor-state artifacts. Inputs bind the source, exact OpenTofu version, region, variables, credential decisions, and plan options. The runner produces bounded logs and encrypted plan artifacts.

At most 100 MB of plaintext is accepted for launch artifacts. Workers forward streams with backpressure and do not aggregate a complete project in memory. If fail-closed launch controls temporarily deny a new run, the native cloud API returns 503 service_unavailable with Retry-After: 60 and creates no run record. Existing runs and read paths remain available.

Policy

Policy publication and evaluation use the same managed sandbox base and generated policy-runner bundle. The sandbox has unrestricted public egress for on-demand toolchain bootstrap; the evaluator still receives only a sanitized JSON derivative and fixed deterministic Rego capabilities, with no customer cloud credentials. Its result is persisted and digest bound before progression. The evaluator can only bind the pending row reserved by that run’s successful plan commit. Rebinding a different input digest, completing a row without a digest, or recording results before binding is rejected.

Approval and apply

Approval binds the plan, dependency bundle, predecessor state, and policy result. Before apply, StackShip acquires a fenced workspace lock and rechecks launch controls, tenant suspension, region, capacity, and the pinned runner release identity against both the runner and policy-runtime allowlists. Runner start repeats those exact checks against the release identity stored on the run; a policy result digest never substitutes for the runtime identity.

Credential capabilities are attempt-, phase-, sandbox-, and expiry-bound. Bootstrap and secret capabilities are one-time. The bootstrap broker may replay the same JTI only for an exact, expiry-bounded attempt, phase, and sandbox redemption so a lost runner-start response remains recoverable; the runner-session store consumes that JTI once. The apply-start capability retains an expiry-bounded exact redemption result so a lost response can be retried without accepting changed input.

Workflow runner-start steps are replay-safe. Before minting or redeeming a new bootstrap handle, the orchestrator asks the control plane whether the exact attempt already has command/session evidence or an authenticated terminal result. A retry with matching persisted authorization, sandbox identity, and deterministic execution nonce completes from that evidence without starting the runner again. Current launch gates still protect new starts but cannot invalidate an already-proven start. Partial or mismatched evidence fails closed.

The apply process reports its start with the manifest digest already verified during bootstrap; the runner never recomputes that authority with a second serializer. OpenTofu reads the run-pinned predecessor and publishes the successor through the Cloud/TFE API. After the command exits zero, a session-bound metadata check must prove that the current published state has the exact run, attempt, sandbox, predecessor, lock, fence, and checksum. Only then can the result callback commit that state and release the writer lock. Unavailable or mismatched confirmation is recovery uncertainty, not success. Ordinary successful Cloud applies create no local state/output artifacts. OpenTofu can publish partial state even when the provider operation exits nonzero. That path remains a failed apply and retains its writer authority for recovery; the published state is not silently reclassified as success.

Recovery posture

StackShip never treats an ambiguous provider response as success. An uncertain apply enters apply_start_uncertain or needs_recovery. A user cancellation that wins the apply-result race stops the bound sandbox and enters apply_interrupted; the writer fence remains held until explicit recovery. The run page’s recovery panel — positioned between the commit-identity header with its approval bar and the stage timeline — compares the exact pre-apply state pointer with the current pointer and, when present, displays the emergency-state digest and size. While that writer authority is held, a new native OpenTofu run fails with an actionable workspace-lock error directing the operator to complete pending run recovery before retrying. It is not reported as ordinary state movement.

Recovery requires both run:recover and state:recover, step-up reauthentication within 15 minutes, a non-empty operator reason, and explicit confirmation that StackShip may revoke the interrupted authority and advance the workspace fence. If the current pointer moved after the run began, the server accepts current_state_verified only when the current version and its published upload carry the exact interrupted run, attempt, sandbox, predecessor, lock, and fence. A contiguous chain of up to 1,024 exact publications is accepted. This covers roughly 720 twenty-second persistence intervals during the four-hour runner authority window with operational headroom. TF_STATE_PERSIST_INTERVAL is platform-reserved so workspace variables cannot expand that publication rate. The apply authorization, exact managed-run lock and writer reservation must still be active. Prepare and commit independently revalidate that provenance while holding the lineage lock. Any other movement is rejected. import_emergency_state still requires the original predecessor to remain current; its emergency snapshot must be the exact next lineage/serial successor and cannot overwrite a newer version.

Durable uploaded or claimed-pending successors are finalized before recovery. Only an expired pending upload with neither a raw nor JSON claim is discarded automatically under a database compare-and-set. An unresolved claimed upload blocks recovery by default. Run review exposes its exact state upload id and claim timestamp. Submission first attempts exact durable finalization. If the artifact is still unavailable, a destructive override must confirm both values and acknowledge that in-flight state can be lost; commit fences the apply and discards that exact row atomically. run.recovered records the superseded id and claim timestamp.

The recovery mutation records a durable operation before copying artifacts. An exact replay by the same principal and request resumes the same target state identity. A completed replay returns the durable result without depending on artifact storage. A prepared emergency replay verifies the persisted target through EOF; corrupt persisted content fails closed and is never silently replaced from the source artifact. A prepared recovery leases that principal for 15 minutes. No takeover is accepted before expiry; after expiry, another operator with both recovery permissions and recent step-up must explicitly confirm takeover while resending the same resolution and exact current/emergency evidence. The handoff reuses the recovery and target-state identities, fences the old principal from commit, and appends bounded audit evidence. The run review reports the database-derived lease_expires_at instant and a clamped lease_remaining_seconds value from 0 through 900, and the dashboard shows both before an operator confirms a takeover. Completion revokes the runner session and apply authorization, advances the fence under a dedicated recovery reservation, optionally publishes the emergency snapshot, releases the reservation, and writes run.recovered and state.recovered audit evidence in the state-commit transaction. It never retries, resets, or replaces tofu apply. The recovered run remains historical needs_recovery evidence; the next infrastructure action is a new plan.

Related pages

• Runs
• Run approvals
• Policy sets
• Architecture