Stateful workflow orchestration

Stateful workflow orchestration records a process's progress, so it can wait for an event and continue after an interruption. For each instance, it needs to know what has been confirmed, what comes next and what it is waiting for. Durable progress alone does not make a call to another system safe to repeat.

What the workflow remembers

Consider a purchase request that needs approval before a purchasing system creates an order. The workflow might retain the request ID, its current stage, a deadline and the resulting order ID. While approval is pending, it need not keep a worker running. When a decision arrives, the instance can continue from its recorded progress.

The purchasing application remains the authority for the request's amount, supplier and business status. The orchestrator stores the coordination facts it needs to find that record and decide its next action. Give states specific meanings, such as awaiting_approval, approved, order_requested and completed, with separate outcomes for rejection, expiry and an uncertain order submission.

When durable state helps

Use durable state when continuation depends on a previous step that cannot reliably be reconstructed after a worker stops. Human decisions, callbacks, timers and ordered actions across services are common reasons. An immediate, independently retryable operation may need only a queue consumer and a reliable destination record.

At each interruption point, ask what a replacement worker would need to know, whether an outbound action might already have succeeded, and which system owns the business fact involved. Persist the answers that cannot be recovered safely.

Waiting and external effects

A waiting instance needs an expected event and, where appropriate, a deadline. Correlate each response with the right instance and decision. Define what to do with a repeated response, a late response or a business record that changed during the wait.

Durable runtimes preserve progress in different ways. Temporal uses recorded history and replay to rebuild orchestration state. Microsoft's Durable Task model records execution history and checkpoints progress when an orchestrator calls await or yield.

In Temporal, a recorded completed activity result can be returned during replay without running that completed activity again. If an Activity attempt fails, it is automatically retried using its Retry Policy, so a downstream write needs a way to identify, check or safely repeat it.

Make the waiting state explicit in the orchestration code as well as in its business meaning. In Durable Functions, an orchestrator can save called-function output in a local variable and checkpoint when it awaits or yields, allowing that local state to survive process recycling or a virtual-machine reboot.

What durable storage contains

A durable runtime may persist more than the current stage. In Durable Functions, the task hub stores instance state and pending messages; orchestration state can include runtime status, history, inputs, outputs and custom status. Messages can contain function inputs or outputs, event payloads and routing or correlation metadata.

Messages are deleted after processing, but instance state remains unless the application or an operator explicitly deletes it. Orchestration history can remain in storage after an instance completes, so decide how retention and privacy policies apply to the information recorded there. The amount and frequency of persisted data can also affect performance and storage transaction costs.

Keep persisted inputs and outputs small where practical. If a step handles a large payload or sensitive information, choose deliberately what belongs in durable state and what should remain in the owning application; the runtime's ability to serialise data does not determine your retention policy.

Events that resume an orchestration

In Temporal, a Workflow Task is scheduled when an execution starts, a Signal or Update arrives, an Activity completes, a Timer fires or a Child Workflow completes.

An orchestration instance can last seconds, days or months, or be configured not to end.

Checkpoints inside long-running work

A workflow's progress and an activity's internal progress are separate boundaries. In Temporal, an Activity attempt normally starts from its initial state; when the activity uses heartbeat details for checkpointing, the next attempt can receive the last recorded details and continue from that point.

Give each instance a stable identity

An orchestration instance needs an identifier that lets incoming events and operators refer to the intended execution. Durable Task uses an instance ID that is unique within a task hub; by default it generates a globally unique identifier, but an application can supply a string instead.

For Durable Task, an instance ID must be between 1 and 100 characters, must not start with @, and must not contain /, \ or a double quote. Treat identity as part of the orchestration boundary: define how a business request maps to an instance, and use the same mapping when locating that instance for a later event.

Orchestration Instance Identity Requirements (Azure Durable Task)

Recovery and data ownership

For each stage, document the expected next event, the last confirmed external effect and the recovery action. A timeout after a write is an unknown outcome until the destination confirms what happened. Record a downstream result ID when one is available.

Keep full business records in their owning applications. Store references, decisions and progress markers in orchestration state. If continuation requires current facts, read the business record again. If an approval applies to a fixed version, retain which version was reviewed and check it before acting.

A durable history is useful for reconstructing coordination state, but it is not automatically a suitable archive for every input or result. In Durable Functions, persisted parameters, return values and other state contribute to task-hub storage.