Durable Execution (for AI Agents)

Early LLM agent prototypes ran as fragile in-memory `while` loops inside a single HTTP request handler. If an external API rate-limited on step 7 of a 10-step research task, or if the agent needed to pause for 12 hours awaiting a manager’s Slack approval before executing a refund, the entire state was lost or required re-executing all prior LLM reasoning steps. Starting in 2024, engineering teams combined event-sourced durable execution engines (Temporal, Restate, DBOS, Inngest) and graph-level checkpointers (LangGraph PostgresSaver) with agent loops, wrapping non-deterministic LLM generations and external tool calls as journaled activities.

Sources: s1, s2, s3

LLM calls are slow, expensive, and stochastic, while external tool invocations (charging a credit card, provisioning cloud infrastructure, sending an email) produce real-world side effects that must not be duplicated on retry. Durable execution separates the deterministic orchestration journal from the non-deterministic LLM/tool activity, ensuring that a worker restart replays completed steps from the log in milliseconds and resumes execution at the exact pending tool call.

Sources: s1, s2

An incident-response agent wrapped in a Temporal workflow or LangGraph checkpointer diagnoses a production alert across four log queries, proposes a database migration, and suspends execution via a durable signal/interrupt while freeing all GPU and CPU compute. Six hours later, when the on-call engineer clicks "Approve" in Slack, a new worker hydrates the checkpointed conversation history and tool outputs without re-invoking the four prior LLM steps and executes the migration.

Sources: s1, s3

Deterministic replay requires strict separation between pure workflow orchestration code and side-effecting activities; mutating workflow code while long-running agent instances are still in flight can cause replay non-determinism errors unless explicit versioning or state-graph checkpointing is used.

Sources: s2, s3