Execution model

Exactly what runs, and what it costs, per invocation

graph has two engines: the plan pipeline (the core) and an agent loop layered over it. Knowing which runs when explains every trace and every token bill.

The plan pipeline

Plans execute as: validate → run steps sequentially → finish.

• Step inputs render against prior results via the template language — typed dataflow, zero inference.
• Seven control steps are intercepted by the executor, never dispatched to a tool: exit ends the plan early, decide forks into one of two branches, filter partitions a list with a per-item gate, map/reduce run a body once per item of a list, agent runs a bounded tool-calling loop that returns schema-conforming JSON, and ask puts a question to the person running the plan. agent is the one control step that costs inference per round; agent, ask, and filter are the three legal inside another control step’s body (a nested filter’s {{item}}/{{index}} shadow the enclosing body’s within its gate). All defer rendering: a decide renders only its gate up front and the non-taken branch never renders; filter renders only over up front and its gate per item; map/reduce render only over up front and the body per item. Gate evaluations and body calls appear as ordinary flat tool events in traces. map is the pipeline’s one concurrency point — its concurrency knob runs independent items in parallel; everything else is sequential.
• The finish is a solver LLM call, an output render (no LLM), or nothing (finish modes).
• For plan_and_execute, a planner-role call authors the plan first, and defects loop back as replans (executed steps preserved). Human-authored plans skip planning and never replan.
• Interactive callers can install an execution gate — a hook consulted before every real tool dispatch (registry tools, plan__*, plan_and_execute, and body calls at any nesting depth) that can proceed, skip the call with an injected result, or abort the run. The gate sees the call’s fully rendered input plus the template scope it rendered against (the results map at the top level; the layered body scope with item/index/accumulator inside bodies). A second hook, on_tool_error, is consulted when a dispatched call fails: it can let the error propagate (the default — identical to ungated behavior), replace the error with a substitute value the run continues with (never entering the replan loop), or abort. Event ordering: tool_finished always reports the real call; step_finished reports the resolution. Aborts are hard stops: no replan, no solver, no error summary — the partial run state comes back, and when the abort was triggered by a failing tool the failing tool’s error rides along with it (so interactive callers can show why the step failed), and nested-plan aborts propagate without being re-asked. Control-step evaluation (exit/decide gates, filter verdicts, map/reduce orchestration) is never gated. The plan workbench’s debug runs are built on this.
• The gate is the out-of-band half of interactivity: an outside observer interrupting a run it did not plan. The in-band half is an interlocutor — a hook the ask step uses to put a question to a human and bind the answer to a step id. Hosts implement it differently (a terminal prompt on stderr, the workbench’s answer editor, an MCP elicitation/create request back to the client) and a host that cannot reach anyone simply installs none. Because whether a human is reachable is a property of the host and not of the plan, each ask step declares its own unattended behaviour (when_unanswered), which is what lets one plan run interactively and in CI. Like every control step, an ask is never gated — it makes no tool call — and questions are serialized even under concurrent map items.
• Sinks observe runs through result-carrying step events: every step (and body call) reports its rendered input when it starts and its full result value when it finishes, addressed by a step path (E3, E3/then, E3/do.2/E10) plus the plan call stack.

The agent loop

ask and chat run a tool-calling loop with the chat-role model:

message + tool catalog → model
  ├─ tool calls? → execute (parallel within a round) → results → model again
  └─ text answer → done

Bounded by max_agent_iterations. Tool failures return into the loop as error results — the agent explains or works around them visibly.

The anatomy of a chat turn using a plan

1. chat model    — decides to call plan__project_status        (inference #1)
   ├─ E0..E3     — four Linear tool calls                      (no inference)
   └─ solver     — writes the report from collected data       (inference #2)
2. chat model    — reads the report, answers you               (inference #3)

The solver’s report streams dim to stderr as progress; the agent’s answer is the authoritative stdout. Yes, calls #2 and #3 partially overlap in purpose — the agent layer adds condensing, conversation, and follow-ups. When you don’t want that layer, plan run invokes the pipeline directly: one inference (solver) or zero (output/silent).

Cost table

Invocation	Inference calls
plan run (output / silent)	0
plan run (solver)	1
exit / decide gate	0 (logical gate) or 1 judge call (infer)
map / reduce	0 — plus whatever the body calls
agent	1 per round (up to max_iterations) + its tool calls
ask → direct tools	1 + one per tool round
ask → plan tool	3
ask → plan_and_execute	4 + 1 per replan

Model roles

Every inference site resolves through [models] role assignment (chat, planner, solver, repair, judge — falling back to default), so cost tuning is pure config: strong model where judgment lives, fast model where volume lives. Named models extend the fixed roles with user-defined entries selectable at the point of use — per-step model routing without touching the role assignments. Structured outputs (planner, prompt tools) get one automatic repair-role fix-up attempt before erroring into the replan loop. The full map of roles, providers, and failover is Models & providers.

Learning across turns

Every successful tool call feeds the shape cache; the planner reads it fresh at each planning attempt. The system genuinely gets better at planning the more it’s used — including within a single run.