AI agent runtime

A framework is not a runtime

Most agent projects start with a framework: a library that gives you a loop, a way to declare tools, and a prompt template. That is enough to get a convincing demo, and it is the right place to start.

The gap opens the first time the agent has to run without you watching. Something has to trigger it at 3am. Something has to retry the tool call that timed out. Something has to remember where the run was when a reviewer went home for the night. Something has to be able to tell you, a week later, which model call produced the answer a customer is disputing, and what it cost.

None of that is agent logic. It is runtime work, and it is the majority of the work. Teams that skip it end up assembling it anyway from a queue, a tracing vendor, a spreadsheet, and a chat channel, and then discovering that no single one of those can answer a question that crosses two of them.

The five things a runtime has to hold at once

These are not features on a list. They are five properties that have to hold simultaneously for an agent to be trusted with production work, and the reason a runtime is a distinct kind of system is that holding them separately does not add up to holding them together.

Build

The agent, its tools, its retrieval, and its branching are modelled as one graph. The graph is both the specification and the thing that executes, so what somebody designed and what runs in production cannot drift apart.

Execute

Independent branches run in parallel, failures retry, sub-workflows compose, and capacity grows by adding instances rather than by rewriting the agent. Scheduling, webhooks, and API calls are all entry points into the same execution model.

Govern

Some actions run unattended and some stop for a person. Approvals, content guardrails, credential scoping, and team permissions sit on the execution path itself, where they can actually block a call, rather than in a policy document beside it.

Observe

Every model call and tool call leaves a record of what it received, what it returned, how long it took, and what it cost, and thresholds on those numbers raise an alert on their own instead of waiting for somebody to open a chart.

Evaluate

A prompt or model change is scored against a repeatable suite before it ships, so the decision to change a model rests on a measurement rather than on the last output somebody happened to read.

How Heym implements each one

Heym is an AI agent runtime built as a visual canvas over an execution engine. An agent is a node on that canvas alongside the retrieval, branching, integrations, and human review steps it depends on, so the five concerns above are properties of one system rather than of five.

Agents that call real tools

The Agent node runs a tool-calling loop over inline Python tools, MCP servers, skills dropped onto the node, and other Heym workflows invoked as tools. An orchestrator agent can delegate to named sub-agents on the same canvas, up to five levels deep, and parallel delegations run concurrently.

Parallel execution by default

The executor schedules the graph as a DAG. Nodes at the same dependency level dispatch concurrently, downstream nodes start as soon as their inputs land, and multiple runs execute in isolation from each other.

Pauses that survive the night

An agent can call for human review mid-run. The execution freezes its full state, including conversation history, variables, and tool results, and a one-time review link lets a reviewer accept, edit and continue, or refuse. The resumed run continues the same execution rather than starting a second one.

A record of every call

Traces hold the request and response payloads, per-call timing, tool arguments and results, and the skills passed to the model, with input and output token counts costed in USD from a pricing table you can edit.

Thresholds that speak first

Alerts watch error count, run duration, token or dollar spend, and execution count over a time window you choose, backtest the condition before it saves, and fire once until the metric recovers.

Evaluation before release

Eval suites attach to a workflow, run test cases across several models at once, and score with exact match, contains, or a judge model on a separate credential so scoring stays independent of the model under test.

Questions worth asking before you commit

These apply to any candidate, Heym included. They are the questions whose answers are hard to change later.

1. 1

   Can a run pause and resume as the same execution?

   Ask what happens to conversation state during a human approval. A system that re-invokes the agent with the reviewer’s answer appended is starting a new run, which means the tool results and variables from before the pause are gone.

2. 2

   Is the cost figure per run, in money?

   Credits and task counts cannot answer "which workflow change doubled our spend". Look for input and output tokens resolved to USD per execution, against a price table you can correct.

3. 3

   Does the trace show the payload the model actually received?

   Intermediate step summaries are not enough to debug a bad answer. You need the assembled prompt, the tool arguments, and the raw response.

4. 4

   Where does policy live?

   Guardrails and approvals that sit outside the execution path can be bypassed by any code path that forgets to call them. On the path, they block the call.

5. 5

   What happens when volume grows?

   Check whether capacity is added by running more instances that share the same queue, and whether work that must stay on one machine is placed there deliberately rather than by accident.

6. 6

   Can you run it yourself?

   For agents that read internal documents or hold production credentials, the deployment model is usually the first constraint, not the last.