Skip to main content

Health

Test drives and golden cases tell you how an Agent behaves on the cases you thought of. Health tells you how it behaves on the ones you didn't.

Open Health from the Agent page's sidebar. Every number here is counted over the Agent's settled turns in production - not sampled, not estimated - and every rate is printed beside the population it is over.

The Agent Health page: total turns, succeeded, failed, failure rate and p50/p95, then Grounding & repairs with the repair counts, the Across revisions table, and the answer-source bucketsThe Agent Health page: total turns, succeeded, failed, failure rate and p50/p95, then Grounding & repairs with the repair counts, the Across revisions table, and the answer-source buckets

Where the answers came from

Each settled turn is filed under exactly one bucket, so you can see at a glance whether the Agent is doing the job you gave it:

BucketMeaning
KnowledgeAnswered from your uploaded documents or bound organization knowledge.
ToolsAnswered from a live tool call - a ticket, a row, an API result.
General knowledge (disclosed)The library had nothing; the model answered from what it knows, and said so.
Said it didn't knowThe Agent declined rather than guessing.
UnsourcedProse with no retrieval and no tool call behind it.

Two of these are the ones to watch. A rising General knowledge (disclosed) is the strongest signal you have that your knowledge base has a hole - and the Knowledge gaps list turns each one into a draft article. Said it didn't know is a good outcome, not a failure: it is the Agent declining to invent an answer.

Grounding & repairs

Some turns get corrected before they ever reach the reader. When an Agent is about to answer without consulting knowledge it was told to search, or cites a document the turn never retrieved, or returns a shape that does not match its declared output schema, the engine nudges it once and the corrected answer is what ships.

The panel counts those repairs by verifier:

RepairWhat it catches
Searched firstThe Agent answered without searching its knowledge, and was asked to search before replying.
Said what it did not knowThe knowledge had no coverage and the answer did not say so, or used general knowledge without the disclosure.
Cited what it readThe answer named a knowledge file this turn never retrieved, and was asked to search for it or drop the citation.
Matched its output schemaThe answer did not satisfy the JSON Schema this Agent declares, and was asked again.

Coverage is stated before any rate, and the denominator is the grounded turn count rather than the total. A cancelled turn, or one that died before a model answered, never had the opportunity to be repaired - dividing by everything would dilute the rate with turns that were never at risk.

Every verifier and every bucket is rendered, including the ones at zero. A verifier that has never fired is information.

Better or worse, by revision

The same counts, cut by the Setup revision each turn actually ran. This is the view that answers "did the last publish help", and the flat numbers above cannot: a window spanning a publish averages the fix in with the bug it fixed.

Column
RevisionThe published Setup the turn ran on
Grounded turnsThe population for this revision
RepairedTurns the engine corrected mid-flight
UnsourcedAnswers with nothing behind them

Each rate is printed beside its own denominator. A revision that handled four turns and one that handled four thousand produce percentages that look alike and mean nothing alike.

Using it

Publish, then come back after real traffic. If repaired falls and from knowledge rises across a revision boundary, the change worked. If unsourced climbs, the Agent is reaching for answers it does not have - add knowledge, tighten the Procedure, or add a learned rule.

To make that loop automatic, turn on the eval gate so a Setup cannot be published until your golden cases pass.