Skip to main content

Let the Agent learn from its runs

An Agent's Procedure is what you wrote for it. Learned rules are the short corrections that accumulate afterwards - "when a customer gives a date with no year, ask which year rather than assuming" - the things you would otherwise say again in every review.

They live in the Learned rules tab of the Agent's Knowledge drawer, beside Files and Knowledge. Every rule is one line the Agent follows on every run, alongside its Procedure, and you decide every one of them: nothing is added without a person accepting it.

The Knowledge drawer's Learned rules tab: the Learning setting, three rules each labelled Written by hand, Taught from a run or Proposed and accepted, and the Reflect now controlThe Knowledge drawer's Learned rules tab: the Learning setting, three rules each labelled Written by hand, Taught from a run or Proposed and accepted, and the Reflect now control

What a rule costs you

Rules are live. Like knowledge files, they apply to all versions of the Agent - published ones included - and adding or removing one changes the next run. There is no draft, no publish step, and no publish diff to catch a change before it ships. Removing a rule cannot be undone: no revision holds the old text.

That is the same bargain the Files tab makes, and it is deliberate. A rule that needed publishing would sit in a draft while the mistake it corrects kept happening.

Two limits bound the set: 40 rules per Agent, 400 characters each. Both exist so the rules stay a short list a person can actually read - an Agent that has learned two hundred things has learned nothing you can check.

Writing one by hand

Add rule in the Learned rules tab. Keep it to a condition and an action: what situation the Agent should recognise, and what it should do. A rule with no trigger condition is indistinguishable from a preference and will fire everywhere.

Each rule shows where it came from - Written by hand, Taught from a run, or Proposed and accepted.

Teaching from a reply

Open a conversation, find the reply that went wrong, and press Teach the agent beside the feedback buttons. The dialog seeds a rule and links it back to that run, so the rule carries its evidence: anyone reading the list later can open the conversation it came from.

This is the cheapest path, and usually the best one. The person who noticed the problem is the person who can say what should have happened.

Collecting rules automatically

The Learning control at the top of the tab has three settings. All of them govern collection only - existing rules keep applying whatever it is set to, and removing a rule is what stops it applying.

SettingWhat it does
OffNothing new is collected.
SuggestRuns that go badly are collected, and proposed changes wait for you to accept them.
AutomaticA proposal that passes the golden-case gate is applied without waiting for you. Rejected ones are kept for review.

Automatic is unavailable until the Agent has five golden cases to replay a proposal against. It stays visible and greyed out with the Agent's own count beside it, because a gate over one or two cases is a coin toss wearing a verdict's clothes - the floor is what makes the setting safe, not a formality.

With collection on, the tab shows three things.

What this agent could learn from

Runs from the last 30 days that ended badly enough to be worth reading - errors, unresolved conversations, replies people marked down. Each links to the run. This is a reading list, not a proposal; nothing here has changed the Agent.

Proposals

Reflect now reads those flagged runs and proposes up to five changes: add a rule, reword one, or retire one that is no longer earning its place. Each proposed change cites the runs it came from, so you can check the evidence before accepting.

A reflection that finds no pattern says so. "No pattern worth a rule" is a real answer - the Agent's bad runs were not all the same problem - and nothing is proposed.

Accept applies the changes to the live rules. Reject asks why, and the reason is kept with the proposal.

The test before you accept

If the Agent has golden cases, press Test first and the proposal is replayed against them with the proposed rules applied. The result reads as a comparison - Gate passed - 92% vs 88% now - so you are looking at the change, not at an absolute score.

A proposal that has not been tested says untested rather than claiming a pass, and untested never blocks Accept: with no cases to replay, the suite has nothing to say about the change, and calling that a pass would be a verdict about nothing. Only a measured drop marks the proposal as failed.

Knowledge gaps

Separately from rules, the tab lists answers people were happy with that did not come from the knowledge base - the Agent got by on a tool call or on general knowledge where the library was silent. Those are the places the library demonstrably should have covered something.

Add to knowledge drafts an article from the answer that actually worked, copied as it was written. It is always a Draft: nothing reaches a reader of the knowledge base until a person publishes it. The draft is filed under a Learned base and remembers which reply it came from, so the same answer cannot be captured twice.

What never changes on its own

Learning writes to the rules list and nowhere else. An Agent's Principles and Procedure are yours: no proposal can reach them, no reflection can reword them, and nothing you wrote is rewritten in the course of the Agent learning something.

Tips

If a rule is not taking effect, check it is still in the list - removing it is the only thing that stops it, and turning collection off does not. If proposals never appear, the Agent may have no flagged runs in the window; the What this agent could learn from section says so directly. See Troubleshooting.