Evals
Evals are enabled per workspace. If the Evals section is not in your sidebar and your agents have no Evals panel, your workspace does not have it yet - ask Yekar.AI to switch it on.
Evals are how you find out whether a change to an agent helped, before a customer does. You keep a set of cases, run them against a Setup, and get a verdict rather than an impression.
Domain members can inspect evaluation reports. Creating or editing cases and assertions, starting or cancelling evaluations, and changing the publish gate require Editor or Owner access. Existing evaluation replay transcripts are read-only. See Roles and permissions.
Cases
Open Setup → Evals. A conversation you were happy with becomes a golden case - a regression test you replay against a changed Setup before publishing.
The drawer has three tabs: Cases (the saved conversations), Replays (individual runs against a Setup), and Suites (a whole set replayed together).
Replays are deterministic. Tool calls are answered from the recording rather than executed, so a replay never sends an email twice, and a half-consumed recording fails the replay rather than quietly reporting a pass.
A case keeps its own copy of the recording, taken when you saved it. Two things follow. It still replays correctly months later, after the original conversation's tool calls have aged out of the 90-day call history. And deleting the conversation does not delete the case - the case keeps working, and says on its row that the conversation it came from is gone. If you need the conversation's content removed everywhere, delete the case as well.
This is what makes an agent safe to keep improving. Save the conversations that went right, replay them after every edit, and you learn whether your change helped before a customer does.
Assertions turn a diff into a verdict
A replay on its own gives you a diff - what the Agent did this time versus what it did in the recording - and a human has to read it. Add assertions to a case and the replay produces a verdict instead.
Each assertion is checked against the run, and the same scorer runs at write time against the golden recording itself: an assertion that fails its own recording is refused as a typo when you save it, not hours later against a real change.
A case with no assertions scores unscored, and every surface renders that as its own state. Unscored is not a pass - it means nothing was asked of this case.
Suites
Run every case together and you get one result for the Setup rather than a pile of individual replays. Tallies are aggregated from the member runs, so a suite can never disagree with the runs inside it.
Gate publishing on your suite
Turn on the eval gate and an Agent refuses to publish unless a suite tested that exact draft and at least one case passed with none failed.
The match is exact: a suite run against a Setup you have since edited does not count, because it did not test what you are about to ship. The one-passed floor is "unscored is not a pass" enforced where it bites - a suite of entirely unscored cases cannot wave a publish through.
The gate is opt-in per agent, by design. A mandatory gate on a young suite teaches people to delete assertions.
What a run costs
Eval runs spend real money - they call the model, and a written case calls your real tools. That spend is billed like any other, and it is deliberately kept out of your usage statistics and cost reports, which answer what your automations did rather than what you were testing. A suite reports its own cost, so the two never have to be untangled.