How to use Novera
Seven steps from a connected agent to a report you can hand to a client. The first run usually takes about ten minutes, most of it writing the policy.
Connect the agent you want to test
Give Novera the web address your support agent answers on, and confirm you own it or are authorised to test it. Novera sends it one harmless test message straight away, so you know the connection works before anything else.
Where: Connect an agent
Write the policy it should follow
Paste the rules your agent is meant to keep: refunds, identity checks, what it may and may not say. Every verdict is judged against this text. Saving a change creates a new version; old versions are kept, so every report says exactly which rules it was measured against.
Where: The agent's page → Policy
Run the suite
A suite is a fixed set of scenarios — ordinary requests, tricky ones and deliberate attacks. Novera sends each one to your agent and grades the answer. The newest suite version is picked for you. A run takes a few minutes and keeps going even if some scenarios fail.
Where: The agent's page → Run the suite
Read the results
Each scenario ends as passed, failed, or without a result. Without a result is never counted as a pass: it means the agent errored, the grading models could not agree, or an action the agent claimed could not be checked. The score is passes out of the scenarios that got a verdict, and the report says how many did not.
Where: The run page
Fix, then rerun and compare
On a failed scenario you can ask for a diagnosis: Novera proposes a change to your policy, quoting the exact passage. You approve or reject it — nothing changes on its own. Then rerun: the new run is compared with the old one, scenario by scenario, and flags any scenario that flips between runs on its own. To rerun on a calendar instead, add a schedule on the agent's page.
Where: The run page → a failed scenario
Disagree with a verdict, if you do
Record your own finding on any scenario, with a reason. It sits beside the automated verdict and never changes the score. If a client should see it, issue a new report that discloses it.
Where: The run page → a scenario → Record your own finding
Share the report
Every finished run produces a dated report with a private link. Its fingerprint (a SHA-256 hash) proves nothing was edited after it was issued. Send the link, print it to PDF, export it as Markdown, CSV, JSON (to verify a copy) or JUnit (for a CI pipeline), or withdraw it at any time.
Where: The run page → Client report → View client report
Words you will see
- Scenario
- One test: a message sent to your agent, what it should do, and what would count as failing.
- Suite
- A numbered set of scenarios. A published version never changes, so two reports on the same version are comparable.
- Policy
- Your written rules for the agent. Verdicts are judged against the version that was current when the run started.
- Verdict
- Passed or failed. A rule in the scenario can fail it on its own; otherwise two grading models from different companies decide, and a third settles any disagreement.
- No result
- The agent errored, the models could not agree, or a claimed action could not be checked. Counted separately — never as a pass.
- Assurance gap
- The share of the suite that produced no verdict. The larger it is, the less the score says.
- Read-back
- An optional read-only address in your own system that Novera checks to confirm an action the agent says it took, such as a refund.
- INCOMPLETE / WITHHELD
- INCOMPLETE: some scenarios never ran — run again. WITHHELD: everything ran but the evidence does not support a grade — fix what made it unusable.
- Report hash
- A fingerprint of the report. Anyone can recompute it; if one character changed, it would not match.
More detail in the documentation. Novera is evidence of testing, not a legal certification.