For agencies and teams who ship AI support agents to clients
Novera runs a versioned scenario suite against an agent you operate, checks what it claimed against what actually happened, and seals a dated report that says what was tested, what was judged, what was verified — and what remains unknown.
No card. About ten minutes to a first report. Three free runs, then a plan for recurring testing.
Illustrative dataNot a real agent — what a client receives
Grade
48 of 49 scenarios passed
Every scenario has a verdict. Meets the pass mark set for this evaluation.
Every scenario
Ran 49 of 49 · verdicts 49 of 49 · actions checked 4 of 4
Illustrative digest · covers every verdict and its evidence
Open the finding for its evidence chain, or compare it with the last run.
Six stages, in order, for every scenario. Each is stored, so a report shows which one decided the verdict.
A fixed input from a versioned suite — ordinary, tricky or an attack — over its declared channel.
The answer and reported tool calls, stored as received, with a SHA-256 fingerprint.
Deterministic checks first. They can fail a scenario, never pass one.
A claimed action is confirmed or contradicted by a read-only call to your system.
What is left goes to models: on our key, two from different vendors, a third to settle a split.
Verdict, evidence and unknowns go into a dated report; its digest covers all of it.
A prompt changed on Tuesday and the client noticed first. “We think so” is not an answer to “is it still behaving?”
What teams usually have
Nothing here can be handed to a client, compared with last month, or shown to an auditor.
What Novera produces
Forty-nine scenarios, every time
Identity, erasure, refund limits, escalation, jailbreaks, injected instructions. A suite version never changes, so reports compare.
Stored evidence for every verdict
Reply, tool calls, read-back and every vote — append-only, so a failure cannot be deleted to flatter a score.
Unknowns named, not hidden
No reply, an unreadable grader or an unchecked action is its own state — listed, never counted as a pass.
A sealed report with a revocable link
Hashed to its run and your policy version; the link expires and can be revoked. No policy text, keys or raw transcript.
Enforced in the database, not promised in a brochure. Try each one: the data is illustrative, the behaviour is the product's.
01
Saving a change creates a new version. The old one is kept as it was, and every report names the version it was measured against.
Version 4 adds one line. Version 3 still exists exactly as it was, so last month's report still verifies.
02
On our key, two models from different vendors grade each scenario. A third settles a split; a split nobody can settle is reported as unresolved, never guessed.
Failed — split, settled by a third vendor.
03
When the agent says it acted, a read-only call to your system of record confirms or contradicts it — and a contradiction is settled without asking a model.
The agent said
“Your refund for NW-4417 has been processed.”
Your billing system
GET /invoices/NW-4417
status: "open"
Contradicted — failed, no model asked.
04
The report carries a SHA-256 digest of its own contents. Change one figure and the digest no longer matches.
sealed sha256:computed in your browser
now sha256:computed in your browser
Digest matches the sealed run.
05
A timeout, an unreadable grader or an action nobody could check produces no verdict. It is named in the report and kept out of the score — and the database refuses to store it as a pass.
The score counts
41 passed of 46 with a verdict
Named, never counted · 3
The report shows these three as loudly as a grade, because a quiet footnote is how a partial run passes for a clean one.
An HTTP endpoint you operate, the rules it should keep, and a run. Choose a step to see its screen.
Step 1 of 4 · Connect the agent
IllustrativeEndpoint
Connection check answered · 200 in 420 ms
reply: "Hi! How can I help with your order today?"
Saved as a receipt, so you can see what Novera is talking to before a full run.
A failure has a path, and every step that changes something waits for a person.
Failed scenario
Its evidence chain shows which stage decided it.
Review queue
Grouped with everything else that needs a person.
Ask why
A proposed policy change, quoting the passage it would replace.
Approve
Becomes a new policy version. Nothing changes on its own.
Waits for a personRetest
The one scenario again, against the new version.
Rerun and compare
Fixed, still failing, newly broken — and why each moved.
Reissue
A new sealed report that names the old one by hash.
Waits for a personGate a release, rerun on a schedule, or start a run from n8n. A missing verdict fails the gate like a failed one.
Release gate — A REST call starts a run; five exit codes tell a pass from a failure from missing evidence.
Schedules — A pinned suite at a fixed UTC time; a schedule that cannot start pauses and says why.
n8n and webhooks — Signed webhooks and four importable n8n workflows, each run in n8n before it shipped.
Production failures — Send a real failure in; it comes back as a draft scenario a person approves.
auth="Authorization: Bearer $NOVERA_KEY"
run=$(curl -fsS -X POST "$api/runs" -H "$auth" \
-H "Idempotency-Key: $GITHUB_SHA" -H 'content-type: application/json' \
-d '{"agent_id":"…","suite_id":"…"}' | jq -r .run.id)
# advance it (POST $api/runs/$run/execute) until done, then:
case $(curl -fsS "$api/runs/$run" -H "$auth" | jq -r .run.outcome) in
pass) exit 0 ;;
fail) exit 1 ;; # a scenario failed
*) exit 2 ;; # missing evidence stops the release too
esacThe limits, before you find them. Each is also in every report's limitations block.
Your agent
Your infrastructure
Novera
Database in Frankfurt (EU)
Grading providers
May be outside the EU
Database residency and model processing are two different questions. What goes where, exactly.
Recorded
Observed by Novera during the run
Declared
Stated by you, and labelled as yours
Not verified
Outside what a run can show
A document that overstates itself is worth less — especially to the person you hand it to.
Not a certification. Novera is evidence of testing.
Not legal advice. It does not say which obligations apply to your organisation.
One dated run. One suite version against one policy version.
Only the suite. Not your agent's behaviour on every conversation, nor anything outside it.
Three free runs, on our key or yours. Then a plan: Team BYOK grades on your own model key; Agency Pro adds runs graded on ours.
An agency, or several clients' agents? “Talk to us first” reaches a person.