Is Your AI Agent Ready for Production?
A checked box is not evidence. Mark each control verified only when you can link a fixture, configuration, log, alert receipt, rollback run, or named approval that proves it.
What qualifies as evidence
| Control | Strong evidence | Weak evidence |
|---|---|---|
| Permission | Denied-action test and scoped credential | Prompt says “be careful” |
| Secrets | Secret scan plus redacted log fixture | No secret appeared in one demo |
| Evaluation | Versioned input, expected trajectory, assertion, result | A human liked the output |
| Idempotency | Same event delivered twice creates one effect | Retries are “unlikely” |
| Budget | Enforced timeout/retry/spend cap with failure test | Cost estimate in a document |
| Alerting | Test incident reached the named owner | Notification integration configured |
| Rollback | Rehearsed rollback or compensating action | Backup exists but was never restored |
| Kill switch | Active run stops and cannot restart itself | Pause file exists |
Critical blockers override the score
The calculator blocks a release when a critical control is failed or unverified. This is intentional. A high average cannot cancel an exposed credential, unbounded action, broken refusal, duplicate charge, missing rollback, or untested kill switch.
This conservative gate does not provide a “not applicable” shortcut. If a control truly does not apply, document the reason in the release record and keep it unverified here; the release owner must make and preserve that exception decision outside the calculator.
Release in authority stages
- Offline fixture only.
- Read-only production observation.
- Shadow recommendation beside the human process.
- Internal reversible write with approval.
- Bounded autonomous action with monitoring.
- Broader authority only after the earlier stage produces stable evidence.
Regrade after model, prompt, tool, dependency, schema, permission, or business-policy changes. Production readiness is a release state, not a badge the system keeps forever.