Skip to main content

Free tool

AI agent production readiness score

This AI agent production readiness score asks twelve questions about evaluation, permissions, cost limits, and operations, then names the three gaps to close first. It is a self-check of practices, not a review of your code.

No sign-up. Runs in your browser. Last reviewed by the Builderz team.

0 of 12 answered

Evaluation

Does a written evaluation set with expected outcomes exist, and does it run before every prompt or model change?

Does a written evaluation set with expected outcomes exist, and does it run before every prompt or model change?

Without a fixed set of cases, every prompt edit or model swap is a guess about what got better and what broke.

Are failure cases from production added to the evaluation set?

Are failure cases from production added to the evaluation set?

Production failures are the cases your first set did not think of. If they are not saved, they come back.

Does a regression gate block a release when the evaluation score drops?

Does a regression gate block a release when the evaluation score drops?

A score that nobody is forced to look at gets ignored the week a deadline lands.

Permissions and tools

Does every tool have a declared permission boundary that the agent cannot exceed?

Does every tool have a declared permission boundary that the agent cannot exceed?

If a tool can do more than the task needs, a bad prompt or an injected instruction can use all of it.

Do actions that move money, send messages, or delete data require a confirmation or an allowlist?

Do actions that move money, send messages, or delete data require a confirmation or an allowlist?

These actions are hard or impossible to undo. One wrong tool call should not be able to finish them alone.

Are credentials scoped per tool and kept out of the prompt?

Are credentials scoped per tool and kept out of the prompt?

A secret in the prompt can be repeated back or logged. One shared key gives every tool the reach of the strongest one.

Cost and limits

Does a per-task token or dollar cap stop runaway loops?

Does a per-task token or dollar cap stop runaway loops?

An agent that loops on a failing tool call can spend a month of budget in an hour.

Is there a monthly budget alert with a named owner?

Is there a monthly budget alert with a named owner?

A spend alert with no owner is a notification nobody acts on.

Are the model and effort level chosen per step, not one setting for everything?

Are the model and effort level chosen per step, not one setting for everything?

Routing and extraction steps rarely need the same model as planning steps. One setting overpays or underperforms.

Operations

Does every run have a trace with inputs, tool calls, outputs, and cost?

Does every run have a trace with inputs, tool calls, outputs, and cost?

When a user reports a bad answer, the trace is the only way to see what the agent did and why.

Can a kill switch pause the agent without a deploy?

Can a kill switch pause the agent without a deploy?

If stopping the agent needs a code change and a release, the incident runs longer than it has to.

Does an incident runbook name who is paged and what gets rolled back?

Does an incident runbook name who is paged and what gets rolled back?

During an incident people guess. A written page list and rollback list shorten the time bad output reaches users.

Answer all 12 questions to see your readiness result. It updates as you change an answer.

How it works

What the twelve questions cover

Three questions each on evaluation, permissions and tools, cost and limits, and operations. They ask whether a practice exists, such as an evaluation set that runs before each change, a confirmation step before irreversible actions, and a per-task cap.

How weights and the grade work

Each question has a weight from 1 to 3. The evaluation set, the confirmation gate, and the per-task cap weigh 3. Yes earns the full weight, Partly earns half, No earns nothing. Not sure also earns nothing and is counted separately. The score is points earned over points possible, mapped to one of four plain labels.

What the tool does not do

No code is read and nothing leaves your browser. Answers are saved in local storage on this device so you can come back. The tool cannot tell whether an answer is true, and the grade is not a security or compliance verdict.

What to do with the result

Work through the three gaps shown first, then answer again. Copy the result as Markdown and paste it into the ticket or design doc where the agent work is tracked.

More free tools

Questions about this tool

Does a high score mean my agent is ready for users?

No. It means you reported that the practices exist. The tool reads no code and checks no evidence, so treat the grade as a prompt for a conversation, not a verdict.

Why do three questions weigh more than the rest?

An evaluation set, a confirmation gate for irreversible actions, and a per-task cap each address a failure that is expensive or hard to undo. Missing any one of them changes the order of work more than a missing alert does.

What if the agent only reads data and never writes?

Answer Yes on the confirmation question only if no tool can move money, send messages, or delete data. If the question does not apply because no such tool exists, Yes is the honest answer.

Is my data stored anywhere?

Only in local storage on this device, so you can leave and return. Nothing is sent to a server. Use Start over to clear it.

Beyond the tool

Need this done for your program? Start with architecture and delivery sprint.

$5K-$8K. Book an intro call, or send a brief with what the tool showed you. We reply within one business day.

  • Acceptance in writing

    Criteria agreed before build starts

  • Proposal in 48h

    After a qualified scoping call

  • Change control

    Scope, exclusions, and owners named

Book intro call
Send project brief