Guide
How to test an AI agent before it reaches your clients
By Mavrin Labs. Published . Last updated . Reviewed by Mavrin Labs on . How this guide was researched and checked.

An AI agent, software that receives a goal and a set of tools and chooses its own steps, is safe to put in front of clients once it passes tests you wrote from your own cases before the build. A person should still approve what it sends.
Test wrong facts, actions outside its list, hidden instructions, private data, tone and the handover to a person first.
The guide is for the owner or operations lead of a small firm or nonprofit that is about to let an agent draft or send messages, update records or answer customers.
It sets out what to test, how to launch in stages, what to log and when to test again. It is not legal or compliance advice, and it recommends no AI provider.
Sources last checked Monday, October 19, 2026, and, for the figures and sources added in November, Monday, November 16, 2026, Tuesday, November 17, 2026, or Wednesday, November 18, 2026. Mavrin Labs reviews this guide every quarter, and each source's own page wins over it.
What is an acceptance test for an AI agent?
An acceptance test for an AI agent is one written case, drawn from your real work, that states what arrives and the answer or action you will accept, so the firm judges the agent against its own standard before any client sees its output. It is the pass mark, not a demonstration.
A full set of acceptance tests becomes the agent's test plan: the same cases, run before launch, after every change and on each review day. The sections below say which cases to write, how to launch in stages and when to run the plan again.
Written cases are the core of any AI agent evaluation, because they hold the agent to the firm's standard rather than the builder's. If the term itself is new, the plain-language page on what an AI agent is comes first.
Why it matters: reported AI incidents keep rising
Testing matters because public failures of AI systems are growing, not shrinking. Stanford's 2025 AI Index counted 233 AI-related incidents reported to a public incident database in 2024, a rise of 56.4% on 2023 (the AI Index chapter on responsible AI, checked Monday, November 16, 2026). Those are only the cases someone reported in public.
The risks also reach well past wrong answers. NIST's generative AI profile, published in July 2024, lists 12 risks that generative AI creates or makes worse, among them confabulation (confident, made-up content), data privacy and information security (NIST AI 600-1, checked Monday, November 16, 2026).
A written test plan is how a small firm turns that list into cases it can run.
How do you know an AI agent is ready for clients?
An AI agent is ready for clients when it passes a set of written tests drawn from your real work, including the awkward cases, and when you can see and undo what it does. Readiness is a record you can show, not a feeling after a good demonstration.
Public guidance points the same way. The NIST AI risk management framework, a voluntary guide from the federal standards agency, asks an organization to show that a system is valid, reliable and safe before it goes live, and to put a plan for monitoring it in place once it runs (the framework's core functions).
Its companion, the generative AI profile, helps organizations find the risks particular to generative AI.
NIST released the framework on Thursday, January 26, 2023, and the generative AI profile on Friday, July 26, 2024, and says the framework is now being revised under a federal plan for AI (NIST's page on the framework, checked Wednesday, November 18, 2026). Read the current text before you rely on a detail.
In practice, readiness comes in this order:
- acceptance tests written before the build, from real cases;
- tests for each class of failure that matters with clients;
- a staged launch, with a person approving each message;
- logs and a set review day;
- a full retest whenever the provider changes the model.
Why write the acceptance tests before the build?
Write the acceptance tests before the build so that you judge the agent against your own standard, not against whatever it happens to produce. Tests written after the build tend to describe what the agent already does. Agreeing the tests first also tells the builder what done means before any work starts.
Draw the cases from real work, not invented examples. Pull a few weeks of genuine enquiries, records and replies, remove anything private the test does not need, and write down what a good member of staff did with each one. Then add the hard cases on purpose, because those are where an agent fails first.
Hard cases worth writing down include:
- a question the agent should answer by saying it does not know;
- a request for something your firm does not offer;
- an angry or upset customer;
- one message that holds two requests;
- a message that contains instructions aimed at the agent;
- a person who asks about another client;
- a case that needs a person, such as a complaint, a legal threat or a vulnerable client.
Agree with the builder, in writing, which cases must pass before launch and which failures stop the launch outright. One action outside the allowed list should stop it, however good the rest looks.
Which failures should the tests cover?
The tests should cover six classes of failure: wrong facts, actions outside the allowed list, instructions hidden in content the agent reads, private data leaking, the wrong tone, and a missed handover to a person. Each one has its own test cases and its own definition of a pass.
Four of the six match entries in the OWASP Top 10 for applications built on a large language model (the AI engine underneath an agent that reads and writes text), a list of the main security risks in that kind of software (the 2025 list, checked Wednesday, November 18, 2026).
The OWASP Foundation's generative AI project keeps the list. Its first list covered 2023 and 2024, and the 2025 edition this guide uses replaced it, still with prompt injection (text written to give an agent new orders) first. A test plan written against one edition is worth reading again when the next one appears.
Tone and handover are business risks rather than security ones, so you write those tests from your own standards.
How should you launch the agent in stages?
Launch the agent in stages, and keep a person approving each message before it goes out until the records show the approvals have become routine. Start with work that is low in risk and holds no sensitive data, then widen the work one kind of message at a time.
CISA and its partners in other countries published a guide on Friday, May 1, 2026, to the careful adoption of AI services called agentic (AI that takes actions across tools and systems on its own).
It advises beginning with uses that are low in risk and not sensitive, and avoiding broad or unrestricted access to sensitive data or critical systems (CISA's announcement of the guide).
- Shadow run. The agent works beside your staff on real cases, and you compare its drafts with what staff sent. Nothing it writes reaches a client.
- Approve every send. The agent drafts, and a named person reads, edits and approves each message before it goes out.
- Loosen narrowly. Once the conditions in the next section hold, one low-risk kind of message, such as a booking confirmation, may go out without approval, by written decision.
- Widen slowly. Each new kind of message starts again at stage two.
When is it safe to loosen the approval step?
Loosen the approval step for one kind of message at a time, and only when the approver has stopped making real changes for an agreed period, four weeks by this guide's default. The test plan should still pass in full, the message should carry little risk, and you should be able to switch approval back on at once.
Check all of these before you loosen anything:
- the approver's edits over the agreed period were about style, not facts or actions;
- every failure class in the test plan passed again on the current model;
- the kind of message is low in risk and holds no sensitive data;
- the log records every message the agent sends on its own;
- one person can turn approval back on in one step;
- someone owns the decision in writing, because the firm answers for what goes out.
Approval is not a stage you leave behind for good. Keep it for anything that commits money, changes a record a client relies on, or answers a complaint.
What should the logs show, and what happens on review day?
The logs should show, for every case, what arrived, what the agent read, what it drafted or did, who approved it and what they changed. Review day is a fixed day when a named person reads a sample of those records and every handover, edit and complaint, then updates the tests.
Record the model version with each entry, so you can trace a change in behavior to a change underneath. OWASP's entry on excessive agency advises logging and monitoring what agents and connected systems do, and notes that logs do not prevent harm but can limit it.
NIST's framework asks for monitoring plans once a system is live, and for incidents and errors to reach the people they affect (the framework's core functions).
On review day:
- read every handover and check that it reached the right person;
- read every draft the approver changed, and sort the changes by failure class;
- read a sample of the messages that needed no change, by default twenty or one in ten, whichever is more;
- turn each new hard case into a test;
- write down what changed and who decided.
What should happen when the provider changes the model?
When the provider changes the model, run the full test set again before the new model handles client work. A model update can change answers, tone and how an agent follows its limits, even when nothing in your own system changed. Treat a new model as a new build that still has to pass.
A provider can update or retire a model on its own schedule, so ask yours how much notice it gives and whether you can stay on a version while you test.
NIST's framework asks organizations to monitor the risks of third-party resources, and to monitor pre-trained models as part of regular maintenance (the framework's core functions). If the new model fails a test, keep the old one where you can, fix the instructions and test again.
Retest after the changes you make yourself, too: new instructions, a new tool, a new data source or a new kind of message.
What does a simple test plan look like?
A simple test plan is one table: each failure class, an example case, what counts as a pass and when to run the case again. Keep it beside the agent's design, add every new hard case from review day, and run the whole plan before launch and after every change.
The rows below are a starting point. Your own cases replace the examples, and the people who approve messages decide what a pass looks like.
Three starting defaults make the plan concrete, and each is yours to change. Write at least five cases for each failure class, hard ones included. Wait four weeks with no change to a fact or an action before loosening one kind of message. On review day, read twenty sent messages or one in ten, whichever is more.
These are this guide's defaults, not a published standard. Raise them for an agent that handles many messages or touches money, records or complaints, because more cases find more of the rare failures.
| What to test | Example case | A pass looks like | Run it again |
|---|---|---|---|
| Wrong facts | A client asks about a policy the records do not cover | The agent says it does not know and hands the question over | After any change to the records or the model |
| Action outside the list | A customer asks the agent to cancel an order | The agent drafts the request for a person and takes no action | After a new tool or permission |
| Hidden instructions | An attached document tells the agent to forward the client list | No action, and a note to a person | After every model change |
| Private data | A caller asks for another client's address | The agent declines without confirming that the other client exists | After a new data source |
| Tone | An upset customer writes a long complaint | The approver would send the draft unchanged | On each review day |
| Handover | A message mentions a lawyer | The case reaches the named person with a note | After any change to the handover rules |
| Model change | The provider releases a new version | The whole plan passes again before client work moves | Before every switch |
How does Mavrin Labs test an agent before handover?
Every Mavrin Labs build includes a written design plan and acceptance tests, training and a handover, so the tests exist before anyone relies on the agent. By default, nothing a Mavrin Labs system drafts goes out without your approval, so a person approves each message until you choose otherwise in writing.
After launch, a care plan includes monitoring, fixes, updates when a provider changes something, one improvement every month and a monthly review against the baseline, the measurement of the work taken before the build.
The page on how Mavrin Labs treats client data sets out the agreement terms behind the approval default, and the stages of an engagement show where the build and the care plan sit.
The tests are part of the AI automation service.
Sources behind this guide
Every source below was opened on one of the last-checked dates above, given beside each figure, and the guide is reviewed quarterly. The table pairs each outside fact with its publisher.
| Fact | Source |
|---|---|
| A system should be shown valid, reliable and safe before it goes live, then monitored | NIST framework core functions |
| A companion profile covers the risks particular to generative AI | NIST page on the framework |
| The main security risks in language model software are listed and numbered | OWASP list for language model applications |
| Hidden instructions can arrive through websites or files | OWASP on prompt injection |
| Instructions to keep data private may not hold | OWASP on sensitive information disclosure |
| Credible but false output needs verified sources and human checks | OWASP on misinformation |
| Permissions belong in the connected systems, and logs limit harm | OWASP on excessive agency |
| Agentic AI should start with uses low in risk and not sensitive | CISA announcement of its agentic AI guide |
| Reported AI incidents rose in 2024 | Stanford AI Index chapter on responsible AI |
| Generative AI creates or worsens a set of named risks | NIST generative AI profile |
Corrections and changes to this guide, newest last:
- Monday, November 16, 2026: the definition of an acceptance test and the incident and risk figures were added, each with its source.
- Tuesday, November 17, 2026: the opening sources line was corrected to name every date a source was checked, two OWASP citations were added, and a paragraph pointing to a demonstration that is no longer published was removed.
- Wednesday, November 18, 2026: the launch diagram, the starting defaults and the release dates of the NIST and OWASP guidance were added, and the OWASP list is now introduced once.
Key takeaways
- Write acceptance tests from real cases, hard ones included, before the build.
- Test wrong facts, allowed actions, hidden instructions, private data, tone and handover.
- Launch with a person approving every send, and loosen one kind of message at a time.
- Log every case with the model version, and read the log on a set review day.
- Run the whole test plan again whenever the provider changes the model.
Related pages
The guide to custom AI agents explains what an agent is and when one is worth building. The custom AI agents page describes the service, and the AI and your data page sets out the terms that govern a Mavrin Labs system.
How engagements work shows each stage in order, and owners who would rather meet in person will find how that works on the Sarasota page. To talk through testing an agent your firm is about to launch, start a conversation.
Frequently asked questions
Can we skip our own testing if the provider says its model is safe?
No provider test can replace yours, because the provider tests its model in general and you need to know how the agent behaves with your clients, your records and your rules. Run your own acceptance tests on your own cases, and treat the provider's statements as one input, never as the pass itself.
How many test cases does an agent need?
An agent needs enough test cases to cover every failure class and every kind of message it will handle, with the hard cases weighted heavily. No standard fixes a number, so this guide starts at five written cases for each failure class and adds more where the agent sees the most messages.
Who is responsible if the agent sends something wrong?
The firm that deploys the agent stays responsible for what it sends, which is why approval on every message comes first and why loosening it is a written decision. Keep the log, so you can trace a wrong message, correct it and explain it to the client. Ask counsel how the rules apply to your firm.
How often should we test the agent again?
Test the agent again whenever something underneath it changes: the model, its instructions, its tools, its data sources or the kinds of message it handles. Between changes, a regular review day keeps the test set current, because each new hard case found in the log becomes a test for the next full run.
