Questions a director should ask before hiring an AI agent
Before hiring a synthetic agent, an executive should be able to get answers to twelve questions: what work it will do, in which systems, who can stop it, what it logs, where the data lives, how it is measured, how it introduces itself to customers and how you exit the contract. If a vendor can't answer clearly and in writing, the agent isn't ready to hold a role.
What exactly will the agent do?
A good answer is a role with concrete tasks (“qualifies leads and books meetings,” “follows up on overdue payments”), not “automates your support.” Red flag: if the description fits in a slogan but not in a task list, there is no scope yet.
Which systems does it act in, and what can it do there?
Ask for the list of systems it connects to (CRM, ERP, calendar, collections system) and the allowed actions in each: read, create, modify, never delete. An agent that only replies and one that executes are different problems; see them side by side in synthetic agent vs chatbot, virtual assistant and call center.
Who writes the rules, and who can change them?
The rules belong to the business. Ask where they are written, who approves them and how each change is recorded. Red flag: rules that live only inside a prompt nobody on the business side can read.
Who can stop it, and what does the customer see when it stops?
Ask for a demo: who pulls the brake, from where, how fast it works and what the customer gets in the meantime. The detail of this control is in how a synthetic agent is governed.
What does it log, and can I audit it?
Every action should be recorded with a date, a cause and a result, along with every human intervention. Ask to be shown one real, anonymized case traced end to end. Red flag: aggregate reports that can't be drilled down to a single case.
Does it pass itself off as a person?
The acceptable answer is no. An agent introduces itself as what it is, doesn't deny it when asked and there is always a person the customer can reach. Beyond being the right thing to do, it is what customers expect; if your industry is regulated, confirm with your advisers which obligations apply.
Where does my data live, and is it used to train models?
Ask in which accounts and infrastructure the data sits, whether it is mixed with other customers' data, whether it is used to train models and what happens to it when the engagement ends. Get the answer in writing.
Which AI model does it use, and what if that changes?
The healthy answer is that the agent doesn't depend on a single model provider and that swapping models doesn't force a rebuild of rules, memory or logs. A summary is in which AI model your agent uses.
How will we measure whether it works?
Metrics, baseline and review cadence should be agreed before you start. Red flag: promised percentages from a vendor who hasn't seen your process. The framework is in how to measure a synthetic agent.
Who maintains it after launch?
An agent changes because your business changes: products, policies, systems. Ask who maintains it, who checks that it keeps running at any hour and what that includes in the commercial model (explained on the pricing page). Red flag: a project that ends at delivery and leaves the agent without an owner.
What do you need from my team, and how long does it take?
A good vendor tells you which inputs it needs (access, documents, a person accountable for the process), what stages there are and what is delivered at each. A written schedule and a tight scope are a better sign than a promise of speed.
How do I exit if it doesn't work or I change vendors?
Ask what you take with you: rules, documentation, logs and data, and in what format. A vendor confident in its work doesn't need you locked in. For a shorter checklist, see the questions to ask before hiring an AI agent provider.
Which question first if I only have five minutes?
Three: who can stop it and what the customer sees, where each action is logged and whether it passes itself off as a person. If those three answers are solid, the rest can be refined.
Should I ask for a trial before signing?
Ask for a demo on one of your own cases, with sample data of yours, and look mostly at how it behaves off script and how it hands a case to a person. A demo that shows only the happy path tells you little.
Who should I involve on my side?
The owner of the process the agent will take on, whoever administers the systems it connects to and, if your industry is regulated, your legal or compliance team.