Before You Sign for an AI Agent, Ask the Vendor to Show the Controls

A free questionnaire and response form for testing an agent’s safeguards live, before purchase.

The demo went well. The agent answered questions, updated a record and handed a tricky case to a person. The contract is ready.

Before signing, you need answers to the questions the demo skipped. Which systems can the agent change? What blocks a bad action? How fast can someone stop the agent mid-task? And what record remains of everything the agent did?

When the agent comes from a vendor, the safeguards sit inside the vendor’s product. A standard vendor security review covers infrastructure and data handling. Agent safeguards need their own checks. The MACH Alliance has published a vendor questionnaire and response form, drawn from the MACH Alliance Agent Governance and Assurance Framework.

Ask to see the controls working

The questionnaire asks the vendor to run nine requests live, in the product, on a test account set up like your planned deployment. Slides and recorded videos count as a fail.

Expect to watch the vendor stop the agent partway through a task and push the agent past a spending limit until the software blocks the attempt. Another request plants a hidden instruction in an email to test whether the agent obeys. The vendor then exports the full record of the run for you to keep.

Five answers pause the evaluation until the vendor demonstrates a fix: a shared login, a limit enforced only by the agent’s instructions, a stop that depends on vendor support, a record that cannot be exported, and AI providers the vendor will not name.

Get the rest in writing

Seven requests ask for documents covering the AI providers that receive your data, notice before the vendor changes the AI model, test results for your setup, a recent monitoring report, incident notification, record keeping after the contract ends, and AI-specific certifications. Four more ask for written answers on memory, customer disclosure, connected agents and tools, and cost limits.

Each request spells out good evidence and weak evidence, so the vendor knows what to prepare and your team can compare responses. The completed response form also records the AI models, versions and hosting behind the agent, ready for the security review.

How to run the questionnaire

Start with six planned-use questions covering who the agent interacts with, the data and regulations involved, money and actions that are hard to undo, and how far a failure could reach. Any yes means the agent needs a good response to every request before purchase.

Send both files to the vendor and book a live session of 60 to 90 minutes. The vendor returns the completed form, with evidence, before the session.

Download the resources

• Vendor Questionnaire: AI Agent Safety (PDF)
• Vendor Response Form: AI Agent Safety (Word)

For support understanding the vendor questionnaire or applying the questionnaire to a purchasing decision, contact info@machalliance.org, and we will be happy to arrange an introduction to a member of the MACH Alliance Agent Adoption & Operations Working Group.