AI Vendor Security Questionnaire: 40 Questions to Assess Any AI Vendor
AI Vendor Security Questionnaire: 40 Questions to Assess Any AI Vendor
Gal Nakash
July 15, 2026
6 min read
16 584 views
Key Takeaways
An AI vendor security questionnaire covers the model-layer and autonomy risks that SIG, CAIQ, and standard questionnaires may not fully address: training on customer data, prompt retention, autonomous action limits, and sub-processor disclosure.
Every question needs a defined red-flag answer and a scoring rubric, so responses become a defensible approve, conditional, pilot, or decline decision rather than a gut call.
Questionnaire answers are claims made on one day about a product that changes weekly, so the strongest programs verify them against live telemetry from their environment.
Quick Solution
An AI vendor security questionnaire is a structured assessment for vendors whose products process your data through AI models or act autonomously in your environment. It covers what standard questionnaires miss: whether your data is used to train models, how prompts and outputs are retained, what the product can do without a human in the loop, and which model providers sit in the request path.
The standard playbook falls short here. A vendor can hold a clean SOC 2, encrypt everything at rest, and still route customer records to a third-party model, retain embeddings indefinitely, and let any employee build automations on top of your CRM. Below you’ll find the questionnaire: 40 questions across six domains, the red-flag answer for each, a scoring rubric, and a way to verify what vendors claim against what their products actually do.
What Is an AI Vendor Security Questionnaire?
It is the AI-specific layer of vendor risk assessment: a due-diligence instrument sent before purchase and at renewal, documenting how a vendor's AI features handle customer data, what autonomous actions the product can take, and which third-party models sit in the request path. The answers feed one decision: approve, approve with conditions, pilot, or decline.
It applies to more vendors than most teams assume. Send it to any vendor that matches at least one criterion:
CRITERION
EXAMPLE
WHY IT QUALIFIES
Processes your data through an LLM
Transcription, email drafting, support copilots
Your data reaches a model, possibly a third party's
Ships autonomous features
Workflow builders, custom GPTs, task execution
The product acts without a human per action
Requests OAuth scopes into core systems
Read/write access to email, CRM, code, files
Blast radius extends past the vendor's own product
Embeds a third-party model
Tools built on GPT, Claude, or Gemini APIs
You inherit that sub-processor's data practices
Offers a plugin or extension ecosystem
Marketplaces, connectors, MCP servers
The vendor's risk includes their ecosystem's risk
Warning: Existing vendors count too. A tool approved two years ago as a project tracker may have shipped AI features since. Re-classification is one of the most commonly skipped steps in AI risk programs.
Why Standard Security Questionnaires Fall Short for AI Vendors
SIG, CAIQ, and in-house questionnaires typically treat a vendor's product as a bounded application. AI products break three of the underlying assumptions:
STANDARD QUESTIONNAIRE ASSUMES
AI VENDORS ACTUALLY DO
WHAT IT MISSES
Data stays in the vendor's app
Data flows to model providers and inference infrastructure
Training use, retention of prompts and embeddings, sub-processor chains
The product only does what users click
Products plan and execute multi-step actions on connected systems
Models get swapped, features ship weekly, users build their own automations
Change notification, drift from the assessed state
A vendor can therefore pass a standard assessment honestly and still pose a materially different risk from the one you documented. The questions below close those gaps, and they supplement your standard questionnaire rather than replacing it.
The 40-Question AI Vendor Security Questionnaire
Copy the six tables below into your assessment tool. Each question includes a defined red-flag answer, so reviewers know which responses should stop the process. Require named responses per domain, not a generic security@ reply.
DOMAIN 1: DATA HANDLING AND RETENTION
#
QUESTION
RED FLAG ANSWER
1
What customer data does your product send to an LLM, and is it your model or a third party's?
“We can't specify” or a third party they won't name
2
How long are prompts, outputs, and embeddings retained, and can we set retention to zero?
Indefinite retention, or retention set only by the model provider
3
Is our data logically or physically isolated from other tenants at the model layer?
Shared vector stores or shared fine-tunes across customers
4
Can we exclude specific data classes (PII, PHI, source code) from AI processing?
All-or-nothing processing with no data class controls
5
Where is inference performed geographically, and can we pin it to a region?
“Wherever capacity is available”
6
What happens to derived artifacts (embeddings, summaries, indexes) when we terminate?
Deletion covers raw data only, not derived artifacts
7
Do human reviewers ever see our prompts or outputs, and under what circumstances?
Routine human review without notification or opt-out
DOMAIN 2: MODEL TRAINING AND IMPROVEMENT
#
QUESTION
RED FLAG ANSWER
8
Is customer data used to train, fine-tune, or evaluate any model, yours or a third party's?
Yes by default, opt-out buried in settings
9
If training on customer data is opt-in, is the opt-in contractual or a UI toggle any admin can flip?
UI toggle only, no contractual guarantee
10
How do you prevent our data from appearing in another customer's outputs?
No cross-tenant leakage testing
11
Do you use our data for “service improvement,” and how is that defined?
Broad, undefined “improvement” language
12
When you upgrade or swap the underlying model, do you notify customers before or after?
Silent model swaps with no notification
DOMAIN 3: ACCESS, IDENTITY AND PERMISSIONS
#
QUESTION
RED FLAG ANSWER
13
What is the minimum OAuth scope set your product functions with?
Only a maximal scope list, no reduced-scope mode
14
Does the product support read-only or scoped deployment modes for evaluation?
Full write access required from day one
15
How are non-human identities (service accounts, keys, tokens) created, rotated, and revoked?
Long-lived static keys with manual rotation
16
Can we restrict which users and groups can invoke AI features?
Tenant-wide enablement only
17
Do AI features inherit the invoking user's permissions, or run with elevated service permissions?
Elevated service account that exceeds any single user
18
Does the product support SSO and SCIM, and are AI features covered by the same session controls?
AI features accessible outside the SSO boundary
19
What audit events do you emit for AI actions, and can we stream them to our SIEM?
No per-action audit trail for AI activity
DOMAIN 4: AUTONOMY AND GUARDRAILS
#
QUESTION
RED FLAG ANSWER
20
Which actions can the product take autonomously versus with human approval?
Vendor cannot enumerate autonomous actions
21
Can we require human approval per action type (sending, deleting, purchasing, code execution)?
Approval gates not configurable per action type
22
How do you defend against prompt injection and instruction hijacking from untrusted content?
“The model handles it” with no specific controls
23
Are tool calls and external actions sandboxed and rate-limited?
Unbounded action execution, no rate limits
24
Can users build their own automations or agents on your product, and can we disable that?
User-created automations cannot be restricted centrally
25
Is there a kill switch to halt all autonomous activity immediately, and who can trigger it?
No emergency stop, or vendor-side only
26
How do you log the decision process or tool sequence behind an autonomous action?
Only final outputs are logged, not the action chain
DOMAIN 5: SUPPLY CHAIN AND SUB-PROCESSORS
#
QUESTION
RED FLAG ANSWER
27
List every model provider and AI sub-processor in the request path for our data.
Incomplete list or refusal to disclose
28
What is your notification window when you add or change an AI sub-processor?
No notification, or notification after the change
29
Do plugins, connectors, or marketplace extensions inherit your security guarantees?
Marketplace items are explicitly out of scope
30
How do you vet third-party integrations before listing them in your ecosystem?
Self-attestation only, no vendor review
31
Which of your certifications (SOC 2, ISO 27001, ISO 42001) cover the AI features specifically?
Certification scope excludes AI processing
32
Have your AI features been independently red teamed or pen tested in the last 12 months?
Testing covered the app but not the AI attack surface
DOMAIN 6: INCIDENT RESPONSE AND ACCOUNTABILITY
#
QUESTION
RED FLAG ANSWER
33
What is your notification SLA for incidents involving AI processing of our data?
No defined notification SLA, or “as required by law” only
34
Do you classify model-behavior incidents (output leakage, successful injection) as security incidents?
Only infrastructure breaches count as incidents
35
Can you reconstruct what an autonomous workflow did during a given time window?
Forensic reconstruction not possible
36
Who is liable when an autonomous action causes damage: you, the model provider, or us?
Liability pushed entirely onto the customer
37
Do you maintain a public trust center or changelog for AI feature changes?
No public record of AI changes
38
How quickly can you revoke our tenant's access tokens and connections on request?
Revocation requires a support ticket with no SLA
39
Do you carry insurance that covers AI-specific failures?
Standard cyber policy that excludes AI incidents
40
Will you contractually commit to the answers in this questionnaire?
Questionnaire answers are “informational only”
Note: Question 40 matters most. A vendor that answers the first 39 questions well but refuses to put those answers in the contract has told you what they’re worth.
How to Score AI Vendor Questionnaire Responses
Unstructured review leads to gut-feel decisions that are difficult to defend in an audit. Score each question 0, 1, or 2, and set thresholds before reading a single answer.
SCORE
MEANING
EXAMPLE
0
Red flag answer or refusal to answer
Trains on customer data by default
1
Answered, but the control is weak or manual
Retention deletion by support ticket
2
Strong answer with evidence
Zero retention, contractually committed, audit report attached
With 40 questions, the maximum is 80:
TOTAL SCORE
DECISION
64 to 80
Approve, standard monitoring
48 to 63
Approve with conditions: reduced scopes, restricted groups, quarterly review
32 to 47
Pilot only, no production data, 90-day re-evaluation
Below 32, or any 0 in Domains 1 and 2
Decline
Warning: Any zero on the training questions (8 to 12) should trigger an automatic decline or escalation, regardless of total score. Once customer data has been incorporated into model training, removing its influence from the resulting model may be difficult or impossible to verify.
A questionnaire captures what a vendor claims on the day they filled it out. The product may change weekly, while its footprint in your environment can change daily. Three failure modes recur:
Drift. The vendor answered honestly in January. By March, they swapped the model, added a sub-processor, and shipped an automation builder. Your assessment now describes a product that no longer exists.
Scope creep. The product now holds write access to systems that were never included in the assessment because an admin clicked accept on an update prompt.
The gap between vendor and product. The vendor's security team answered the questionnaire. Your employees then built custom GPTs and workflows on their platform that those answers never contemplated.
The fix is not a longer questionnaire. It is verifying claims against the one source a vendor cannot edit: your own environment. That means auditing AI usage across your organization and comparing what you find with what the vendor disclosed.
Verify Vendor Answers Against Your Environment with Reco
Most teams lack continuous visibility into what a vendor's product actually does after deployment. Reco's AI Agent Security discovers every AI agent across your environment as part of its daily discovery cycle. The AI Agent Inventory then turns questionnaire answers into claims you can test against observed activity.
Navigate to AI Agent Security → AI Agent Inventory
Reco's AI Agent Inventory. Every discovered agent with its risk, connections, owner, exposure, AI model, and authorization status.
Each column tests a specific claim:
INVENTORY COLUMN
CLAIM IT VERIFIES
MISMATCH EXAMPLE
AI Model
Q27: disclosed model providers
Vendor claimed a proprietary model, inventory shows GPT-3.5 Turbo
Connections
Q13: minimum scopes needed
“CRM read-only” vendor connected to Salesforce, Slack, and GitHub
Exposure
Q3: tenant isolation
Agent set to Anyone With Link instead of Private
Owner
Q16: restricted user groups
Agents created by users outside the approved pilot group
The second check is the one no questionnaire can answer: blast radius. Graph view shows every vendor platform and the systems its agents can reach, with edges colored by severity.
Graph view of the full environment. Dense clusters are concentration risk the questionnaire never surfaces.
Zoom into one vendor and the assessment changes character. A platform fanning out into GitHub, Slack, Salesforce, Teams, and file storage is a different risk than the same vendor connected to one system, even at an identical score. The edges past the first hop reveal transitive reach: an agent wired into your automation platform may inherit access to systems that platform can touch.
One vendor platform, dozens of reachable systems. Red edges mark high-severity paths. This is the blast radius behind the questionnaire.
The decision then becomes enforceable. Every agent tied to the vendor gets an authorization status, so the outcome is reflected where the risk lives:
ASSESSMENT OUTCOME
AUTHORIZATION STATUS
FOLLOW-UP
Approved
SANCTIONED
Annual reassessment
Approved with conditions
RISK ACCEPTED
Quarterly review, conditions documented
Pilot only
TO REVIEW
90-day re-evaluation with fresh telemetry
Declined
UNSANCTIONED
Detection policies alert on new usage
Action: Attach the filtered inventory export and the vendor's subgraph next to the questionnaire. Treat discrepancies as formal findings and adjust the score based on what the telemetry supports.
When to Reassess AI Vendors
Annual reassessment is the floor. Each of these invalidates part of the original assessment and should trigger an early review:
The vendor swaps or upgrades its underlying AI model
A new AI sub-processor appears, whether in the vendor's disclosure or first in your telemetry
The vendor ships new autonomous features or an automation builder
Observed connections expand beyond the assessed blast radius
Risk on any of the vendor’s agents escalates to High or Critical
Make the Questionnaire an Ongoing Practice
A completed questionnaire is a snapshot of one vendor’s answers on one day, and the scoring rubric only holds up if reviewers revisit those answers against what the product actually does once it is live in your environment. Treat the 40 questions as the opening move in an ongoing assessment, not the final word on it.
Score Before You Sign: Run every new AI vendor through the six domains and the 0-to-80 rubric before granting access. Escalate any zero in Domains 1 or 2 regardless of the total score.
Verify After You Sign: Pull the vendor’s agents into the AI Agent Inventory and compare disclosed models, scopes, and exposure with observed connections and activity, not restated claims.
Reassess on Trigger, Not Just on Schedule: A model swap, a new sub-processor, expanded connections, new autonomous capabilities, or a risk escalation to High or Critical should pull a vendor back into review well before the annual cycle.
Enforce Through Authorization Status: Keep the decision tied to the agent, not just the contract. If an approved vendor’s agents begin operating outside the conditions you assessed, the change should become visible where the risk actually lives.
The strongest AI vendor risk programs do not choose between questionnaires and continuous monitoring. They use the questionnaire to establish what the vendor claims, telemetry to verify those claims, and reassessment to catch the moment the two diverge.
Gal Nakash
ABOUT THE AUTHOR
Gal is the Cofounder & CPO of Reco. Gal is a former Lieutenant Colonel in the Israeli Prime Minister's Office. He is a tech enthusiast, with a background of Security Researcher and Hacker. Gal has led teams in multiple cybersecurity areas with an expertise in the human element.
%201.svg)
%201.svg)
.svg)