Enterprise-Ready AI Security
The AI Security Questionnaire Playbook
A strong AI security questionnaire answer is a short claim backed by a current control, a named owner, and evidence a reviewer can inspect.
Build one standing pack around the 24 questions below, then map each buyer's wording to that pack instead of rebuilding the truth under deadline pressure.
Start with the system
Treat the questionnaire as an evidence-indexing problem
The painful part of an enterprise AI review is rarely the number of questions. It is the scramble to discover whether the company has one defensible answer. Security knows the access policy. Engineering knows where prompts are logged. Legal knows the provider terms. Product knows where a person confirms an action. The questionnaire arrives, and somebody has to reconstruct that system from chat threads, screenshots, and memory.
The durable solution is a standing evidence pack: a maintained map from common buyer questions to control statements, owners, implementation evidence, and review dates. It is not a set of polished yes answers. It is a compact index into the way the product actually works. When the product changes, the evidence pack should make the affected answers visible.
Start by defining the AI system boundary. Include every model call, retrieval path, tool, cache, trace, evaluation job, feedback dataset, and provider that can touch customer information or change customer-visible behavior. Then draw one representative request from authenticated user to final result. Reviewers become uncomfortable when an answer says data is isolated but the diagram cannot show where tenant identity reaches the vector query or tool authorization check.
Reusable evidence
Build the standing pack before the next buyer asks
Store the pack in a system your engineering and security owners already maintain. A small repository works well because diagrams, policies, tests, and control records can be versioned together. A GRC platform also works when it can link to technical evidence without copying sensitive material into every questionnaire. The format matters less than the ownership and update path.
- AI system inventory with feature, owner, model route, and data class
- Current data-flow and trust-boundary diagrams
- Provider register with retention, region, and training settings
- Tenant-isolation and tool-authorization test results
- Prompt-injection and exfiltration regression suite
- Golden dataset, eval thresholds, and latest release report
- AI incident procedure and tabletop exercise record
- Control crosswalk with owners, links, and review dates
Write each control statement in plain language. Name what is enforced, where it is enforced, and what proves it. For example: "Every retrieval query includes a tenant identifier derived from the authenticated session; the retrieval service rejects calls without that identifier; the cross-tenant regression suite runs in CI." That sentence is more useful than "we use industry-standard isolation" because it gives the reviewer a testable mechanism.
| Field | What to record | Why it matters |
|---|---|---|
| Control statement | Short, precise description of the enforced behavior | Creates a stable answer across buyer wording |
| Owner | Named role responsible for implementation and review | Prevents evidence from becoming orphaned |
| Evidence link | Policy, configuration, test, diagram, or report | Turns a claim into something inspectable |
| Scope | Features, tenants, regions, and providers covered | Makes exceptions and gaps visible |
| Review date | Last verified date and next review trigger | Shows that the answer reflects production today |
The core reference
The 24-question AI security breakdown
Buyer questionnaires use different wording, but most questions reduce to the six groups below. Use the suggested answer guidance as a structure, then replace it with the truth of your system and link the matching evidence. Where a control is incomplete, mark the gap and remediation owner. A precise partial answer is safer than a confident claim that implementation cannot support.
System boundary and data flow
These questions establish what the AI system is, where data travels, and which party controls each step. A vague boundary makes every later answer harder to trust.
Question 01
Which product features use AI?
Inventory every customer-visible and internal AI feature, including summarization, search, classification, generation, tool use, and support automation. Name the production owner and the data classes each feature touches. Avoid answering only with the flagship assistant; reviewers are looking for overlooked AI paths in analytics jobs, internal tooling, and background workers. The evidence should be a current system inventory linked to architecture diagrams and service ownership.
Question 02
Which models and providers are in the production path?
List provider, model family, deployment region, purpose, and fallback behavior for each production route. Distinguish hosted APIs, cloud-managed models, and models you operate. State whether versions are pinned and how a model change is approved. Attach the provider register, data-processing terms, and the configuration or policy that selects a route. The useful answer explains the decision path, not a frozen list that becomes stale after the next release.
Question 03
What customer data enters the AI system?
Classify prompts, retrieved documents, conversation history, metadata, tool results, and generated output under the same data-classification policy used elsewhere in the product. Say which fields are prohibited, redacted, tokenized, or minimized before a model call. Include a data-flow diagram showing ingress, storage, retrieval, inference, logging, and deletion. A reviewer should be able to trace one representative request without guessing which copy of the data is authoritative.
Question 04
Is customer data used to train or improve models?
Answer separately for your company and each model provider. State the contractual setting, the technical control, and the verification evidence. If training is disabled by account configuration, link the provider term and capture the configuration state. If selected examples enter an internal improvement dataset, explain consent, access, de-identification, retention, and deletion. Do not use a blanket no when product feedback or trace review creates a secondary improvement path.
Isolation, identity, and access
Enterprise reviewers treat the AI path as another privileged data path. The same identity and tenant rules must survive retrieval, tool calls, caches, and traces.
Question 05
How is tenant isolation preserved?
Describe the tenant identifier from request authentication through retrieval filters, cache keys, tool authorization, and logs. Provide tests that attempt cross-tenant retrieval and tool access. If shared vector indexes are used, explain mandatory metadata filters and the layer that enforces them. If indexes are isolated per tenant, explain provisioning and deletion. The strongest evidence is a negative test that fails when a tenant boundary is removed, not a diagram that merely labels a box multi-tenant.
Question 06
How is data encrypted?
Cover transport, storage, backups, vector stores, trace stores, and temporary processing locations. State which service manages keys and whether sensitive tenants can use customer-managed keys. For external model calls, identify the encrypted transport boundary and any provider-side persistence. Link infrastructure policy or configuration evidence rather than repeating a generic encryption claim. If an ephemeral buffer cannot be encrypted at the application layer, document its lifetime and compensating controls.
Question 07
Who can access prompts, retrieved context, and outputs?
Map roles to concrete permissions across production databases, observability tools, support consoles, and provider dashboards. Explain just-in-time access, approval, time limits, and review cadence. Include break-glass access and how its use is logged. AI traces often contain more sensitive context than normal application logs, so a general production-access statement is insufficient. Attach the role matrix and a recent access review with personal data removed where necessary.
Question 08
How are tool actions authorized?
The model may propose an action, but trusted application code must authorize it using the authenticated user, tenant, resource, and requested scope. State that credentials and tenant identity are never accepted from model-generated arguments. Show the policy enforcement point and a denied-action test. For high-impact tools, describe confirmation, idempotency, and transaction limits. Reviewers want proof that persuasive text cannot convert into permission by being placed in a prompt or tool payload.
Model and application security
This section moves beyond normal application security into attacks that use instructions, context, and model behavior as the path.
Question 09
How do you defend against prompt injection?
Explain the layered controls for direct instructions, retrieved content, tool output, and remote content. Useful controls include trust labeling, instruction hierarchy, input and output filtering, tool allowlists, least privilege, and result validation. No single classifier is a complete defense. Attach adversarial cases covering indirect injection, multi-turn escalation, encoded payloads, and attempts to reveal system instructions. State which failures block release and which trigger a manual review.
Question 10
How do you prevent data exfiltration?
Start from the data the model can see and the destinations it can reach. Restrict retrieval by tenant and permission, constrain outbound tools and URLs, redact sensitive fields, and validate generated tool arguments. Add canary records or synthetic secrets in adversarial tests to detect leakage. Explain how response streaming is stopped when a policy fails. Evidence should cover both direct answers and indirect channels such as URLs, encoded output, tool calls, or stored memory.
Question 11
How are model outputs validated?
Separate presentation text from outputs that drive code, database writes, policy decisions, or external actions. Use typed schemas, constrained decoding where available, deterministic validators, bounds checks, and business-rule enforcement outside the model. Reject unknown fields and unsafe values. Attach examples of malformed outputs and the resulting failure behavior. A successful parse is not proof of a safe action; authorization and domain validation still apply after structure validation.
Question 12
How do you test jailbreak and misuse resistance?
Maintain a versioned adversarial suite tied to product capabilities and realistic attacker goals. Include single-turn and multi-turn attacks, role-play, instruction conflicts, tool misuse, privilege escalation, unsafe retrieval, and persistence across state. Record model, prompt, policy, and tool versions for reproducibility. Track the pass rate by attack class and preserve failed transcripts. The suite should run before meaningful model, prompt, retrieval, or tool changes reach production.
Reliability, evaluation, and change control
A secure system can still create material risk when model behavior changes silently. Reviewers increasingly ask how quality is measured and how releases are controlled.
Question 13
How do you measure hallucination and groundedness?
Define the product-specific failure classes instead of claiming the model does not hallucinate. Measure unsupported claims, wrong citations, missing evidence, retrieval misses, and refusals on answerable requests separately. Use a golden dataset built from production-shaped questions, deterministic citation checks, calibrated rubric graders, and human review for ambiguous cases. Report thresholds by high-risk slice because a single average can hide failure for one tenant, language, or workflow.
Question 14
What evaluation happens before release?
List the changes that trigger evaluation: model versions, system prompts, retrieval settings, chunking, rerankers, tools, policies, and major data-source changes. Explain the baseline, comparison method, critical slices, and release thresholds. Attach a sample pull-request report with failed cases. Make clear who can accept a regression and how that decision is recorded. Evaluation is strongest when it gates the release path rather than producing a dashboard that nobody must act on.
Question 15
How do you monitor production AI behavior?
Connect model and tool traces to the completed customer workflow, tenant, release version, latency, cost, policy result, and user feedback. Define alerts for error bursts, unsafe outputs, retrieval degradation, abnormal tool use, and cost anomalies. Explain sampling and redaction so monitoring does not create a second uncontrolled data store. Include the on-call owner and the route from a production failure into a new regression case.
Question 16
How are model and prompt changes managed?
Treat prompts, routing policy, eval datasets, and model identifiers as versioned production artifacts. Require review, evaluation results, staged rollout, rollback criteria, and an owner. Pin versions where the provider supports it and document the response when it does not. Preserve the previous prompt and route so rollback is an operational action, not an emergency rewrite. Attach the change record and a recent example that shows the control operating.
Human oversight, safety, and incidents
The right control depends on consequence. A draft summary and an autonomous account change should not share the same approval path.
Question 17
Where is human review required?
Classify AI actions by reversibility, customer impact, legal effect, financial effect, and access to sensitive data. Define which actions can run automatically, which require confirmation, and which remain human-only. Show the review interface and the context available to the reviewer. Avoid saying a human is in the loop when the person can only approve without understanding the source evidence, model uncertainty, or proposed action.
Question 18
How can a user correct or contest an AI result?
Describe visible AI labeling where appropriate, feedback controls, correction paths, and escalation to a person. Explain whether a correction changes only the current output, updates source data, or creates an evaluation case. Preserve enough provenance to investigate the result without exposing internal prompts or another tenant's information. Enterprise buyers want to know that a wrong answer can be contained and corrected through a defined product workflow.
Question 19
What is the AI incident-response process?
Extend the existing incident process with AI-specific triggers: sensitive-data disclosure, unsafe tool action, widespread quality regression, provider behavior change, poisoned retrieval content, and abnormal cost cascades. Define severity, containment, evidence preservation, provider coordination, customer notification, and post-incident evaluation updates. Run a tabletop exercise using an AI scenario. The evidence is the playbook, ownership roster, exercise record, and one example of a corrective regression test.
Question 20
How do you assess harmful or biased behavior?
Tie the assessment to the product's users, decisions, and foreseeable misuse. Identify protected or sensitive slices where applicable, define unacceptable behavior, and include representative cases in evaluation. Explain reviewer qualifications and escalation when automated scoring is unreliable. Avoid generic fairness claims for products that have not defined the affected population or decision. Document residual risk and product limits plainly rather than turning a nuanced assessment into a universal pass badge.
Providers, privacy, and lifecycle
The final questions test whether controls extend through providers and through the full life of customer data.
Question 21
Which AI subprocessors handle customer data?
Maintain a provider register that names the service, purpose, data classes, processing region, retention terms, training setting, and contractual role. Include model APIs, vector databases, trace platforms, moderation services, and human-labeling vendors where used. Link the public subprocessor notice and the internal approval record. If traffic can route across providers, explain the policy and ensure every possible route is represented rather than documenting only the preferred provider.
Question 22
What are the retention and deletion rules?
State retention separately for application data, prompts, outputs, embeddings, caches, traces, feedback datasets, backups, and provider logs. Explain how a customer deletion request propagates to derived stores and when deletion completes. If an embedding cannot be reversed into source text, it still needs a lifecycle rule. Attach configuration evidence and a deletion-flow test. Make exceptions, such as security-log retention, explicit and tied to a policy basis.
Question 23
Where is AI data processed and stored?
Map each production route to processing and storage regions, including failover, observability, support access, and provider abuse monitoring. Distinguish configurable residency from provider-wide commitments. Explain how restricted tenants are prevented from using an incompatible route. The evidence should combine architecture, provider terms, account configuration, and a routing test. A statement that the main database is regional does not cover model inference or trace storage.
Question 24
Which standards and regulations do these controls map to?
Create a crosswalk from implemented controls to the frameworks your buyers ask about, such as SOC 2 criteria, ISO 27001, ISO 42001, NIST AI RMF, the CSA AI controls, and applicable AI regulation. Keep the implementation evidence primary and the mapping secondary. A crosswalk can reduce repeated work, but it does not turn one control into universal compliance. Record applicability, owner, evidence link, review date, and any open gap for every mapped control.
Answer once
Use claim, mechanism, evidence, and boundary
Most weak answers fail in one of two ways. They give a bare yes with no mechanism, or they paste a full policy that never answers the question. A reusable response needs four parts: the claim, the mechanism that makes it true, the evidence that shows the mechanism operating, and the boundary where the claim stops.
- Claim: answer the question directly in the first sentence.
- Mechanism: name the enforcement point, responsible system, or required workflow.
- Evidence: link a current artifact and give its review date.
- Boundary: state exceptions, unsupported routes, or planned remediation.
Keep the first answer concise enough to paste into a questionnaire cell. Put deeper context in a linked appendix. This gives procurement a direct response while allowing a security reviewer to inspect architecture, tests, or policy without opening a new email thread. Label sensitive evidence and share it through the same controlled channel used for other security-review material.
Stay defensible
Make freshness an engineering property
Evidence becomes stale when ownership depends on a calendar reminder that nobody connects to product changes. Add event-driven review triggers. A new model provider should trigger the provider register, data-flow diagram, retention answer, residency answer, and subprocessor notice. A new tool should trigger authorization, incident, audit, and human-oversight controls. A new trace platform should trigger access, retention, region, and deletion controls.
Put technical evidence as close as possible to the change. Tenant-isolation tests and adversarial cases belong in CI. Model and prompt versions belong in source control. Provider configuration should be captured by infrastructure policy or a recurring verification job where possible. Human evidence, such as access reviews and incident exercises, needs a named owner and review cadence.
| Change | Review these controls | Required output |
|---|---|---|
| New model or provider | Training, retention, region, change control, evals | Updated provider record and release evidence |
| New retrieval source | Data flow, access, tenant isolation, deletion | Diagram update and negative isolation tests |
| New tool action | Authorization, validation, oversight, incident response | Threat review and denied-action tests |
| New observability store | Access, redaction, retention, residency | Data classification and lifecycle update |
| Material AI incident | Affected control plus related eval slices | Corrective action and regression cases |
Buyer response
Run a fast review without improvising
When a questionnaire arrives, classify questions into direct matches, wording variations, evidence requests, and genuine gaps. Direct matches should be populated from the standing pack. Wording variations need a small adaptation while preserving the same claim and boundary. Evidence requests should use approved links or exports. Genuine gaps need an owner, risk statement, and remediation decision before the answer is submitted.
- Intake: record the buyer, deadline, data scope, deployment model, and requested standards.
- Map: connect each question to the standing control register and flag unmapped items.
- Verify: ask owners to confirm only changed or high-risk claims, not every unchanged sentence.
- Review: security, engineering, legal, and product inspect the sections they own.
- Submit: preserve the exact answer set, evidence version, approvals, and buyer follow-up.
- Learn: add novel questions and useful evidence back to the standing pack.
Keep an answer ledger across buyers. It reveals where the same control is described inconsistently and where one buyer's question exposes a real gap. The ledger should reference control identifiers rather than becoming another independent copy of the answer. That distinction keeps improvements flowing back to the source instead of multiplying stale spreadsheets.
Avoid these traps
The answers that create more questions
Absolute claims without a defined scope
Statements such as "customer data never leaves our environment" fail when a hosted model or trace provider processes it. Define the environment and name the external boundary. A narrower true claim builds more confidence than an absolute claim that the architecture contradicts.
Policies with no operating evidence
A policy may say access is reviewed, but the reviewer needs to know that the review happened and covered AI traces. Pair policy with a recent control record, configuration state, test result, or release report. Avoid screenshots without date, account, or scope context.
Provider assurances presented as product controls
A provider's certifications do not prove your tenant filter, tool authorization, output validation, or incident path. Use provider evidence for the part the provider operates, and maintain application evidence for your part of the system. Shared responsibility is a map, not a sentence.
One average score for every AI risk
A high aggregate eval result can hide failure on the workflow an enterprise buyer cares about. Report critical slices, attack classes, and regression thresholds. Explain who can approve an exception and how long it lasts. Reviewers trust an explicit residual risk more than a score stripped of consequence.
Before submission
The final reviewer-ready checklist
- Every yes has a named mechanism and current evidence
- Every answer uses the same system boundary and data classification
- Provider claims are separated from application controls
- Tenant identity is traced through retrieval, tools, caches, and logs
- Model, prompt, retrieval, and policy changes have release evidence
- Retention covers prompts, outputs, embeddings, traces, and backups
- Exceptions and open gaps have owners and target dates
- The submitted answer set and evidence version are preserved
The goal is not to make every answer look perfect. It is to make the system understandable, the controls testable, and the remaining risk explicit. That is what lets a buyer's security team finish its work, and it gives your own team a durable baseline for the next review.
Next step
Bring the failure, the evidence, and the system you have today.
We identify the highest-leverage scope, the artifact your team should own, and whether a focused implementation is the right move.
30 minutes. No deck. Leave with a clear next step.