How to check AI-drafted security questionnaire answers before you submit
You have a 200-question security questionnaire due Friday. An AI tool drafted most of the answers from your policies and last year's responses. They read well. The question is which ones say more than you can prove.
This guide is for the person who signs off on that document. It gives you a check you can run in an afternoon, before the answers become a commitment in a customer's file.
Why AI drafts are risky in this document
A security questionnaire is a list of claims your company makes in writing. Customers keep them. Some get copied into contracts. If an answer says "all production data is encrypted at rest with customer-managed keys" and that is true for one system out of three, you have a problem that shows up later, usually at a bad time.
AI drafting tools are good at producing fluent, complete-sounding answers, and that fluency is where the risk sits. A model that saw "we encrypt data at rest" in one policy can write a stronger, broader sentence than the policy supports. The answer still reads like every other answer in the file, so nothing tells the reviewer to slow down on it.
The lines to look for
Most AI-drafted answers are fine. The ones that need a second look tend to share a few traits.
Absolute words. "All," "always," "every," "fully," "100%." Each one is a claim that a single exception breaks.
Specifics with no source. A retention period, a review cadence, a named standard, an uptime figure. If you can't point to the document or system setting it came from, treat it as unverified.
Present tense for planned work. "We perform annual penetration tests" when the first one is scheduled for next quarter.
Scope creep. The question asks about production. The answer covers "all environments." Or it asks about employees and the answer adds contractors without a source for that.
Borrowed answers. Text reused from last year's questionnaire or a different customer, where something has changed since.
A pre-submit check you can run
- Export the answers into one sheet. Question, answer, and a blank column for the source.
- Fill the source column. For each answer, name the policy, ticket, config screen, or person who can confirm it. Leave it blank if you can't.
- Sort by blanks first. Every blank row is a claim with no backing you can show.
- Scan the blanks for the traits above. Absolutes, unsourced specifics, future work in present tense, wider scope than asked.
- Rewrite or confirm. Either narrow the answer to what the source supports or get the owner to confirm it in writing.
- Keep the sheet. When the customer asks a follow-up in six months, you know where each answer came from.
This takes time on a long questionnaire. Spend that time on the answers most likely to be wrong first.
Where a confidence check helps
The slow part of step 4 is finding the answers that sound more certain than their support. That is the job ClearRun does. You paste in an AI-drafted answer, and it returns a score for whether the answer's confidence is backed by what it cites, plus a pass, retry, or escalate call. It reads the text you give it. It does not reach into your systems or confirm your controls exist.
Use it as a first pass on long drafts, so the human reviewer starts with the answers flagged as overconfident. Scores can vary between runs and can change over time, so treat a score as a signal, not a fixed number. Treat a borderline score as a cue to check the source, and keep the human sign-off.
What this check does not do
It does not replace your security owner or your evidence. A questionnaire answer is true only if the control exists. This check helps you find the answers most likely to overstate.
It is also not legal review. If an answer will be copied into a contract, the people who sign the contract should see it.
Questions
Should we stop using AI to draft questionnaire answers?
That is your call. Many teams keep the drafting speed and add a review step like the one above before anything is submitted.
Which answers matter most?
The ones a customer is likely to rely on: encryption, access control, incident response, data retention, subprocessors, and anything about third-party audits.
How do I handle an answer I can't confirm before the deadline?
Narrow it to what you can confirm, or say the control is in progress and give the planned date. You can widen it once the control is confirmed.
Can ClearRun confirm our controls are in place?
No. It checks whether the wording of an answer is more confident than the support given in the text. Confirming the control is still your team's job.
Is the score the same every time?
Scores can vary between runs and can change over time, so treat a score as a signal, not a fixed number.
Try it on your next questionnaire
ClearRun checks whether the confidence in an AI answer matches the justification behind it. A free account gets 30 credits a month, and Pro starts at $25/month.
Related reading on ClearRun: