Using AI to Check Claims: What’s Supported and What’s Missing?
You find the relevant passage in the transcript. The words are there. The claim appears to check out.
But then you read the next question and the answer changes what the passage means.
Perhaps the witness only saw part of what happened. Perhaps they were describing what someone else told them. Perhaps a confident statement in a summary came from a much more uncertain answer.
The problem is not always an invented fact. Sometimes, it is a real statement presented without the context that limits it.
That distinction came up in our interview with a legal professional who uses Scoopie to review testimony. Alongside understanding what witnesses said, they described another task: checking whether someone else’s account fairly represented that testimony.
One category was particularly revealing: “accurate, but missing context.”
This article draws on a real customer interview. Identifying details have been removed. The specific testimony example below is fictional and does not reproduce any customer records.
The work begins with understanding who said what
The interviewee works with records containing testimony from multiple people. Before evaluating a claim about an event, they need to understand each person’s connection to it.
Their approach is deliberately focused.
They organize the testimony into separate text files and start a separate conversation for each witness. An initial question establishes who the person is and how they relate to the matter. Follow-up questions then explore that person’s account.
The interviewee explained that they still read the transcript. Having an overview first helps them understand what they are reading when they return to the record.
This creates a useful starting point for the next task. Instead of looking at a statement in isolation, the reviewer has some orientation: whose account it concerns, what that person described, and which records need closer attention.
Then comes the harder question.
Does the claim being made actually match the testimony?
A claim can contain the right words and still go too far
The interviewee described bringing a statement about what happened into the conversation and asking the AI to check it against selected witnesses’ testimony.
The request was not simply to summarize the material again. It was to examine the relationship between a particular claim and its supposed support.
Was the statement supported? Did the testimony contradict it? Or did it contain something accurate while leaving out important context?
That last possibility is easy to overlook.
A summary might preserve what a witness observed but remove the limits of that observation. It might report a conclusion while dropping the uncertainty attached to it. It might present someone’s account as an established fact.
Finding a related passage is therefore only part of the work. The reviewer also needs to ask whether the claim preserves the meaning of that passage.
What “missing context” looks like
Consider this fictional example.
A written account states:
“The witness confirmed that the delivery was undamaged.”
The underlying testimony reads:
Question: Did you notice any damage when the delivery arrived?
Answer: I didn’t see any damage on the two boxes by the door.
Question: Did you inspect the rest of the delivery?
Answer: No. I didn’t open any of the boxes.
There is some support for the account: the witness did not notice damage on the boxes they saw.
But “the delivery was undamaged” goes further. It turns a limited observation into a conclusion about the entire delivery, including contents the witness never inspected.
Here is how a reviewer could separate the two:
What the claim says | What the testimony supports | What is missing |
|---|---|---|
The delivery was undamaged. | The witness saw no damage on two boxes by the door. | The observation did not cover the entire delivery. |
The witness confirmed its condition. | The witness described what they could see. | They did not inspect the contents or the remaining boxes. |
This table is an illustrative editorial analysis, not a recorded Scoopie response.
A more faithful version would be:
“The witness said they saw no damage on two boxes by the door, but did not inspect the rest of the delivery or open any boxes.”
That sentence does not establish whether the delivery was actually damaged. It accurately describes what this witness’s testimony can support.
The important change is not stylistic. It restores the boundary between what someone observed and what a reader might otherwise assume.
The useful result is a passage to examine, not just a verdict
This is where the interviewee’s use of source references becomes important.
They described clicking a citation note in Scoopie to see where the supporting material appeared in the connected text file. That allowed them to move from an AI-generated statement back to the words behind it.
For the fictional delivery example, a useful review would bring attention to both parts of the exchange: the initial observation and the later qualification.
A result that only points to “I didn’t see any damage” would leave the central problem unresolved.
The reviewer needs to read the surrounding question and answer, check who was speaking, and decide whether the proposed wording goes beyond what the passage establishes. Where other relevant records exist, those may need examination too.
The interviewee also described a further check: before formally using the material, they manually verified it against the original PDF.
That distinction matters. A citation into a working text file helps locate the evidence. Checking the original record confirms that the working text and the interpretation built on it holds up.
The AI’s answer is part of that review process. It is not the final authority.
Why this is different from asking for another summary
A summary asks what a document says overall.
A claim check starts with a sentence someone wants to rely on and asks what the records actually support.
In the interviewee’s workflow, the first task helped them become familiar with the testimony. The second helped them examine how that testimony was being represented.
The questions serve different purposes:
Understanding the record: What did this person say about the event?
Checking the claim: Does that testimony support this particular description of the event?
Checking the context: What qualifications would change how a reader understands the claim?
The last question is especially useful when a statement sounds plausible. It directs attention to the details most likely to disappear when a long exchange becomes a short paragraph.
It also changes the desired outcome. The goal is not to label everything true or false. It is to identify what can be supported, what needs narrower wording, and what remains unresolved.
Try it with one claim and a clearly defined source
You can adapt this approach without asking AI to assess an entire matter at once.
Choose one statement and the records relevant to checking it. Use only material you are authorized to process, and begin with a fictional or non-confidential example if you are testing a new tool.
The following prompt is adapted from the approach described in the interview; it is not a verbatim transcript of the interviewee’s prompt.
Check the claim below against the selected source documents.
Identify what is supported, what is contradicted, and what cannot be established from these sources. Pay particular attention to statements that have some support but omit important context.
Show the supporting passages and any passages that qualify or contradict the claim. Preserve uncertainty, limits on what someone observed, and the distinction between what someone said and what actually happened.
Cite the source locations where available. Do not invent quotations or page numbers. If you cannot find support, say so rather than treating the claim as false.
Suggest narrower wording where needed, and identify anything that still requires human review.
Claim: [Paste one statement here.]
After receiving the answer, open the cited passages. Read beyond the highlighted sentence. Check the proposed revision against the source rather than assuming that a revised statement must be correct.
Keep the claim within the limits of the evidence
The most useful lesson from this interview is a specific way to divide the work.
AI can help locate relevant testimony and draw attention to possible discrepancies. The reviewer examines the evidence, considers its context, and decides what can responsibly be said.
The interviewee described this as reducing the burden of working through the material while retaining their own verification. It was an account of an individual workflow, not a measured claim that AI catches every omission or replaces reading the record.
A sentence can be grounded in a real source and still misrepresent it. The question worth carrying into your next review is:
Does the evidence support this claim as written or only a more limited version of it?
Bring the claim and its source together with Scoopie
Scoopie is an AI work assistant that answers questions using connected files and provides source references you can inspect. For this workflow, its value is helping you move from a claim back to the relevant passage not guaranteeing that an interpretation is correct.
Try checking a claim with Scoopie.
Sources and editorial background
The user workflow described above comes from a private customer interview. The interview records are not linked to protect the participant’s identity. The fictional example and adapted prompt were created for this article.
The following public articles informed the editorial research; they are not evidence of this customer’s results.
References
How to Scale Defensible Generative AI Results for Document Review
AI, Action, and Community: Turning Complex Data into Insights at Scale
Rebuilding the Review Algorithm to Increase Accuracy and Speed
Comparing deterministic AI to generative AI for cite-checking and last mile litigation tasks
AI hallucinations in the facts section are a growing threat to litigators