Study notes · 3.5% of the exam

Apply fact-checking and validation techniques

Validate claims in a Claude output against primary sources using techniques that are independent of the model: quotes and citations you can open, independent recomputation, and second queries, never Claude's self-confirmation or self-rated confidence.

Key points

  1. 1

    The gold standard is the primary source: the regulation text, the contract page, the published study, the official legislation site, the original spreadsheet. Confirm the number, its date, and what it measures (population, definition, market), because a correct figure about the wrong thing is still wrong.

  2. 2

    For claims drawn from an uploaded document, ask Claude to quote the exact passage word for word and give the section and page it relied on, then open that page and read it. Direct quotes make the claim auditable; reading the source is the verification.

  3. 3

    For claims from web search or Research mode, citations are pointers, not proof. Open each cited source and confirm it says what the summary says; the source can be out of date, about a different population, or misread. The help centre describes research citations as "easy-to-check", which is an invitation to check them.

  4. 4

    For unsourced claims, run a fresh search or research query to find a primary source. If none can be found, remove the claim. Anthropic's guidance: have Claude find a supporting quote for each claim and retract any claim it cannot support.

  5. 5

    For numbers Claude computed, ask which cells, columns and years it used, then recompute independently (a spreadsheet formula, a calculator). Code execution removes arithmetic error but not wrong-input error (wrong column, wrong start year, mixed currencies).

  6. 6

    A second query in a fresh chat, or best-of-N comparison, is a useful cross-check for consistency: disagreement between runs flags a problem. Agreement does not prove correctness, because both runs can share the same error, so it supplements primary-source verification rather than replacing it.

  7. 7

    Asking Claude "Are you sure?", "Can you confirm that clause exists?" or "Rate your confidence from 1 to 10" in the same conversation is not validation. Leading questions invite reaffirmation, and self-rated confidence does not track accuracy (exam guide sample 1).

  8. 8

    Turning on web search and asking Claude to confirm its own citation confirms that a document exists, not that the cited section says what the draft claims. You still have to read the passage.

  9. 9

    Repetition is not confirmation: several websites echoing the same statistic often trace back to one misquoted or outdated origin. Find the origin.

  10. 10

    Softening an unverified claim ("roughly a fifth", "reports suggest") does not make it true. Either source it or remove it.

  11. 11

    Fact-checking has a competence boundary. You can verify that a statute section exists and what it literally says; whether it permits a fee is a legal interpretation. When the remaining question is beyond your expertise, remove the claim or route it to a qualified reviewer.

  12. 12

    Claude's own limitation notice applies: even with these techniques hallucinations are reduced, not eliminated. Always validate critical information, especially for high-stakes decisions.

Test yourself on Apply fact-checking and validation techniques

Ten questions, with the answer and explanation after each one.