4.1 Design prompts with explicit criteria to improve precision and reduce false positives
Write prompts whose criteria are explicit and categorical, so automated review and extraction report the right things, skip the rest, and classify consistently, instead of relying on vague adjectives or confidence hedges.
Key points
- 1
Explicit criteria beat vague instructions. "Flag a comment only when the behaviour it claims contradicts what the code actually does" is testable; "check that comments are accurate" leaves the model to decide what accurate means.
- 2
General hedges such as "be conservative", "be careful", or "only report high-confidence findings" do not improve precision. They do not define the target, so the model keeps finding the same things and merely reports fewer or more of them at random.
- 3
Confidence-based filtering (asking for a score and dropping findings below a threshold) tends to remove real findings the model was unsure about while keeping confidently wrong ones. Fix the criteria, not the threshold.
- 4
Write report-versus-skip lists: name the categories to report (logic bugs, security issues, broken error handling) and the categories to skip (naming, formatting, patterns already used locally in the file). Include thresholds where numbers are involved ("differs by more than 1%").
- 5
False positives in one category erode trust in every category. Developers who dismiss noisy style findings soon stop reading security findings too.
- 6
When a category has a high false-positive rate, temporarily disable it, keep the accurate categories flowing, rewrite the disabled category with explicit criteria, measure it on a sample, and only then re-enable it.
- 7
If a required category has systematic false positives with an identifiable pattern (for example, string-built SQL in test fixtures with constant values), add that pattern as an explicit skip condition rather than disabling the category or adding a confidence hedge.
- 8
Severity must be defined, not described with adjectives. Give each level concrete criteria and a code example (critical = data loss, security exposure or a production-path crash; low = no runtime impact) so the same defect gets the same severity in every PR.
- 9
Emphasis (capitals, "MUST", "IMPORTANT") is not a criterion. Neither is temperature,
max_tokens, or a larger model tier; none of them tells the model what to look for. - 10
Majority voting across repeated runs does not remove systematic false positives, because every run applies the same misunderstanding; it only suppresses intermittently caught real issues.
- 11
Rationale fields make inconsistent judgements easier to audit but do not make them consistent. Path-based severity rules ignore what the defect actually is.
- 12
The same principle applies to extraction flags: replace "flag suspicious invoices" with the concrete conditions (tax ID mismatch, line items not summing to total, duplicate invoice number).
Read the source
Test yourself on 4.1 Design prompts with explicit criteria to improve precision and reduce false positives
Ten questions, with the answer and explanation after each one.