Communicate architectural decisions and trade-offs

Key points

1. 1

   Translate technical metrics into business consequences for the people who fund and approve the work: cost per case, turnaround time, expected error rate and the cost of an error, exposure and risk. A CFO cannot act on benchmark scores, context-window sizes or tokens per second.

2. 2

   Show the options, not just the winner. A recommendation is defensible only when the alternatives and what each trades away (accuracy vs cost vs latency vs risk) are visible. Hiding rejected options may speed approval but the trade-off resurfaces later and costs trust.

3. 3

   Always make a recommendation. Presenting numbers without one abdicates the architect's role; decision rights belong to the sponsor, but the sponsor is paying for a professional view.

4. 4

   Quantify the trade-off where you can: accuracy difference times volume times error cost, compared with the cost difference between options. A 3-point accuracy gap on 200,000 monthly requests at $40 per error is $240,000 a month, which dwarfs a $30,000 model-cost delta. State the assumptions (error cost, volume) so stakeholders can challenge them.

5. 5

   Be explicit about residual risk after mitigation and who owns it: the expected rate of outputs needing correction, what happens when the system is wrong, and the review or escalation path.

6. 6

   Never present a prompt as a guarantee. "The system prompt prevents incorrect claims" is over-promising; prompts reduce error rates, they do not eliminate them.

7. 7

   Do not present curated demo output as representative of production quality. Ten hand-picked examples show feasibility; they say nothing about the error rate across 40,000 SKUs.

8. 8

   Record decisions in architecture decision records (ADRs): context, options considered, decision, consequences and status, kept with the project. When context changes, write a new ADR that supersedes the old one rather than editing history. Chat threads, code comments and slide decks do not survive staff turnover.

9. 9

   When a stakeholder objects to a control such as human review ("it makes the AI look like it doesn't work"), reframe it as risk management: show where the error cost concentrates, present the review rate as a dial, and publish the evidence-based criteria for turning it down.

10. 10

    Recommend the simplest option that meets the agreed criteria within the constraints, and reserve more complex architectures (agents, multi-agent) for when measurement shows they are needed. A hard deadline and an agreed acceptance threshold are usually the discriminators between defensible options.

11. 11

    Distractors the exam uses: presenting only the recommendation, technical specification tables for executives, live demos as decision tools, "the most capable model is the safest", promising 100% after fine-tuning, prompt wording as a control, and recommending the most complex option because it has the best headline number.

Read the source

• Define your success criteria
• Building effective agents