Document architectures and provide implementation guidance

Key points

1. 1

   A complete handoff package contains: architecture diagrams at the right levels, prompt and output-schema versions, evaluation results and the harness that produced them, runbooks, a cost model, and ownership (a RACI). Each item serves a specific stakeholder; missing one leaves that stakeholder unable to do their job.

2. 2

   Layer diagrams by audience, C4-style: a system-context view for sponsors and executives, container and component views for engineers, and sequence or flow views where the logic matters (retries, routing, escalation). One 40-box diagram for everyone serves no one; prose is not a substitute.

3. 3

   Runbooks are what make a system operable: known failure modes, symptom-to-diagnostic mappings, escalation paths, and a tested procedure for rolling back a prompt or model version. Diagrams and ADRs explain the system; runbooks and version records let people recover it at 2 a.m.

4. 4

   Prompts are versioned artifacts. Keep them in version control with a changelog and attach the evaluation results obtained with each version, so maintainers can see what changed, what it did to quality, and what to roll back to. Shared editable documents and live console edits leave no usable record, and the model does not know which version it is running.

5. 5

   Output schemas carry a version number and a list of downstream consumers per field, so changes can be assessed for impact and coordinated.

6. 6

   Record the prompt and model version used for each request in logs. That per-request traceability is what audit needs to show which prompt and model produced a given output, and it is what a rollback targets.

7. 7

   A cost model ties token usage per request type to expected volumes, with an expected range and alert thresholds. It is what gives a sponsor predictable monthly cost and lets operations notice a doubling of spend before the invoice does.

8. 8

   Implementation guidance for another team means interface contracts for inputs and outputs, the evaluation harness with acceptance thresholds, latency and cost budgets, the constraints and decisions already made (what not to reopen), and named owners. "Use Claude with retrieval and well-crafted prompts" is not guidance; neither is a proof-of-concept notebook, a standing daily call, or writing the code yourself.

9. 9

   Ownership: name who owns the prompts, the evaluation suite, the cost model, and the response to model deprecation. Systems built by consultants and handed over without owners decay.

10. 10

    Distractors the exam uses: demo slide decks, vendor benchmark summaries, raw discovery transcripts, evaluation data without a scoring method, detailed sequence diagrams as a substitute for traceability, and rationale documents (why JSON was chosen) when the need is operational (why JSON broke yesterday).

Read the source

• Develop test cases
• Define your success criteria