Conduct structured discovery and requirement gathering
Run structured discovery that captures the current process, volumes, error costs, data sources and sensitivity, measurable success criteria, constraints and decision rights before proposing a model or architecture.
Key points
- 1
Discovery comes before any model, pattern or prompt decision. A request such as "build us a Claude chatbot" is a symptom, not a requirement; the architect's first job is to find out what the business actually does today and what it is trying to change.
- 2
Cover the standard checklist: the current process step by step, who performs it, volumes and peaks, turnaround times, how often errors occur and what each error costs (money, time, customer or regulatory impact), the data sources involved, and existing systems the solution must fit into.
- 3
Classify data sensitivity during discovery, not during build: what personal, financial, health or confidential data is in scope, whether it may leave the organisation's boundary, what must be redacted or tokenized, and which owner (compliance, legal, security) signs off. A late discovery of card numbers or PHI is a reason to extend discovery and bring in the missing stakeholder.
- 4
Establish decision rights: who funds, who approves, who accepts the result, and who can say no. Conflicting stakeholder positions (support wants full automation, legal wants review of everything, finance wants a cap) are usually resolved by discovery data, such as error cost by segment, rather than by picking a winner.
- 5
Leave discovery with agreed, measurable success criteria that are specific, measurable, achievable and relevant. "Classify sentiment well" is vague; "F1 of at least 0.85 on a held-out set of 10,000 posts, a 5-point improvement over the current baseline" is a criterion. Most use cases need several criteria (task fidelity, consistency, tone, privacy, latency, price).
- 6
Establish a baseline before agreeing a target. If nobody has measured the current manual error rate, measure it on a sample first; an accuracy figure with no baseline cannot be judged achievable, and a blanket figure across fields of unequal importance should be weighted by the cost of each error.
- 7
Error cost is the discovery fact that most shapes architecture: it sets the acceptance threshold, decides where humans stay in the loop, justifies spend on accuracy, and determines whether a single call, a review queue or a richer workflow is appropriate.
- 8
Discovery outputs are a current-state description with numbers, success criteria with a baseline and an accepting owner, a data and sensitivity inventory, known constraints (budget, deadline, regulation, integration), and a named decision-maker. Prompts, embedding and chunking choices, SLAs and demos belong to later phases.
- 9
Distractors the exam uses: jumping to a solution or pattern before discovery, building a demo to "get a reaction" first, asking the business which model it wants, treating budget and timeline as the only constraints, accepting a sponsor's accuracy number without a baseline, and countering with a generic industry figure.
- 10
Do not over-promise during discovery. Committing to an accuracy guarantee or an SLA before anything has been measured undermines every later conversation.
Test yourself on Conduct structured discovery and requirement gathering
Ten questions, with the answer and explanation after each one.