Don't patch the question: fix the step that broke
Think of a spelling test
A teacher marks one wrong word. You could memorise just that word. But the next test has new words, and you get those wrong too. The better plan is to learn the spelling rule.
AI teams make the same mistake. A test question fails. Someone adds a rule that mentions a word from that question. The test now passes. Ten rules later, nobody understands the system, and the next ten questions fail anyway.
A small example
Someone asks: "How do I reset the printer?" The assistant gets it wrong. A bad fix: "If the question has the word printer, show page 12." Tomorrow someone asks about the scanner, and it fails. So you add another rule. And another.
A good fix starts with a question: why did it fail? Maybe the right page calls it a "power cycle", not a "reset", so the search never found it. Teach the search about that, and printer, scanner and many other questions start working at once.
Ask which step broke
An assistant answers in steps. Each step can fail in its own way:
- Step 1, find the pages. It never found the right page.
- Step 2, put the best first. It found the right page, but ranked it too low.
- Step 3, pick the right part. It looked at the wrong part of the page.
- Step 4, check the answer. It never checked the answer against the page.
Each step has its own proper fix, a fix to the design of that step, not to one question.
A rule of thumb
A fix should never name a special word or a special question. If you catch yourself writing one, stop. Ask: what could the system use instead? And after the third patch in the same place, change the design.
Keep a notebook of what you tried
Keep a simple table: what you tried, what did not work and why, and which ideas work together. Put the ideas that give the most gain for the least cost at the top. A notebook like this teaches a team faster than any meeting. It also stops the same failed idea from coming back.
The limits of this advice
- Sometimes a quick patch is the right choice, for example during an urgent problem. Mark it as temporary, and plan the real fix.
- Finding the broken step takes some detective work and good test questions.
- A fix to one step can change the others, so run all your tests again after each change.
Words explained
- Assistant: an AI helper you ask questions.
- Patch: a quick fix that covers one problem without fixing its cause.
- Ranking: putting the best answers at the top.
- Keyword: a special word the system looks for.
- Test question: a question with a known right answer, used to check the system.
For engineers
This section has the exact details. Everything above is the simple version of it.
- Failure layers. A wrong answer comes from one of a few causes: the right passage was never retrieved (recall), it was retrieved but ranked too low (ranking), the wrong part of the passage was chosen, or the answer was not verified against the evidence.
- Structural fixes. Examples are where a term is first introduced, what kind of section it is, and how scores are blended. A fix must not name a term or a question.
- Trade-off table. Record the levers tried, what did not work and why, and which combinations do work, ordered by gain per cost.
- Where it fits. See evaluation engineering in Chapter 12.