Main idea

Indexing all of a product's code in a meaning-based search engine is devastatingly effective for finding a rule buried in the middle of a file, with nothing in its name to flag it. That is what these engines do best, and no enumeration replaces it.

But an engine of that kind always answers, and it answers with a ranking: here are the ten passages closest to your question. It never says "there aren't any". So a search that brings back nothing interesting can mean two things — the thing doesn't exist, or the question was badly asked — and nothing lets you tell which.

Hence the order: enumerate first, search second. What is certain comes first, what is probable comes as a complement. The other way round, you get a description made of whatever the engine chose to bring back.

Why it matters

This is the difference between an instrument of recall and an instrument of inventory, and it decides what you are entitled to conclude. Mistaking one for the other produces work that looks exhaustive while its coverage in fact depends on how the questions were phrased.

It holds for any device that ranks instead of counting — document search, assistant answers, recommendation: none of them flags what it failed to find.

Nuances and limits

Two details cost a lot in practice and have nothing to do with the principle: an index has a date of its own, often one step behind the source it claims to cover; and the question has to be asked in the language of the corpus — a query in French against code written in English gives mediocre results even when the answer is in there.

The rule doesn't forbid starting with a search to get a feel for the ground: it forbids drawing a conclusion of absence from it.

Open questions

• What would an engine that could answer "there is nothing" look like, and what enumeration would it have to be given for that to be possible?