Idea

Retrieving the right fragment and travelling to its neighbors are two distinct capabilities

Info

Originally written in French. Translated by AI — the meaning has been preserved, not the prose.

Main idea

A vector search engine plugged into a code repository answers "where is the rule that refuses to let a cancelled item be edited?" perfectly well. It brings back the fragments closest to the question, including those that use none of its words, and it does so as well in a large mass of documentation as in a sizeable repository.

It retrieves. It doesn't travel. And yet the next question is almost always a question of neighborhood: which module calls this one, which other rule bears on the same object, which event triggers the one you have just read, which screen exposes this behavior. The engine doesn't answer that, and it isn't a matter of tuning: its metric is the closeness in meaning between a fragment and the question asked, never the relation between two elements of the corpus.

Two capabilities cover that move, and they don't look alike. A semantic graph explicitly connects concepts, modules and events, and lets you go from a node to its declared neighbors. Walking the filesystem — listing the directory where the fragment was found, opening the adjacent files, reading the local structure — is cruder and often more effective, because the tree already carries part of the relation without anyone having had to declare it.

So a system built on search alone gives exact answers about what you knew to ask, and stays silent about what sat right next to it.

Why it matters

This explains a common failure: plugging a model into a Git repository with a vector index and finding that the answers stay superficial. The problem isn't the quality of the search, it is that only one of the two capabilities is present.

It also gives an evaluation criterion that doesn't bear on the first result. A querying system isn't judged on the relevance of the fragment it brings back, but on what it can do next: reach what the person asking didn't know they had to ask for.

Nuances and limits

The distinction isn't a hierarchy. On a small or very homogeneous corpus, retrieving is enough: the neighbors are in the same file, and the move happens as you read.

And travelling never replaces retrieving. A graph with no entry point is useless; it is the search that supplies the starting node, and a bad entry sends you exploring a neighborhood unrelated to the question.

Lastly, the two capabilities don't age the same way. A file tree is always up to date, since it is the repository; a graph of declared relations is one more artifact, and it diverges as soon as the code moves without the graph being rebuilt.

Open questions

  • What determines the stopping point of a walk through the neighborhood — how many hops before the context brought back stops illuminating the question and starts drowning it?
  • On a repository that changes every day, at what point does rebuilding a graph of relations cost more than it gains against a simple walk through directories?