The structurally-right tool beats the trendy default

Ask most teams how they’ll let a model answer questions over a body of documents and you’ll hear the same three letters: RAG. Retrieval-augmented generation has become the reflex — the default you reach for before anyone has described the problem. That’s the tell. A default that fires before the problem is understood isn’t an answer; it’s a habit.

Defaults are chosen for you

The trendy tool is trendy for a reason — it fit someone’s problem well enough to get written up. But the write-up travels further than the conditions that made it work. By the time it reaches you it has shed its context and arrived as a recommendation. Adopting it means inheriting a decision someone else made about a problem that wasn’t yours.

RAG is genuinely good at a particular shape: many loosely-related passages, where the answer lives in one or two of them and the job is to find the right paragraph. It’s a strong fit for a sprawling, flat corpus. It is a poor fit when the answer depends on how things relate— when “which suppliers are two hops from a sanctioned entity” matters more than “which document mentions sanctions.”

Start from the shape of the problem

When the value is in the relationships, the structurally-right tool is usually a knowledge graph: model the entities and the edges between them, and the questions that were impossible to answer by retrieval become a traversal. Not because graphs are fashionable — they aren’t — but because the structure of the answer matches the structure of the data. The tool stops fighting the problem.

This is the move underneath most of what we do: decompose the problem until its shape is visible, then pick the tool whose shape matches. Sometimes that’s RAG. Sometimes it’s a graph, a small deterministic rule, a classical statistical model you can defend in a meeting, or a single well-placed prompt. The discipline isn’t knowing the trendy tool — everyone knows it. It’s being willing to not reach for it.

Why it pays

A tool matched to the problem is smaller, cheaper to run, easier to explain, and less likely to fail in ways no one can diagnose. A tool chosen by fashion carries complexity you didn’t need and failure modes you didn’t choose. The structurally-right tool isn’t the clever answer. It’s the boring one that keeps working after the demo.