Fine-tuning is not a worse version of retrieval. It is a different tool, and it
is being asked to do a job it was never shaped for.
Fine-tuning teaches a model how to behave. Retrieval supplies what is true.
Most enterprise AI disappointment starts with someone using the first to attempt
the second.
The instinct is understandable. You have twenty years of documents and a model
that does not know your business. Training it on your documents sounds like
exactly the fix. It is a reasonable-sounding idea that fails on four properties
of business knowledge, none of which is about model quality.
1. Facts change, and weights do not
A fine-tuned model knows what was true on the day the training set was
frozen.
Your pricing changes. A policy is superseded. A contract is amended. A supplier
is replaced. When knowledge lives in the weights, updating it means another
training run — and in the interval between runs, the model is confidently
asserting things that have stopped being true.
With retrieval, you re-index the changed document. The next question gets the
current answer. The gap between "the fact changed" and "the system knows" is the
ingestion lag, measured in minutes or hours rather than in training cycles.
That gap is the whole argument, and it does not close as models improve.
2. A fine-tuned answer cannot be cited
When knowledge is in the weights, there is no source to point at.
Ask a fine-tuned model what your standard payment terms are and it will tell
you. It cannot show you the clause it came from, because there is no clause — it
has an aggregate impression formed across everything it was trained on, and that
impression may blend three superseded contract templates into an answer that
matches none of them.
Retrieval produces an answer and the passage it came from. That changes what
kind of thing the output is: not an assertion to be trusted, but a claim with
its evidence attached, which a person can check in ten seconds.
Being precise about this matters. Citation does not prove the answer follows
from the passage. It makes the error checkable rather than impossible. That
is a smaller claim than the one usually made for RAG, and it is the true one —
and for business use it is worth far more than a confident, unsourced answer.
3. Fine-tuned knowledge cannot be access-controlled
This is the one that should end the conversation in most organisations.
Your document estate has permissions. Some people can see the board pack, the
salary bands, the legal advice, the acquisition model. Train a model on all of
it and those distinctions dissolve — the weights do not remember who was allowed
to read what, and the model will answer anyone who asks.
You cannot retrofit entitlement onto a set of weights. You can filter the
questions, which is a guess, or filter the answers, which is a different guess.
Neither is an access-control boundary.
Retrieval keeps the boundary intact, because the entitlement travels with the
document. Permissions are captured at ingestion and enforced at retrieval, so a
user's candidate set is filtered before anything reaches a model. The model
never sees material the user was not entitled to see, so it cannot leak it.
4. Fine-tuned knowledge cannot be deleted
"Remove this document" is a routine request. Under fine-tuning it has no
routine answer.
A contract ends and its terms should no longer inform anything. An employee
exercises a data right. A document turns out to have been indexed in error, or
to be subject to a legal hold. In a retrieval system, you remove it from the
index and it stops influencing answers immediately.
In a fine-tuned model, the influence is distributed across the weights. There is
no delete. There is retraining without it — expensive, slow, and in practice
deferred until it becomes several deletions batched into an annual event.
Any organisation with retention obligations should read that paragraph twice.
So what is fine-tuning good for?
Behaviour, format and domain vocabulary — not facts.
There are real uses, and being dismissive about them would be its own kind of
wrong:
- Output format. Reliably producing a specific structure, tone or style
without spending context on instructions every request. - Domain vocabulary. Adapting a model to specialised language it handles
awkwardly — a technical field, an underserved language, an internal shorthand. - Task shape. Teaching consistent behaviour on a narrow, repetitive task
where the pattern matters more than the content. - Latency and cost at scale. A small fine-tuned model doing one narrow job
well can be much cheaper than a large general model doing it adequately.
Notice what every one of those has in common: none of them is a fact that will
change next quarter.
The two together
The pairing is not a compromise, it is the design. A model tuned for how your
organisation writes and reasons, answering over documents retrieved fresh with
their entitlements intact and their sources attached.
Behaviour in the weights. Facts in the index. When the facts change you re-index;
when the behaviour needs changing you tune. Two different maintenance rhythms,
which is exactly right, because they are two different problems.
The question to ask before anyone proposes fine-tuning
"When this fact changes, what has to happen — and who does it?"
If the answer involves a training run, you are about to encode your business's
current state into an artefact that cannot be updated, cited, scoped or deleted.
Ask it early. It is a much cheaper question than the project it prevents.