RAG became the default answer to a simple question: how do you get an LLM to talk about things it was never trained on, using data that changes every day? Most teams implement the same recipe. Split documents into chunks, stuff them into a vector store, run a similarity search on user queries, feed the top few chunks into the prompt, hope hallucinations go away.

The "chatbot" metaphor was useful at the beginning. It let people map a strange capability onto something familiar: a text box, a reply, a back-and-forth. As soon as teams tried to build serious systems on top of that metaphor, they hit the wall. A chatbot is a UI. A modern LLM stack is closer to a programmable runtime.

Most people still picture a "large model" as one big uniform block: same layers, same weights, every token marching through the same path. You want more capacity, you make the block bigger. You pay almost linearly in memory, compute, and power. That picture breaks the moment you try to push capacity far beyond what you can afford to run for every single token.

Most teams wait too long before thinking about compression. They train or adopt a big model, get excited by its benchmark numbers, ship a prototype, then hit the wall: GPU bills, latency, memory limits, mobile deployment, compliance constraints. At that point, "make it smaller" becomes an urgent request, not a design choice.

"Let's just fine-tune it" is one of the most expensive sentences in this field. People use the same phrase to describe wildly different things: nudging tone, injecting domain knowledge, fixing safety issues, matching a house style, or building a brand-new capability on top of a base model. Under the hood, those goals map to different techniques, different data requirements, and very different risk profiles.