What is fine-tuning?

At a glance

• You don’t build from scratch. It starts from an expensive finished model (GPT, Llama) and just nudges it^[1] — like sending an experienced generalist on a short specialty course.
• It changes HOW the model answers (tone, format, behavior), not WHAT facts it knows. For changing facts, connect it to your documents (RAG) instead.
• It needs curated example pairs — typically hundreds to a few thousand. Quality beats volume; bad examples teach bad habits.
• Reach for it last. Most business goals are met by cheaper options first^[5].

When to use it

Follow the cheaper-first rule: write better prompts, then add document retrieval (RAG) for your facts, and fine-tune only when you need a consistent style or behavior those two can’t deliver^[2]. It pays off on narrow, repetitive, high-volume tasks where a locked-in voice or format saves real money and removes long instructions from every prompt^[6].

The hidden costs

Pushing a model toward narrow examples can make it worse at general tasks — called catastrophic forgetting^[4]. A custom model is also yours to maintain: when the base model upgrades, you may need to re-tune and re-test. Lightweight methods like LoRA adjust only a tiny slice of the model, cutting cost and reducing forgetting — the practical default today^[3].

Bottom line

Fine-tuning is a focused upgrade, not a from-scratch build — the expensive last resort after prompting and retrieval, made practical by LoRA.