In this guide · 7 sections
Short answer
Start with better prompts (clear instructions and examples). If the AI needs your facts, add RAG, which retrieves your documents at question time. Consider fine-tuning only for consistent format or style at high volume, after prompts and RAG fall short. Most SMEs never need it.
The three options at a glance
Each approach changes something different. Mixing them up is how businesses end up paying for a training project when a two-page instruction document would have done.
Better prompts
Start here- Clear instructions, role, tone and rules
- A few example answers (few-shot)
- Changes take minutes
- No extra infrastructure
Best for: Tone, format and rules that fit in a page or two.
RAG (retrieval)
For your facts- Searches your documents per question
- Answers can cite sources
- Update by editing documents
- Needs good content and permissions
Best for: Assistants that must know prices, policies, products or procedures.
Fine-tuning
- Trains a model on hundreds or thousands of examples
- Bakes in format or style
- Retrain to change it
- Needs evals and a data pipeline
Best for: High-volume, narrow tasks where prompts and RAG are not consistent enough.
Our RAG explainer covers how retrieval works in detail.
What fine-tuning actually does (and does not do)
Fine-tuning takes an existing model and trains it further on example inputs and outputs. OpenAI's model optimization guide lists its benefits over prompting alone: you can give more examples than fit in one prompt, use shorter prompts that cost less at scale, and train a smaller, cheaper model to do one task well. It lists typical uses such as classification, nuanced translation, a specific output format and fixing instruction-following failures.
Notice what is missing: keeping facts up to date. Fine-tuning teaches patterns, not a reliable lookup table. If your price list changes next month, a fine-tuned model still "remembers" the old one until you retrain it, and it cannot show which document an answer came from. That is the job RAG does.
Why most small businesses need RAG and instructions, not training
Look at what an SME assistant usually has to do, and which method fits:
| What you want | Best fit | Why |
|---|---|---|
| Answer customer questions about prices, services and hours | RAG | Facts change; answers should cite the current source |
| Write in your brand tone | Prompt with examples | A style guide and 3 to 5 sample replies usually work |
| Follow rules (never quote discounts, escalate complaints) | Prompt plus testing | Explicit instructions, checked with test questions |
| Answer from staff handbook and SOPs | RAG with permissions | Large, changing documents; access must be controlled |
| Classify thousands of emails or tickets a day into fixed categories | Prompt first, fine-tuning if volume and accuracy demand it | Narrow, repetitive task where a smaller tuned model can be cheaper |
| Extract fields from invoices into a fixed format | Prompt with structured output | Modern models handle schemas well without training |
Google's Vertex AI tuning overview gives the same advice: start with prompting to find the best prompt, then move to fine-tuning only if needed to boost performance or fix recurring errors, and only with high-quality, well-labelled data.
A simple decision path
Work through these in order and stop at the first step that gets you a reliable result.
Write down 20 to 30 real test questions
Use real customer or staff questions with the answers you expect. Without a test set, you cannot tell whether any change helped.
Improve the prompt
Add a clear role, rules, tone guide and a few example answers. Re-run the test set.
Try a project or assistant with your files
If your material is small, a ChatGPT or Claude project with uploaded files may be enough. Our projects comparison explains the options.
Add proper RAG for larger or changing content
Index your documents, show sources, and set up permissions and a re-indexing routine.
Consider fine-tuning only for narrow, high-volume tasks
If a consistent format or classification still fails at scale, and you have hundreds of good examples, test tuning on a platform that still offers it.
What each option costs in practice
Exact costs depend on volume, but the cost structure differs sharply:
- Prompts: mainly your time. Longer prompts add token costs on every request; see our LLM API pricing comparison.
- RAG: set-up work, plus storage and search. As an example, OpenAI's retrieval guide lists vector storage as free up to 1 GB across all stores, then US$0.10 per GB per day (as of October 2026). The ongoing cost is mostly keeping content current.
- Fine-tuning: preparing hundreds or thousands of labelled examples, training runs, evaluation, and retraining whenever requirements change. The data work usually costs far more than the compute.
Do it yourself or get help?
Prompt improvements and a project with uploaded files are well within reach of a capable staff member. A RAG assistant that customers use, or one with access to internal documents, needs care with permissions, content hygiene and testing. If you are weighing a fine-tuned or self-hosted model for privacy, see our guide to running an AI model on your own PC.
We build website assistants that answer from your approved content with sources as an AI Website Assistant (S$790), and scope internal assistants with retrieval and permissions as a Custom Tool or Dashboard (from S$1,990).
Frequently asked questions
Not in the way most people mean. ChatGPT business plans let you upload files to projects or connect company drives, which uses retrieval rather than training. OpenAI's self-serve fine-tuning API is closed to organisations that had not used it before 7 May 2026. For company knowledge, retrieval is usually the better approach anyway.
They solve different problems. Fine-tuning changes how a model behaves, such as its format or style, using many training examples. RAG gives the model your current documents at question time, so it can answer about your facts and cite sources. For business knowledge that changes, RAG is usually the right choice.
Rarely. It can make sense for a narrow, high-volume task, such as classifying thousands of messages a day into fixed categories, where prompting and retrieval are not consistent enough and you have hundreds of good labelled examples. Even then, test whether a better prompt on a newer model solves it first.
Only if your organisation was already an active fine-tuning customer. OpenAI's deprecations page says new organisations lost access on 7 May 2026, inactive users on 2 July 2026, and all customers will stop being able to create new jobs on 6 January 2027. Existing fine-tuned models remain available until their base models are deprecated.
Better prompts: clear instructions, rules, a tone guide and a few example answers. They cost little beyond your time and can be changed in minutes. If the assistant needs your facts, add your documents through a project, connector or RAG system before considering any training.