Online Tool Store
· 4 min read
How to Estimate What a Fine-Tuning Run Costs
Manesh Jayawardhana
CIO & Co-founder
Before committing to a fine-tuning run, someone will ask what it costs. The arithmetic is straightforward, the inputs are usually estimated badly, and the more useful question sits behind it.
The calculation
Training cost is driven by total tokens processed:
total training tokens = examples × average tokens per example × epochs
Four thousand examples averaging 850 tokens is 3.4 million tokens per pass. Three epochs processes 10.2 million. Multiply by your provider’s published rate per million training tokens and you have the figure.
Two things to be careful about.
Tokens, not words. A token is roughly three-quarters of a word in English and considerably less for code, other languages, or text with a lot of punctuation and formatting. Estimating from word count and multiplying by a rule of thumb is fine for a rough figure and wrong enough to matter for a budget. Use a tokeniser for the actual number.
Epochs multiply linearly. Doubling epochs doubles the cost exactly. That’s a straightforward relationship and it makes epoch count the easiest lever to get wrong in an expensive direction.
Why the rate is an input, not a constant
Provider pricing changes, differs by model, and differs between training and inference. A figure hard-coded into a calculator goes stale silently, and a stale number in a budget is worse than no number.
Take the current rate from the provider’s own pricing page at the moment you’re estimating, and note the date alongside the estimate.
Also remember training cost is not the whole cost. A fine-tuned model has its own inference pricing, which is often higher than the base model’s, and that recurring cost usually exceeds the one-off training cost within weeks of production use. Budget both.
| Cost | Nature | Usually |
|---|---|---|
| Training tokens | One-off per run | Scales with dataset × epochs |
| Failed or repeated runs | One-off, often forgotten | 1-3 extra runs is normal |
| Inference on the tuned model | Recurring | Frequently the larger total |
| Evaluation runs | Recurring during development | Small but not zero |
The question behind the question
Fine-tuning is frequently the wrong tool for the problem people bring to it.
It’s genuinely good at teaching a model a consistent output format, a specific tone, or a narrow classification task where you have plenty of labelled examples.
It’s a poor fit for giving a model knowledge. Facts change, and a fine-tuned model’s knowledge is frozen at training time. Retrieval over your own documents handles that better, cheaper, and updates without retraining.
It’s also often unnecessary for behaviour that better prompting or a handful of in-context examples would achieve. That’s worth genuinely testing first, because the iteration loop on a prompt is minutes and on a fine-tune is hours plus money.
Common mistakes to avoid
- Estimating tokens from word count rather than tokenising.
- Budgeting one training run when two or three is normal.
- Forgetting that inference on the tuned model has its own ongoing cost.
- Running more epochs than the evaluation justifies, which costs linearly and risks overfitting on a small dataset.
- Fine-tuning for knowledge that retrieval would serve better and keep current.
How to do it with Fine-Tune Cost Estimator
The Fine-Tune Cost Estimator does the arithmetic with rates you supply.
- Count your training examples and their average token length.
- Set the number of epochs — start low.
- Enter the current rate from your provider’s published pricing.
- Add a margin for repeated runs, and estimate the ongoing inference cost separately.
Other budget calculators are in the tools directory.
Frequently asked questions
Why does it ask me for the rate?
Because provider pricing changes and differs by model, and a figure hard-coded into a calculator goes stale quietly. Take the current number from the provider’s own pricing page.
How many epochs do I need?
Fewer than people expect. More epochs increase cost linearly and risk overfitting on a small dataset. Start low, evaluate, and add only if the evaluation says to.
Is fine-tuning the right approach at all?
Often not. Better prompting, few-shot examples, or retrieval over your own documents solve many problems that look like fine-tuning problems, at a fraction of the cost.
Final thought
Estimate the inference cost alongside the training cost. The training run is the number people ask about; the inference bill is the one that arrives every month.