Fine Tuning LLMs on OCI: Practical Walkthrough

Fine tuning adjusts a pretrained model's weights using your examples, so the model behaves your way by default instead of being talked into it on every request. Done for the right reasons it produces shorter prompts, lower latency, consistent formatting, and command of a domain's voice. Done for the wrong reasons it produces an expensive model that knows nothing your documents did not already say, ages badly, and still hallucinates. This article, part of our complete guide to AI on OCI, is the practical path through the decision and the build.

First, the honest test: do you need fine tuning at all

The escalation ladder has three rungs, and each is an order of magnitude cheaper than the next. Prompting comes first: system instructions, few shot examples, and output schemas solve formatting, tone, and basic task behavior more often than teams expect. Retrieval comes second: if the problem is that the model does not know your facts, the answer is grounding it in your documents, the pattern covered in our reference RAG architecture on OCI, because facts change and weights should not be your database. Fine tuning is the third rung, and it earns its place for a specific set of jobs: enforcing a style or format that prompting holds only loosely, teaching a specialized vocabulary or task pattern the base model fumbles, compressing long prompts into trained behavior to cut latency and token spend at volume, and lifting the quality of a small fast model toward what a large one does at several times the price.

The test is simple to state. If a clearly written prompt with five good examples fixes it, prompt. If the failure is missing knowledge, retrieve. If the failure is behavior that survives good prompting and good retrieval, tune. And note the combination, because it is the production sweet spot: a tuned model for behavior, RAG for facts, each doing the job the other does badly.

The two paths on OCI

OCI gives you a managed lane and a self managed lane, and the choice is mostly about control versus operational burden.

The managed path, described more broadly in our overview of the OCI Generative AI service, removes the infrastructure problem entirely: no drivers, no distributed training debugging, no serving stack. Its constraint is menu depth, you tune what the service supports, the way the service supports it. The self managed path on the accelerators covered in our GPU shapes guide gives you the whole open model ecosystem and full methodological control, and in exchange you own a training pipeline as a production system. The honest default for most enterprises is managed first; go self managed when a requirement, not a preference, forces it.

The part that decides everything: your dataset

Every fine tuning effort succeeds or fails on the training set, typically prompt and completion pairs that demonstrate the behavior you want. Four properties matter more than volume. Quality: every completion should be an example you would be happy to see the model produce verbatim, because it will. Consistency: if your examples disagree about format or judgment, the model learns the disagreement. Coverage: include the awkward cases, the refusals, the edge inputs, not just the happy path, or the model will improvise precisely where you can least afford it. Hygiene: nothing confidential that the model should not later say, no personal data that has no business in weights, and a provenance note for every source, the governance questions explored in governing AI workloads on OCI.

On size: hundreds of excellent examples routinely beat tens of thousands of scraped ones for behavioral tuning with parameter efficient methods. The expensive part is not GPU time, it is the subject matter experts' hours writing and reviewing examples. Budget for that honestly and the rest of the project tends to behave.

The walkthrough

Choosing the base model to tune

The base model decision deserves more thought than it usually gets, because tuning amplifies what is already there rather than repairing what is not. Three rules of thumb hold up across projects. Tune the smallest model that can plausibly do the job, because the economic case for tuning usually rests on serving something small and fast, and a tuned mid size model that matches a frontier model on your narrow task is the textbook win. Prefer a base model whose untuned behavior is already close, since closing a small gap takes hundreds of examples while closing a large one takes a research team. And check the licensing twice for open weight models: some licenses carry conditions on commercial use or on training derivatives that legal will want to have seen before the model is in production rather than after. On the managed path the catalog constrains the choice for you; on the self managed path the freedom is real and so is the obligation to record why this checkpoint, this version, this license, a record that slots into the model register described in our governance article.

What it costs, roughly and honestly

Three buckets, in ascending order of surprise. Training compute is the smallest and most visible: parameter efficient tuning jobs on managed clusters or a few hundred GPU hours self managed, bounded and one time per iteration. Serving is the bucket that compounds: a tuned model on the managed service needs a dedicated hosting cluster with committed unit hours, and a self hosted model needs GPUs that run around the clock, so a tuned model must displace enough prompt length, latency, or large model spend to pay rent that never stops. People are the bucket nobody budgets: expert hours building the dataset and grading evaluations usually exceed the compute bill. Run the displacement math before you start, per the discipline in controlling AI workload costs on OCI, and fine tuning becomes an investment case rather than an act of faith.

Failure modes we keep seeing

Tuning facts instead of behavior, then wondering why the model confidently recites last quarter's prices: facts belong in retrieval. Skipping the baseline, which makes the after picture unfalsifiable and the project unkillable. Overfitting on a narrow dataset until the model nails the test set and fumbles everything adjacent. Treating the tuned checkpoint as finished software with no retune budget. And the quiet one: deploying a tuned model without regression testing its refusal behavior, then discovering the safety habits of the base model did not all survive the trip. Every one of these is prevented by the sequence above, which is the point of having one.

The independent view

Fine tuning on OCI is no longer exotic. The managed path makes the mechanics a configuration exercise, and GPU shapes make the deep end available when it is justified. What separates the production wins from the abandoned checkpoints is everything around the training job: the honest baseline, the dataset discipline, the displacement math, and the retune cadence. That surrounding system is what we build through our OCI implementation practice, on a fixed Project fee, with Managed Monthly retainers keeping the evaluation and retune loop running after go live. Bring us the use case and the skeptical finance partner; both will be put to work.