An embedding model converts text into a vector of numbers such that similar meanings land near each other in vector space. Everything a retrieval system does, the search, the filters, the grounding that keeps a chatbot honest, happens in the geometry that this one model defines. Pick a model that understands your domain and retrieval feels like magic; pick one that does not and no amount of clever orchestration upstream will find what the geometry cannot see. This article, part of our complete guide to AI on OCI, treats the choice with the seriousness its switching costs deserve.
Why this choice is stickier than every other model choice
Vectors produced by different embedding models are not comparable, not even approximately, not even between versions of the same model family. The moment you embed a corpus, the model becomes part of the data. Changing it means re embedding every chunk, rebuilding every index, and re validating retrieval quality, a project measured in days of pipeline time for small corpora and weeks for large ones, with compute bills attached. Queries face the same constraint: the question must be embedded by the same model that embedded the documents, forever, which is why the model belongs in your configuration management and your model governance register with a version pin and an owner. The practical rule: treat the embedding choice like a schema decision, not a library import.
The three places an embedding model runs on OCI
| Path | How it works | Strengths | Watch for |
|---|---|---|---|
| OCI Generative AI service | Hosted embedding models, Cohere's Embed family, called per request | No infrastructure, strong general quality, multilingual coverage | Per token cost at corpus scale, catalog limits your choice |
| Inside the database | ONNX format models imported into Oracle Database 23ai, embedding generated next to the data | Data never leaves the database, SQL native pipeline, tight governance | Model size limits, database CPU shares the work |
| Self hosted on GPU shapes | Open models served from your own instances or OKE | Any model you can license, fixed cost at volume, full control | You own serving, scaling, and patching forever |
The managed service is the right default for teams already using the OCI Generative AI service for generation: one vendor surface, one bill, models tuned for retrieval out of the box. The in database path is the quiet star for Oracle estates running 23ai vector search, because embedding where the data lives eliminates both an export pipeline and a security review finding in one move. Self hosting earns its keep in two situations: corpus scale so large that per token pricing loses to a GPU doing nothing but embedding, or a domain specific open model that measurably beats the hosted catalog on your data. Mixed answers are legitimate; mixed answers nobody wrote down are how the same corpus ends up embedded twice by two different models, which is the expensive way to discover the comparability rule.
The criteria that actually matter
Retrieval quality on your domain
Public benchmarks rank models on general corpora, and the rankings compress: the top tier models are close on averages. Your corpus is not an average. Legal boilerplate, Oracle ERP field descriptions, support tickets full of product codes, these are exactly the texts where general rankings mispredict, which is why the evaluation framework below outranks every leaderboard.
Dimensions, and what they cost
A model's output dimension multiplies your storage and your query compute. A million chunks at 1024 dimensions of four byte floats is roughly four gigabytes of raw vectors before index overhead; at 384 dimensions it is a third of that, with faster similarity math at query time. Several modern models support shortened output dimensions or quantized storage with modest quality loss, and at tens of millions of vectors those options stop being exotic and start being the budget. Size the math before choosing, not after, alongside the broader spend discipline in controlling AI workload costs on OCI.
Multilingual coverage
If your documents or your users cross languages, a multilingual embedding model puts equivalent meanings near each other across languages, and a monolingual one quietly does not. This is a binary requirement: discover it before the corpus is embedded, because no reranker repairs a geometry that separated English from German at the foundation.
Context window and chunking fit
The model's input limit constrains your chunk size, and chunks that get truncated silently lose exactly the sentences someone will eventually search for. Match the chunking strategy from our reference RAG architecture to the model's real limit, with room to spare for the metadata you prepend.
Licensing and residency
Hosted models inherit the service's data boundary; open models inherit their license terms. Both are governance facts that belong in the decision record, not discoveries for the audit.
The evaluation framework
- Build a golden set from real questions. Fifty to a hundred genuine queries, each mapped to the documents that answer them. An afternoon with the help desk logs beats a week of synthetic guessing.
- Shortlist three candidates across the paths. One hosted model, one in database option, one open model, chosen for plausible domain and language fit.
- Embed the same corpus sample with each. Identical chunks, identical metadata, three parallel indexes. Keep the pipeline scripted; you will run it again someday.
- Measure recall at k on the golden set. Does the right document appear in the top five and top ten results. This single number predicts production behavior better than any published benchmark.
- Price the steady state. Initial corpus embedding, monthly re embedding of changed documents, query embedding at projected volume, and vector storage at each dimension count.
- Check latency where it hurts. Query embedding sits in the user's critical path; measure it from your VCN, not from a benchmark blog.
- Pick, pin, and write the exit plan. Version pin the winner, record the runner up, and script the re embedding pipeline now. The migration you prepared for is a maintenance window; the one you did not is a quarter.
Special cases: code, tables, and identifiers
General purpose embedding models are trained mostly on prose, and three kinds of enterprise content punish that assumption. Source code and configuration files have semantics that prose models capture only loosely; if a meaningful share of the corpus is code, evaluate a model trained for it, because the golden set will expose the difference immediately. Tables are worse: a table flattened to text loses the row and column relationships that made it meaningful, so high value tables usually deserve extraction into structured summaries before embedding rather than naive flattening. Identifiers are the sneakiest case: part numbers, error codes, invoice references, and usernames carry no distributional meaning for any embedding model, and queries that hinge on them will miss in pure vector retrieval no matter which model wins your bake off. The fix is architectural rather than model selection: hybrid retrieval that pairs vector similarity with keyword search, exactly as recommended in the retrieval layer of our reference architecture, catches the exact match cases that geometry cannot. Knowing which of your queries are identifier shaped is, conveniently, another thing the golden set tells you before production does.
Living with the choice
Two operational habits keep the decision healthy. First, monitor retrieval quality on a cadence: a monthly automated run of the golden set catches drift when the corpus composition shifts, new product lines, new document types, new languages, away from what the model was evaluated on. Second, budget a re embedding rehearsal once a year: run the scripted pipeline against the runner up model on a corpus sample and compare. Most years the answer is keep what you have, and the rehearsal cost an afternoon. The year the answer is switch, you will be the only team in the building for whom that is a calm sentence. When embedding model upgrades do land, the same discipline applies as for generation models: evaluation evidence first, alias swap second, never the reverse.
The independent view
Embedding choice is the rare AI decision where boring diligence visibly outperforms enthusiasm: the model is invisible in the demo, decisive in production, and expensive to regret. The pattern we deploy through our OCI implementation practice is the one above, golden set first, three candidates, recall measured on your corpus, exit scripted from day one, delivered on a fixed Project fee and kept honest afterward under a Managed Monthly retainer that runs the drift checks for you. If your vectors are already embedded and retrieval is disappointing, the same harness diagnoses whether the model, the chunking, or the corpus is the culprit, which is a far cheaper question to answer deliberately than by re platforming on a hunch.
Part of a series
This guide is part of Data & AI on OCI — our complete pillar guide on the topic.
Moving Oracle workloads to OCI, or already running on OCI and not sure the architecture or the spend is right? Most teams bring in a specialist before they commit to a region, a shape, or a Universal Credits number. OCISpecialists.com plans the landing zone, runs the migration, and manages the estate after go live, on a fixed project fee, a managed monthly retainer, or a cost optimization fee paid only on verified savings.