Oracle 23ai Vector Search: RAG on Your Own Data

Retrieval augmented generation has a quiet dependency that most architecture diagrams underplay: the retrieval layer is a database, and databases come with all the obligations that the AI conversation likes to skip. Access control, durability, consistency, patching, audit. When the corpus you are retrieving from is the enterprise's own data, contracts, tickets, policies, product records, those obligations are not optional, and the question of where the vectors live becomes a governance question wearing a performance costume.

Oracle's answer in Database 23ai is to make vectors a native type in the converged database, so similarity search runs in the same SQL engine, inside the same transaction and security model, as everything else. This article is part of our complete guide to AI on OCI; here we go deep on the vector layer itself.

What vector search actually does

An embedding model turns a piece of content, a paragraph, an image, a code snippet, into a list of numbers that encodes its meaning. Content with similar meaning lands close together in that numeric space. Vector search is simply the operation of finding the stored vectors nearest to a query vector, which is how a question phrased one way retrieves a policy paragraph phrased another way, with no keywords in common.

In a RAG pipeline this is the heart of the system: the user's question is embedded, the nearest chunks of your corpus are retrieved, and those chunks are handed to a language model as grounding context. The model writes the answer; the retrieval layer decides what the answer can be based on. Quality lives or dies in retrieval, which is why the choice of embedding model, covered in detail in choosing embedding models for vector workloads on OCI, deserves more deliberation than it usually gets.

The building blocks in 23ai

The VECTOR data type

23ai adds VECTOR as a first class column type. A table can carry a vector column next to its relational columns, declared with a dimension count and a number format, and populated like any other column. That one design decision carries most of the architectural weight: the vector is an attribute of the business row, not a foreign copy of it in another system. When the row is updated or deleted, its vector goes with it. When row level security hides the row from a user, the vector is hidden too.

Similarity in plain SQL

Queries use a distance function with the metric of your choice, cosine, Euclidean, or dot product, and standard SQL clauses to fetch the nearest rows. Because it is just SQL, similarity predicates compose with everything else the engine can do: joins to business tables, WHERE clauses on structured columns, and aggregation. A query like "the five most similar clauses, but only from contracts signed after 2024 in the EMEA region" is one statement, not a pipeline across two systems with a reconciliation problem in the middle.

Vector indexes: HNSW and IVF

Exact nearest neighbor search scans every vector and gets slow as the corpus grows. 23ai provides two approximate index types to fix that. The HNSW index is an in memory graph structure that delivers very fast, high recall search and is the default choice when the vector set fits in the memory you can give it. The IVF index is partition based, lives in storage like a conventional index, and suits larger corpora and instances with tighter memory. Both let you trade a controlled amount of recall for speed, and the right setting is a measured decision, not a default.

Embedding inside the database

23ai can load an embedding model in ONNX format and generate vectors inside the database process, so chunking and embedding happen next to the data without an external call per document. Alternatively, the database can call out to a hosted embedding endpoint such as the one in the OCI Generative AI service. In database ONNX embedding keeps sensitive text inside the database boundary entirely, which for some regulated corpora is the deciding argument.

In database vectors versus a dedicated vector store

The fair comparison is not "Oracle versus a vector database" in the abstract. It is two architectures: vectors beside the data they describe, or vectors in a separate specialized engine with a synchronization pipeline between the two.

The honest caveat runs the other way too. If your corpus has no relationship to data in Oracle, if your team has no Oracle estate, or if you are chasing billion vector scale with exotic index tuning, a specialized store can be the simpler answer. The architecture should follow the data, and the full set of storage options is weighed in our reference RAG architecture on OCI.

Where it runs on OCI

AI Vector Search is a feature of the database, not a separate service, so it runs wherever 23ai runs: Autonomous Database, Base Database Service, and Exadata platforms. Autonomous Database is the natural starting point because it removes the patching and tuning burden, scales compute independently of storage, and pairs vector workloads with the broader AI features of the platform, including the natural language interface described in Select AI in Autonomous Database. For estates already standardized on Exadata, vector search inherits the same performance machinery as the rest of the workload. We design and operate this layer as part of our Autonomous Database practice.

Building the RAG pipeline

A production pipeline on 23ai has five stages, and each has one decision that matters more than the rest.

Ingest. Documents arrive from Object Storage, application tables, or external systems. The decision is provenance: keep a source reference on every chunk so answers can cite their origin, because uncited answers are unverifiable answers.

Chunk. Long documents are split into retrievable pieces. The decision is granularity: chunks must be small enough to be specific and large enough to carry context. Start near a few hundred tokens with modest overlap and tune against real queries rather than folklore.

Embed. Each chunk becomes a vector, in database via ONNX or through a hosted endpoint. The decision is the model, and the commitment is real: change the embedding model later and every stored vector must be regenerated.

Retrieve. The query is embedded and the nearest chunks are fetched, filtered by the structured predicates that enterprise queries almost always need. The decision is how many chunks to pass and whether to rerank them before generation.

Generate. The retrieved context and the question go to a language model, on OCI typically through the Generative AI service. The decision is the grounding contract: instruct the model to answer only from the context, and measure how often it complies.

A deployment framework

Performance and sizing realities

Three numbers drive sizing. Vector dimensions multiply storage and memory: a corpus embedded at 1024 dimensions costs roughly half the memory of the same corpus at 2048, with a quality difference that is often invisible for your data, which is an argument for testing smaller models first. Index memory determines HNSW viability: the index wants to live in RAM, so the instance must be sized for corpus growth, not corpus today. And concurrency shapes the experience: similarity queries are short but CPU intensive, so a chat application with many simultaneous users behaves like an OLTP workload and should be capacity planned like one. None of this is exotic, but all of it belongs in the design review rather than the incident retrospective.

Operating the vector layer

The operational story is the underrated half of the in database argument. Vectors stored in 23ai are inside the database's existing protection machinery: they are backed up with the database, replicated by Data Guard with the database, and recovered with the database, so the disaster recovery plan for the RAG system is the disaster recovery plan you already have. Patching, encryption at rest, and audit all apply without a second system's versions of each. Monitoring follows the same logic: vector queries appear in the same performance views and tuning workflows as the rest of the workload, so the DBA team can see a slow similarity query the way they see any slow query. The new skills are real but narrow, index parameter tuning and embedding pipeline care, and they sit on top of a practice the organization already runs rather than beside it. For estates we operate under Managed Monthly retainers, the vector layer is simply another schema with two extra dashboards, which is exactly the amount of operational novelty an enterprise wants from its AI infrastructure.

The independent view

Vector search in 23ai is the rare AI feature whose strongest case is boring: it removes a system, a pipeline, and a second security model from the architecture. That case is strongest when the corpus is already governed Oracle data and weakest when it is not, and an independent assessment of which describes your estate is worth far more than a platform preference. We run that assessment, build the pipeline, and operate the result under a fixed Project fee or a Managed Monthly retainer, with the same independence we bring to every platform question: we sell the architecture that fits, not the one with the logo on it.