The OCI Data Platform: HeatWave, GoldenGate, and Lakehouse

Every organisation now claims to be data driven, and most of them are paying three different vendors to move the same data between four different systems to prove it. The modern data stack has become a procurement category of its own: a database here, a warehouse there, a streaming platform, an ETL tool, a catalog, a BI layer, each with its own bill, its own operations burden, and its own copy of the data. OCI's answer to this sprawl is a data platform assembled from services that share a cloud, a security model, and in several cases an engine: Autonomous Database, HeatWave, GoldenGate, Streaming, Data Integration, Data Flow, Data Catalog, and an object storage lakehouse underneath. This pillar maps the whole platform, explains what each piece is actually for, and links to the deeper guides in this series.

A framing note before the tour. OCI's data services are less famous than their competitors and in some cases less broad, but they have two structural advantages that compound at estate scale. First, several of them collapse two or three products into one engine, which removes pipelines rather than optimising them. Second, they sit on infrastructure with cheap egress and uniform global pricing, which matters more for data platforms than for any other workload class, because data platforms are the workloads that move data.

The platform at a glance

The database layer: Autonomous and HeatWave

Autonomous Database

Autonomous Database is the centre of gravity for Oracle estates: a converged engine that runs relational, JSON, graph, and spatial workloads in one database with self tuning and self patching operations. The converged argument matters for platform design because each capability it absorbs is a separate product somewhere else. Document workloads that would otherwise justify a dedicated document database can run as collections with full SQL access, a pattern we unpack in JSON workloads on Autonomous. The same engine exposes governed shares to external consumers through the open Delta Sharing protocol, covered in data sharing from Autonomous Database, and carries a free application layer in APEX, which we put to work in building data apps fast with APEX.

HeatWave MySQL

HeatWave is the most underrated service on OCI. It is MySQL with an in memory columnar accelerator attached, which means the same database that takes the transactions also answers the analytics, without an ETL pipeline and without a separate warehouse. For the large population of applications built on MySQL, this collapses an entire architecture into one service, and the full story is in our HeatWave MySQL guide. The same engine extension also queries data sitting in object storage at warehouse speeds, which we cover separately in HeatWave Lakehouse.

The movement layer: replication, pipelines, and streams

Data movement is where platforms quietly fail and where bills quietly grow, so OCI's movement services deserve precise definitions rather than marketing ones.

GoldenGate for change data capture

OCI GoldenGate is the managed form of Oracle's replication technology: it reads database change logs and applies them elsewhere in near real time, which makes it the tool for zero downtime migrations, cross region distribution, and feeding analytics stores without touching source applications. The patterns and the sizing model are in our GoldenGate on OCI guide, and the decision between GoldenGate and Data Guard, which solves a different problem despite the surface resemblance, is settled in data replication strategies on OCI.

Data Integration for batch ETL

OCI Data Integration is the serverless ETL service: a visual designer for extract, transform, and load pipelines that runs without any servers to provision or patch. It is the right default for scheduled batch movement and shaping, and we walk through its concepts, pricing behaviour, and limits in OCI Data Integration: ETL without servers.

Streaming for events

OCI Streaming is the Kafka compatible event backbone: partitioned, durable topics for event producers and consumers, billed by throughput rather than by cluster. Where events rather than tables are the natural shape of the data, it is the platform's front door, and the architecture patterns are in OCI Streaming: Kafka compatible event pipelines.

Data Flow for heavy Spark

OCI Data Flow runs Apache Spark jobs serverlessly: submit the job, the service provisions the cluster, runs it, and tears it down. It exists for the transformations too heavy or too code shaped for a visual ETL tool, and we cover when that line is crossed in OCI Data Flow: serverless Spark for heavy lifts.

Database Migration Service for getting here

The platform also includes a purpose built on ramp. OCI Database Migration Service wraps validation, initial load, and GoldenGate based replication into a managed migration workflow, and knowing when to use it rather than raw tooling is the subject of OCI Database Migration Service: when to use it.

The lakehouse layer

Underneath everything sits Object Storage, and the modern pattern is to treat it not as a dumping ground but as the lake half of a lakehouse: open formats such as Parquet, organised into zones, queried in place by engines that do not require the data to be loaded first. Autonomous Database queries object storage through external tables, HeatWave Lakehouse scans it at engine speed, and the result is warehouse convenience at object storage prices. The full architecture, zones, formats, and engine choices included, is in building a lakehouse on OCI.

For estates with existing Hadoop and Spark investments, OCI Big Data Service provides managed Cloudera based clusters, a pragmatic landing zone for platforms that are not ready to be re architected, covered in OCI Big Data Service.

Governance: knowing what you have

A platform without a catalog becomes a swamp on a schedule measured in months. OCI Data Catalog harvests technical metadata from the databases, object storage, and pipelines, attaches business glossaries, and traces lineage, so that the question what is this column and where did it come from has an answer that does not involve archaeology. We cover deployment and the habits that keep a catalog alive in OCI Data Catalog: metadata and lineage for the estate.

Consumption: analytics and applications

Data platforms exist to be consumed. Oracle Analytics Cloud is the BI layer: dashboards, self service exploration, and a semantic model over the warehouse and the lakehouse, with deployment and sizing choices that matter more than the feature list, as we explain in Oracle Analytics Cloud on OCI. For consumption that looks like an application rather than a dashboard, APEX on Autonomous turns governed data into working tools in days, and external partners consume through Delta Sharing without needing OCI accounts at all.

Choosing engines without overbuying

The most common platform design mistake we see is buying one of everything. The table below is the short decision guide; the long version is each linked article.

The cost dimension

Data platforms leak money in characteristic ways: duplicate copies of the same data in three stores, pipelines that run whether anyone reads the output or not, warehouses sized for a peak that happens quarterly, and egress charges on every hop in clouds that charge for egress. OCI's pricing posture blunts the last of these, but the others are design problems, and they are cheaper to prevent than to fix. We collected the patterns, and the design rules that avoid them, in designing an OCI data platform that does not leak money, and it is the article we most wish clients had read before their first build.

A seven step framework for building the platform

Where to go next

The articles in this series go deep on each component: HeatWave MySQL, HeatWave Lakehouse, GoldenGate, replication strategies, Data Integration, Streaming, Data Flow, Database Migration Service, lakehouse architecture, Big Data Service, Data Catalog, Analytics Cloud, JSON workloads, data sharing, APEX data apps, and cost design. If the platform you are planning, or the one you have inherited, needs an experienced eye, our Autonomous Database practice designs and runs these estates, and an OCI assessment is the fastest way to get a concrete architecture and a defensible cost model on the table.