HeatWave Lakehouse: Querying Object Storage at Speed

Every lakehouse architecture rests on one bargain: keep the bulk of the data in object storage because it is durable and nearly free, and accept that querying it will never feel like querying a warehouse. Most platforms soften the trade with caching layers, external table mechanisms, or separate query engines, and each softening adds either cost or another system to operate. HeatWave Lakehouse takes a more direct route. It loads data from Object Storage files into the same in memory, columnar, massively parallel engine that HeatWave uses to accelerate MySQL queries, and serves both from one SQL surface. The files stay in object storage at object storage prices; the queries run at memory speed.

This article explains the mechanics, where the approach genuinely beats the alternatives, where it does not, and how to size a deployment so the economics hold. It assumes the basic HeatWave context from our HeatWave MySQL guide and the lakehouse zone structure from building a lakehouse on OCI, both part of the OCI data platform series.

The mechanics: load once, query at memory speed

HeatWave is a cluster of nodes holding data in memory in a compressed columnar format, attached to a MySQL DB system that remains the SQL front door. Lakehouse extends the loading path: alongside MySQL tables, the cluster loads external files, Parquet, CSV, and export formats, directly from Object Storage buckets. A mapping declares the bucket location and structure, an automated tool samples the files to infer schema and recommend cluster size, and a load command pulls the data across all nodes in parallel.

Once loaded, external data is queried with ordinary SQL, joined freely against native MySQL tables, with the optimizer pushing work to the cluster. The performance characteristics follow from the architecture: scans, joins, and aggregations parallelise across all nodes and run against memory rather than storage, which is why query times compete with dedicated warehouses rather than with external table mechanisms that read object storage on every query. The load is the toll gate: data must be loaded before it is fast, loading takes time proportional to volume, and the cluster must be sized to hold what you load.

Refresh follows the lakehouse cadence. When pipelines land new files in the bucket, the affected tables reload on schedule or on demand. Lakehouse data is read optimised: the files are the source of truth and the engine is a query accelerator over them, not a transactional store for them. Writes belong in the MySQL side or upstream in the pipeline.

Where it sits among OCI's lakehouse query options

The decision logic is about query frequency and engine gravity. Data queried occasionally does not earn a memory load; external tables in Autonomous read it in place at zero standing cost, which is the right answer for cold zones. Data queried constantly, dashboards, repeated analyst workloads, product analytics, pays back the load and the cluster because every query after the first is nearly free in time. Heavy transformation belongs in Spark regardless, as a pipeline stage rather than a query layer. And estates already centred on Autonomous for analytics should weigh the Oracle side first, because two query engines over one lake is a real operational cost; the honest comparison is usage shaped, which is exactly the analysis we run in engagements.

The economics, concretely

The bill has three parts: Object Storage for the files, which is the cheap part and stays cheap; the MySQL DB system, modest and constant; and the HeatWave cluster, which is the dominant line and scales with node count. The cluster must hold the loaded working set in memory, so cost tracks the volume you choose to make fast, not the volume you own. That distinction is the whole game. An estate with 200 TB in the lake and 5 TB of genuinely hot data should size the cluster for the 5 TB and leave the rest to in place query paths or scheduled jobs.

Compression works in your favour, with columnar encoding typically fitting data into meaningfully less memory than its raw size, and the sizing advisor estimates this per dataset rather than leaving it to guesswork. The failure mode to design against is load creep: every team loads their tables to make them fast, nobody unloads anything, and the cluster grows node by node until the lakehouse costs warehouse money. Quarterly review of what is loaded against what is queried is the discipline, and it is the same discipline we describe estate wide in data platform cost design.

A six step sizing and adoption framework

Limits worth naming

Independence requires the edges. The load step is real: first query speed on a new dataset is gated by load time, so Lakehouse suits stable hot sets, not ad hoc exploration of arbitrary corners of the lake, which remains external table or Spark territory. The SQL dialect is MySQL, and analyst tooling or SQL written for Oracle or another warehouse needs porting. The engine is read optimised over files: it is not a transactional store, and late arriving or mutating data is handled by refresh, not by updates. And the cluster bills while it runs, so a deployment serving one report a week is mis sized by definition. None of these are surprises if the framework above was followed; all of them are if it was not.

Bringing it together

HeatWave Lakehouse closes the gap that made lakehouses feel like a compromise: warehouse speed queries over data that stays in cheap object storage, from the same engine that already serves the operational MySQL estate. Used with discipline, hot data loaded, cold data left in place, refresh matched to the pipeline, it delivers interactive analytics at a fraction of dedicated warehouse cost. Used without discipline it quietly becomes the warehouse it replaced. The storage foundation matters as much as the engine, and our OCI storage practice designs the bucket and lifecycle layer underneath these builds. For the whole picture, an assessment sizes the hot set, the cluster, and the honest monthly number before any commitment is made.