Event Driven Architecture on OCI: A Reference Build

The problem event driven architecture solves is coupling. In a synchronous design, the service that takes an order must call the inventory service, which calls the payment service, which calls the fulfillment service, and the customer waits while four systems agree. If any one of them is slow, the order is slow. If any one of them is down, the order fails. The teams that own those services cannot deploy independently, scale independently, or fail independently, because the call chain has welded them together. An event driven design replaces the chain with a statement of fact: an order was placed. Whoever cares about that fact reacts to it on their own schedule, and the service that published it has already moved on.

This article is part of our series on application development and serverless on OCI. We will cover why OCI is a comfortable home for this style, what each of the five core services actually does, a reference build for order processing that you can adapt, the delivery guarantees you get and the idempotency you owe in return, schema and payload design, dead letter queues, observability, what it all costs, and the mistakes that turn elegant diagrams into operational grief.

Why event driven on OCI

The architectural case is the same on any cloud: decoupled producers and consumers, independent scaling, burst absorption, and an audit trail of business facts as a natural side effect. What makes OCI specifically pleasant is the economics and the integration. The messaging services are priced per request or per throughput unit at rates that round to noise for most estates, the serverless compute that consumes events scales to zero between bursts, and the platform itself emits events for almost everything that happens inside it, so the eventing fabric you build for business traffic doubles as an automation fabric for operations. Teams running lift and shift estates also find that event driven patterns are the most forgiving first step into application modernization: you can bolt an event publisher onto a legacy system long before you decompose it.

The five services that matter

OCI splits the eventing space into five services, and choosing the wrong one is the most common early mistake. They are not interchangeable. Each has a distinct delivery model, retention story, and consumer shape, and the reference build later in this article uses four of them together precisely because each does one job well.

OCI Events is the platform listener. It watches the audit stream of your tenancy, matches events against rules you define, and routes matches to Functions, Notifications, or Streaming. It carries no business payloads of your own design; its job is to make the platform itself programmable. OCI Queue is the workhorse for task distribution: producers enqueue messages, competing consumers poll for them, and the visibility timeout ensures a message picked up by a crashed worker reappears for another. Streaming is the log: ordered within a partition, retained for replay, and readable by any number of consumer groups at their own pace, which makes it the right substrate when several teams need the same event history. Notifications is the megaphone: publish once to a topic and every subscription receives a copy, which is how one business event reaches a function, an external webhook, and an operations mailbox at the same time. And Functions is where the reacting code lives, billed only while it runs.

A reference build: order processing

Theory is cheap, so here is the build we deploy as a starting point for retail and B2B order flows. The shape generalizes to claims processing, onboarding, document pipelines, and most other transactional domains; only the event names change. The defining property is that the customer facing step does almost nothing synchronously.

Notice what this build avoids. No service calls another service directly. The order API does not know fulfillment exists. Inventory can be down for an hour and orders still flow into the queue, draining when it returns. Black Friday traffic lands on a queue that absorbs it rather than a thread pool that exhausts, and the fulfillment fleet scales on queue depth instead of guesswork.

Delivery guarantees and retries

Every messaging service on OCI, and on every other cloud, gives you at least once delivery, not exactly once. The retry machinery that makes delivery reliable is the same machinery that occasionally delivers a message twice: a consumer processes a message, crashes before acknowledging it, and the message reappears. You do not fight this; you design for it. Every consumer must be idempotent, which usually means carrying a unique event identifier in the payload and recording processed identifiers so a duplicate becomes a cheap no op. Reserving inventory twice for the same order identifier should be impossible by construction, not unlikely by luck.

The retry knobs differ per service and deserve deliberate settings. On Queue, the visibility timeout must exceed your worst case processing time, or healthy workers will compete for messages other healthy workers are still holding; the maximum delivery attempts setting decides when a message is declared poison. Notifications retries failed deliveries to a subscription across a multi hour window with backoff, which is generous for transient outages but means a permanently broken webhook fails silently for hours unless you alarm on it. Streaming does not retry at all in the push sense, because consumers pull: your consumer owns its offset, commits it only after successful processing, and resumes from the last commit after a crash. Ordering is guaranteed only within a Streaming partition, so events that must be processed in sequence, everything about one order, for example, must share a partition key.

Schema and payload design

Events outlive the code that produced them, so the payload is a contract, and contracts need discipline. Adopt a standard envelope, the CloudEvents specification is the obvious choice, so every event carries a type, a source, a unique identifier, and a timestamp in the same place regardless of which team produced it. Version the event type explicitly, OrderPlaced.v1, and treat the rules of compatibility as law: adding optional fields is safe, removing or renaming fields requires a new version published alongside the old until every consumer migrates.

The other design decision is thin versus fat payloads. A thin event carries identifiers and a type, the OCID or order number, and consumers call back for the rest; it stays small, never goes stale, and never leaks data to consumers that should not see it. A fat event carries the full order so consumers need no callback, which keeps them decoupled from the producer API but bloats the message and freezes a snapshot that may age badly. Our default is thin events for facts that reference mutable entities and fat events for immutable ones, and either way keep payloads under the size limits with comfortable margin: large blobs belong in Object Storage with a reference in the event, never in the message body.

Error handling and dead letter queues

In an event driven system, failure handling is not an exception path; it is half the architecture. The first category is the transient failure, a timeout, a throttle, a deploy in progress, and retries with backoff absorb it invisibly. The second category is the poison message: a payload that fails every attempt because the bug travels with the message. Without a limit, a poison message blocks a worker forever; with a maximum delivery attempt count, it is moved aside to a dead letter queue after the configured number of failures, and the healthy traffic flows on.

The DLQ is only useful if somebody owns it. Alarm on its depth from the first message, because one poison message is usually the leading edge of a class of them. Build the replay path on day one: a small tool that lets an engineer inspect a dead letter, fix the consumer or the data, and resubmit the message to the main queue. Teams that skip this end up doing surgery in production consoles at 2 a.m. Finally, distinguish retryable from terminal errors inside your handlers: a validation failure will never succeed on retry and should go straight to the DLQ with a reason attached, while a downstream timeout deserves its full retry budget.

Observability

Synchronous systems fail loudly; event driven systems fail quietly, by falling behind. The single most important signal is lag: queue depth and oldest message age on Queue, consumer group offset lag on Streaming, and failed delivery counts on Notifications, all of which surface in OCI Monitoring as metrics you can alarm on. A queue that is growing faster than it drains is an outage in progress even though every health check is green.

The second requirement is correlation. A single business transaction now spans five services and three messaging hops, so every event must carry a correlation identifier minted at the edge and propagated through every hop, and every log line must include it. With that in place, OCI Logging lets you reconstruct one order's journey across the estate with a single query; without it, you are matching timestamps by hand. Add a small number of end to end probes, a synthetic order placed every few minutes whose progress events are checked against a deadline, and you will know the pipeline is healthy as a whole rather than inferring it from parts.

Cost behavior

The pleasant surprise of this architecture on OCI is that the messaging layer is rarely visible on the invoice. Queue charges per million requests, Notifications per delivery beyond a free allowance, and Events nothing at all for rule evaluation; for a system doing a few million transactions a month these line items are coffee money. Streaming is priced on throughput and storage, a per GB rate for data in and out plus retention, so it scales with volume but stays modest at transactional scale. Functions bills in GB seconds of execution, and consumers that wake on events and sleep otherwise cost a fraction of an always on fleet doing the same work.

Two behaviors deserve watching. First, aggressive empty polling: a fleet of workers polling an empty queue at high frequency pays per request for the privilege of finding nothing, so use long polling and sensible intervals. Second, fan out multiplication: one event delivered to six subscriptions is six deliveries, and a chatty event design multiplies through every topic, so keep event granularity meaningful. Neither behavior makes the architecture expensive; both make it less absurdly cheap than it should be.

Common mistakes

Four patterns account for most of the trouble we see in reviews. First, using Streaming as a queue: standing up partitioned, offset managed infrastructure for simple work distribution that Queue handles with a tenth of the operational surface. Match the service to the job. Second, assuming exactly once delivery and skipping idempotency, which works perfectly until the first duplicate reserves inventory twice during a busy hour. Third, events as remote procedure calls: publishing an event and then blocking until a reply event arrives, which recreates the synchronous coupling with extra steps and worse latency. If you need a reply now, make a call. Fourth, no DLQ and no replay plan, which converts the first poison message into a production incident with no tooling. None of these is hard to avoid on day one; all of them are expensive to retrofit under pressure.

Bringing it together

Event driven architecture on OCI is not a leap; it is an assembly job with good parts. Events makes the platform programmable, Queue levels the load, Streaming keeps the replayable history, Notifications does the fan out, and Functions supplies compute that costs nothing while it waits. The reference build above is deliberately boring: an outbox, a topic, a queue, a DLQ, and consumers that assume duplicates. Boring is the point. The teams that struggle with this style are almost never fighting OCI; they are fighting missing idempotency, unowned dead letters, and event contracts that changed without a version. Get those three disciplines right and the architecture pays for itself in the first traffic spike it absorbs without anyone being paged.