Stop Publishing Events After Your Database Commit

The Bug You Can't Reproduce in Dev

You have an order service. It saves the order to Postgres, then publishes an OrderCreated event to Kafka. Both succeed. Downstream services pick up the event and do their thing — provision inventory, trigger invoicing, send a confirmation email.

One night, your service gets a SIGKILL mid-request. Postgres committed. Kafka never got the event. The order exists in your database with no corresponding downstream work. Nobody noticed for three days because there was no error — just silence.

This is the dual-write problem. Two separate writes to two separate systems, with no transactional guarantee between them. One succeeds. The other might not.

Why It's Hard to Catch

The failure window is tiny: the microseconds between your database commit returning and your broker publish completing. In tests, everything runs in sequence on the same machine. In production, you have process crashes, OOM kills, network timeouts, and deploys happening mid-request.

Your observability won't catch it cleanly. The HTTP response was 200. The database row exists. No exception was thrown. The event just... never happened. You only find out when someone notices the warehouse has no record of an order that's already been shipped.

The inverse is just as bad: publish the event first, then write to the database. The consumer wakes up before the row exists and tries to look it up. Race condition. Retry storm. Inconsistent state.

*The event is lost. Postgres has the order. Kafka doesn't.*

The Outbox Pattern

The fix is to stop treating your message broker as a second write target. Treat your database as the source of truth for both the business state and the intent to publish.

Add an outbox table. In the same transaction as your business write, insert the event:

BEGIN;

INSERT INTO orders (id, customer_id, total, status)
VALUES ($1, $2, $3, 'pending');

INSERT INTO outbox (id, aggregate_id, event_type, payload)
VALUES (gen_random_uuid(), $1, 'OrderCreated', $4);

COMMIT;

Either both writes commit or neither does. The event row is your durable intent to publish.

A separate relay process polls the outbox and forwards pending events to Kafka:

const pending = await db.query(`
  SELECT * FROM outbox
  WHERE processed_at IS NULL
  ORDER BY created_at
  LIMIT 100
  FOR UPDATE SKIP LOCKED
`);

for (const row of pending) {
  await kafka.produce(row.event_type, row.payload, { key: row.id });
  await db.query(
    `UPDATE outbox SET processed_at = now() WHERE id = $1`,
    [row.id]
  );
}

FOR UPDATE SKIP LOCKED prevents two relay instances from racing on the same rows when you scale the relay horizontally.

The relay can crash between the Kafka produce and the UPDATE. The same event might publish twice. Design your consumers to be idempotent — deduplicate on the event id, or lean on Kafka's exactly-once delivery if your broker configuration supports it.

CDC as an Alternative

If polling latency matters — you need sub-second event delivery — look at Change Data Capture. Debezium tails your Postgres WAL and streams outbox table changes directly to Kafka. No polling loop, no relay code to deploy.

The tradeoff is operational. You're now managing a replication slot on your database. Slots accumulate lag and eat disk if Debezium falls behind. WAL retention has to be tuned. For most teams, polling every 200–500ms is fast enough and the operational surface is much smaller. Start with polling. Reach for CDC when you've measured that the lag actually hurts.

What This Doesn't Solve

The outbox pattern fixes one specific problem: the event fires when and only when the database row commits. It doesn't fix:

• **Cross-service sagas** — if multiple services need to agree on an outcome, you still need a saga orchestrator or choreography layer. The outbox gets the first event out reliably; coordinating the response is a different problem.

• **Read consistency** — consumers see eventual consistency regardless. Events propagate with some lag, and your read models will be behind.

• **Strict global ordering** — if events from multiple aggregate types must arrive in a global order, Kafka partition keys and careful schema design are separate problems the outbox doesn't touch.

The pattern also adds real maintenance overhead. You need the outbox table, an index on (processed_at, created_at), a relay deployment with its own monitoring, and a retention job to prune old processed rows. On a high-write service, that extra insert per transaction is measurable. Profile before assuming it's free.

The Practical Test

Any code that looks like this is a latent bug:

await db.transaction(async (trx) => {
  await trx('orders').insert(order);
});

// Outside the transaction. If this throws, the order exists with no event.
await kafka.publish('OrderCreated', order);

The gap between those two blocks is where your data integrity lives. The outbox pattern closes it by making the event part of the atomic operation, not a follow-up side effect that runs after the transaction closes.

Add this to your next code review checklist: any place you see a database commit followed by a broker publish is a failure scenario waiting for a bad deploy or a 3am OOM kill. The outbox pattern is the fix. The cost is operational complexity you can't avoid. The alternative is silent data loss you can't detect until a customer calls.