The Outbox Pattern in Rails: How to Publish Events Without Losing Data

The bug is always the same shape: a payment gets captured, the database commits, and then the process crashes half a second later, before the "PaymentCaptured" message reaches the queue. The customer was charged. Nobody downstream ever finds out. The outbox pattern exists to make that sequence impossible.

The problem with publishing after commit

The intuitive approach, save the record, then publish a message, has an unavoidable gap between the two steps. Anything that fails in that gap, a crash, a network blip, a deploy landing at the wrong second, silently drops the message while the database change stands. Publishing before commit has the opposite problem: the message goes out and the transaction then rolls back, and now a consumer reacted to something that never actually happened.

Write the event in the same transaction as the data

The fix is to never treat "save the row" and "announce the row" as two separate operations. Instead, write the outgoing message into an outbox table, in the same database transaction as the business change it describes. Since it's the same transaction, both commit together or neither does: there's no gap left for a crash to hide in.

ActiveRecord::Base.transaction do
  payment.update!(status: "captured")
  OutboxMessage.create!(
    topic: "payments",
    event_type: "PaymentCaptured",
    payload: { payment_id: payment.id }.to_json
  )
end

A relay moves messages out, on its own schedule

A separate, small process, a poller running every few seconds, or a change-data-capture tool reading the database's write-ahead log, reads unpublished rows from the outbox and pushes them to the real broker, marking each one as sent once the broker acknowledges it. If that relay crashes mid-batch, it simply resumes from the last unsent row on restart: nothing is lost, and consumers just need to tolerate an occasional duplicate.

The one requirement this pushes onto consumers

Because the relay can, in rare cases, deliver the same message twice, every consumer of an outbox-backed topic has to be idempotent: processing the same event twice must produce the same end state as processing it once. That's a small, well-understood problem to solve on the consumer side, and a much better trade than losing events silently.