Zero-Downtime Database Migrations in Rails: A Practical Playbook
Why the migrations that look small are the ones that break production, and the sequence we follow to change a schema without locking a busy table.
Publié le 15 octobre 2023 · 5 min de lecture
The migrations that break production are rarely the ones that change a lot. They're the ones that look small: adding a NOT NULL column, renaming something a background job still references, or adding an index the obvious way. Here's the playbook we default to on Rails apps that can't afford a maintenance window.
Locks are the real enemy, not size
Postgres and MySQL both need to briefly lock a table for certain DDL operations. On a table with a few hundred rows that lock is invisible. On a table with forty million rows and a steady write rate, that same lock queues every write behind it, and every request behind those. The migration itself might run in two seconds; the outage it causes can last minutes while the lock queue drains.
Adding a column safely
Adding a nullable column with no default is cheap on modern Postgres, it's a metadata-only change. The moment you add a default value on an older Postgres version, or a NOT NULL constraint outright, you force a full table rewrite. The safe sequence is always the same: add the column nullable, backfill in batches, then add the constraint once every row already satisfies it.
# migration 1
add_column :invoices, :currency, :string
# backfill, in a rake task or a one-off job, batched
Invoice.in_batches(of: 5_000) { |batch| batch.update_all(currency: "EUR") }
# migration 2, once the backfill is done
change_column_null :invoices, :currency, false
Indexes without locking writes
A plain add_index takes a lock that blocks writes for the duration of the build. On any table that matters in production, use a concurrent build instead, and disable the transaction wrapper Rails adds by default so the statement can actually run outside a transaction.
class AddIndexToInvoicesOnStatus < ActiveRecord::Migration[7.1]
disable_ddl_transaction!
def change
add_index :invoices, :status, algorithm: :concurrently
end
end
Renames are never really renames
Renaming a column in one migration and shipping the code change in the same deploy guarantees a window where old application code and the new schema disagree. We split it into separate deploys instead: add the new column and dual-write to both, backfill, then switch reads to the new column, and only drop the old one in a final, separate deploy once nothing references it.
None of this is exotic. It's slower than a single migration, on purpose, and it's the difference between a deploy nobody notices and an incident channel lighting up on a Friday evening.
Articles liés
How to Backfill Millions of Rows in a Rails Database Without Downtime
Backfilling a table with forty million rows is a different problem than backfilling one with forty thousand. What changes, and how we throttle it safely.
Database Transactions in Rails: What ACID Really Means for Your Code
ActiveRecord makes transactions easy to reach for. What each of the four ACID guarantees actually promises, and where Rails developers get isolation wrong.