Postmortem

A deadlock Postgres can't see

A Rails migration waited forever on a lock held by its own transaction, through a second connection pool. Why Postgres' deadlock detector can't catch that, how to spot it in pg_stat_activity, and why the logs were empty.

Why my Rails dashboard took two seconds to load

A performance postmortem on Railyard's Rails and Hotwire control plane: an image build on the live box, a hidden tab that streamed logs into Solid Cable, and a page that rendered fourteen hidden tabs to show one.

The deploy that hung for an hour

Two bugs in Railyard's Go agent about the end of a process's life: a build timeout that killed one process out of a tree, and a stale Puma pidfile that survived every restart.

A cloud firewall broke every deploy, and the fix was to stop dialing Postgres

Railyard created each app's Postgres database over the server's public port 5432, so a cloud firewall made every deploy time out. I moved the SQL into the agent's existing gRPC channel and put app-to-database traffic on the Docker network.