Froodl

How We Handle Zero-Downtime Deployments

There is a moment every developer knows. You push to main, the deploy starts, and you refresh the site just to be sure. For a few seconds you wonder whether a user is staring at a 502 error because of your change.

Zero-downtime deployment is meant to remove that moment. This post explains what it means in practice, the trade-offs behind it, and how we approach it at Antryk, including the parts that are still hard.

Why Deployments Cause Downtime in the First Place

The naive deploy looks like this: stop the old version, start the new version, hope it boots. The gap between those two steps is downtime. It can last a few seconds or a few minutes, depending on boot time, migrations and cache warm-up.

Even when you avoid that gap, there are quieter failure modes:

  • Traffic reaching an instance that isn't ready. The process has started, but it hasn't finished connecting to the database or loading config.
  • In-flight requests being cut off. A user submits a payment, the old instance is killed, and the request disappears.
  • Incompatible database changes. The new code expects a column that doesn't exist yet, or the old code breaks on a column that was dropped.
  • Bad releases reaching everyone. A bug that passes tests still takes down 100% of traffic.

The Core Idea: Never Remove the Old Version Until the New One Is Proven

A zero-downtime deploy follows one principle: the old version keeps serving traffic until the new version has shown it can.

  1. Build the new version from your Git push (we support auto deploys from GitHub and GitLab).
  2. Start the new instances alongside the running ones.
  3. Health check the new instances before they receive any user traffic.
  4. Shift traffic to the new instances once they pass.
  5. Drain the old instances, letting in-flight requests finish before shutting them down.
  6. Roll back automatically or manually if something goes wrong.

Each step exists to prevent one of the failure modes above.

Step by Step: What Actually Happens on Antryk

1. Build and Start in Parallel

When a push triggers a deploy, the new version is built and started while the current version continues to serve users. [FILL IN: build system, how long a typical build takes, whether builds are cached.]

2. Health Checks Gate the Traffic Switch

A running process isn't necessarily a ready process. New instances must pass health checks before they receive requests. This is what prevents users from hitting an instance that is still booting.

A tip for readers: make your health endpoint check what actually matters, such as database connectivity, instead of returning 200 OK unconditionally. A health check that always passes protects nobody.

3. Traffic Shifting

Once the new instances are healthy, traffic moves over to them. [FILL IN: all at once vs. gradual, how the load balancer switches, how long it takes.]

4. Graceful Shutdown of the Old Version

Old instances don't vanish. They stop accepting new connections, finish the requests already in progress, and then shut down. This is called connection draining.

[FILL IN: drain timeout, what signal the app receives (e.g. SIGTERM), and what developers should do in their code to handle it.]

5. Auto-Restart and Recovery

If an instance crashes after the deploy, health checks detect it and the platform restarts it automatically. This matters because many outages after a release come from slow-burning problems such as memory leaks and unhandled edge cases, not from the deploy itself.

6. Rollbacks

The most useful safety net is the ability to go back quickly. [FILL IN: how rollbacks work, whether a failed health check aborts the deploy automatically, how fast a manual rollback is.]

The Hard Part: Database Migrations

Platform-level zero downtime only covers your application. Your database schema is a shared resource that the old and new code both touch during the switch, so the safest approach is the expand and contract pattern:

  1. Expand: add the new column or table in a backward-compatible way. The old code still works.
  2. Deploy the new code that uses the new structure.
  3. Migrate data if needed, in the background.
  4. Contract: in a later release, remove the old column once nothing depends on it.

The rules of thumb:

  • Never rename or drop a column in the same release that stops using it.
  • Add new columns as nullable or with defaults.
  • Avoid long locking operations on large tables during peak traffic.

It takes more releases, but each one is safe. If you use our managed PostgreSQL or MySQL, automated backups and point-in-time recovery give you a net underneath risky changes. They are not a replacement for careful migrations.

Things the Platform Can't Do for You

Being honest about limits builds more trust than overpromising:

  • Stateful in-memory data: if your app keeps sessions in local memory, users can lose them when instances change. Store sessions in Redis or a database instead.
  • Long-running requests and WebSockets: connections that last longer than the drain window will be cut. Design clients to reconnect.
  • Breaking API changes: if your frontend and backend deploy separately, version your API or keep changes backward-compatible.
  • Slow-booting apps: if startup takes minutes, deploys will take minutes. Optimize boot time and keep health check timeouts realistic.

A Practical Checklist for Your Next Deploy

  • Expose a meaningful health endpoint
  • Handle SIGTERM and finish in-flight work
  • Keep sessions and state out of instance memory
  • Make every migration backward-compatible
  • Know how to roll back before you need to
  • Set up uptime monitoring so you hear about problems before users do

Wrapping Up

Zero-downtime deployment isn't magic. It is a set of simple rules applied consistently: start the new version beside the old one, prove it's healthy, shift traffic carefully, drain gracefully, and keep a way back. The platform handles most of that, and the rest comes down to how your app and your database changes are designed.

If you want to try this workflow, you can deploy a web service from GitHub or GitLab on Antrykand watch a deploy go out without a single dropped request. 

Have a deployment horror story or a technique we missed? Share it in the comments.

0 comments

Log in to leave a comment.

Be the first to comment.