Upgrades & draining

This page covers moving to a new release: replacing the binary, applying schema changes, and stopping the old process cleanly.

Before you upgrade

1. Read the release notes on GitHub for the version you are moving to.
2. Back up the database and the identity key. See Identity keys.
3. Check that your config file has no keys the new version does not know: an unknown key stops the server at startup. config.example.yaml in the new release lists every key.

Upgrade the .deb

VERSION=1.1.0
BASE=https://github.com/stephanfeb/go-ricochet/releases/download/v${VERSION}
curl -LO "$BASE/ricochet-server_${VERSION}_amd64.deb"
curl -LO "$BASE/SHA256SUMS"
sha256sum --check --ignore-missing SHA256SUMS

sudo apt install ./ricochet-server_${VERSION}_amd64.deb

The package handles the restart:

1. Before the files change, it notes whether the server is running and stops it through supervisord.
2. It installs the new binary, run.sh, health_check.sh and schema.sql.
3. It keeps your /etc/ricochet/config.yaml and /etc/supervisor/conf.d/ricochet.conf (both conffiles) and your /etc/ricochet/env, and resets config.yaml and env to mode 640, owner root:ricochet.
4. If the server was running before, it starts it again on the new binary. A server that was stopped stays stopped.

If the restart fails, the package prints a warning pointing at supervisorctl tail ricochet stderr.

Upgrade a manual install

Stop the server, replace the binary, apply the schema, start it:

sudo supervisorctl stop ricochet          # or your own supervisor
sudo cp ricochet_1.1.0_linux_amd64/ricochet /usr/local/bin/ricochet
psql -h localhost -U ricochet -d ricochet -f ricochet_1.1.0_linux_amd64/schema.sql
sudo supervisorctl start ricochet

Keep the old binary until the new one has run cleanly.

Schema changes

The server never changes the schema itself. Schema changes ship in schema.sql, which is written to be run again on an existing database:

• Tables and indexes are created with IF NOT EXISTS.
• Columns added to existing tables are in an “in-place upgrades” section near the end, with ADD COLUMN IF NOT EXISTS and, where needed, a backfill of existing rows.
• The grants at the end are re-applied.

So on every upgrade, run the new release’s schema.sql against your database, as the database owner or a superuser, before you start the new binary. Running it when nothing changed is harmless. There are no numbered migrations and no down-migrations: to roll back, restore your database backup.

The .deb does not run schema.sql for you. It installs it at /opt/ricochet/schema.sql:

psql -h localhost -U ricochet -d ricochet -f /opt/ricochet/schema.sql

Graceful shutdown

On SIGTERM or SIGINT, the server shuts down in this order:

1. Withdraws from load balancing. /readyz starts returning 503 with "draining":true, and the server logs ops surface draining — /readyz now reports unready.
2. Waits ops.drain_delay (default 0) while still serving, logging draining before shutdown. This gives a load balancer polling /readyz time to take the instance out of rotation.
3. Refuses new requests with a 503 in each protocol’s own error format, and waits up to ops.shutdown_timeout (default 5 s) for requests already in flight. It logs in-flight requests drained, or shutdown timeout reached with requests still in flight if the timeout ran out.
4. Tears down background services, the libp2p host, the ops listener, and finally the database pool, and logs server stopped.

A second signal during this sequence forces an immediate exit with code 130 (SIGINT) or 143 (SIGTERM), and logs forced shutdown: second signal arrived before the graceful stop finished. A shutdown that fails exits with code 1.

/etc/ricochet/config.yaml

ops:
  drain_delay: 20s        # behind a load balancer: a couple of probe intervals
  shutdown_timeout: 5s

Set your supervisor’s kill timeout above drain_delay + shutdown_timeout plus a few seconds for teardown. The .deb’s supervisord config waits 30 seconds (stopwaitsecs=30), which covers the defaults; raise it if you set a long drain_delay.

Check it works

ricochet --version                         # or /opt/ricochet/ricochet_server --version
curl -s http://127.0.0.1:9090/readyz
grep 'running. Peer ID' /var/log/ricochet/stdout.log | tail -n 1

The startup line shows the new version, and the peer ID should be unchanged.

Troubleshooting

• The new version will not start: parse config file: ... field X not found: the key was removed or renamed. Delete or rename it. A logging: block from before v1.0.0 is a common case; delete it.
• Errors about a missing column after the upgrade: the new schema.sql was not applied. Run it.
• The server was running before the .deb upgrade and is now stopped: the restart failed. Check supervisorctl tail ricochet stderr, fix the cause and start it.
• Clients see a burst of failures during each restart: set ops.drain_delay and point the load balancer’s health check at /readyz.