English
Deployment and operations
Step 1. The first deployment
Why. Everything is in place: the server from topic 23, the activation script, the deploy key, the environment and both workflows on dev. A merge into main starts the pipeline.
Do. On GitHub, open a pull request with base main and compare dev (or reuse the one from topic 22). CI runs for it; when Backend and Frontend are green, merge it. The push to main starts the workflow Deploy: open Actions → Deploy and the new run.
- CI / Backend and CI / Frontend run as in topic 22, about five minutes.
- Package builds the release and uploads the artifact chatex-release, a few minutes more.
- Deploy shows
Waitingand a banner that the deployment toproductionneeds review. Click Review deployments, tickproductionand Approve and deploy. Only now does the job start and receive the secrets.
Check
The log of the step Activate release of the first deployment:
[chatex-activate] installed /opt/chatex/releases/<sha>
[chatex-activate] applying migrations
Failed executing DbCommand (18ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
SELECT migration_id, product_version
FROM "__EFMigrationsHistory"
ORDER BY migration_id;
Acquiring an exclusive lock for migration application. See https://aka.ms/efcore-docs-migrations-lock for more information if this takes too long.
Applying migration '20261029094940_InitialCreate'.
Done.
[chatex-activate] switching to <sha>
[chatex-activate] <sha> is liveThe "failed" query is harmless and appears only once: the bundle first looks for its history table, which does not exist in an empty database, then creates the schema. Smoke test prints Healthy, the run is green, and the run page shows the environment production with a link to your site.
Step 2. Checking the live site
Do. Open https://<your domain> in the browser. The sign-in page of Chatex appears, with the lock icon of HTTPS. Register a user, open a private window and register a second user, start a direct chat between them and send messages back and forth: messages, typing indicators and online status arrive in real time — over the WebSocket that Nginx passes through to the hub.
Then look at the server:
bash
readlink /opt/chatex/current/opt/chatex/releases/<sha>bash
systemctl status chatex-apiThe status is active (running), with the main process /usr/bin/dotnet /opt/chatex/current/api/Chatex.Api.dll.
bash
journalctl -u chatex-api -n 20The journal contains the start-up of the API, each line prefixed with chatex-api[<pid>]::
… chatex-api[252]: Microsoft.Hosting.Lifetime[14] Now listening on: http://127.0.0.1:5000
… chatex-api[252]: Microsoft.Hosting.Lifetime[0] Application started. Press Ctrl+C to shut down.
… chatex-api[252]: Microsoft.Hosting.Lifetime[0] Hosting environment: ProductionCheck
ls /var/lib/chatexlistskeysanduploads: the API created its Data Protection key on the first start; uploaded files appear underuploadsonce someone sends an attachment.- From your computer,
curl.exe -i http://<droplet ip>/healthanswers400 Bad Request: the bare IP address is not inAllowedHosts, so the API refuses requests that do not name your domain. curl.exe -I https://<your domain>/showsCache-Control: no-cacheand the security headers; a hashed file such ashttps://<your domain>/main-<hash>.js(take the name from the page source) showsCache-Control: public, max-age=31536000, immutable.
From now on, every merge into main goes the same way: CI, package, your approval, deploy. A typical deployment interrupts the API for a few seconds while it restarts; the Angular client reconnects to the hub by itself (topic 20).
Step 3. Everyday commands
These commands are collected in Docs/Deployment.md under "Useful commands".
Follow the API logs live (Ctrl+C stops following, not the API):
bash
journalctl -u chatex-api -fShow the active release:
bash
readlink /opt/chatex/currentRestart the API, for example after changing a setting in /etc/chatex/chatex.env:
bash
systemctl restart chatex-apiCheck the disk usage — the database, the uploads and the kept releases grow over time; a droplet's disk is small:
bash
df -h / && du -sh /var/lib/postgresql /var/lib/chatex/uploads /opt/chatex/releasesBack up the database to a file:
bash
sudo -u postgres pg_dump -Fc chatex > chatex.dump-Fc is PostgreSQL's compressed custom format, which pg_restore can load, also table by table. A backup on the same disk does not survive the loss of the server: copy it to your computer, from your computer:
powershell
scp root@<droplet ip>:chatex.dump .The uploaded files in /var/lib/chatex/uploads belong to the backup too: the database only stores their paths.
Background work of the API
Two parts of the application help the operator without any command. MaintenanceService (topic 13) runs every hour and deletes uploads that were never sent and refresh tokens that expired or were revoked long ago. And /health checks the database on a plain connection without retries, so it answers 503 within seconds when PostgreSQL is down — which is what makes the automatic rollback and monitoring possible.
Step 4. Rolling back
Why. Sometimes a release passes every check and still has a bug users notice. The fastest fix is to go back to the previous release and repair the bug calmly.
Do. List the kept releases, newest first:
bash
ls -t /opt/chatex/releasesActivate the one you want:
bash
chatex-activate <commit>The script finds no upload for this commit and re-activates the existing release:
[chatex-activate] no upload for <commit>, re-activating the existing release
[chatex-activate] applying migrations
…
No migrations were applied. The database is already up to date.
Done.
[chatex-activate] switching to <commit>
[chatex-activate] <commit> is liveRolling back does not undo migrations
The older release runs on the newer database schema. That works because migrations are written to be compatible with the previous release (expand–contract, see Activating a release). If a migration was destructive, a rollback needs a database backup, which is why the backup comes before risky releases.
Remember that main still contains the bad commit: the next merge deploys a fixed version and replaces the rollback. To deploy main again without a new commit, use Actions → Deploy → Run workflow.
Step 5. Rotating secrets
Why. Secrets leak: a screenshot, a laptop that is lost, a teammate who leaves. Each secret must be replaceable in a few minutes, without a new release.
The JWT signing key
Generate a new key and put it into the settings file:
bash
openssl rand -base64 48bash
nano /etc/chatex/chatex.envbash
systemctl restart chatex-apiAll access tokens signed with the old key become invalid at once. Users do not notice: the client receives 401, refreshes its token with the refresh cookie and repeats the request (topic 16). Refresh tokens are random values stored as hashes, independent of the signing key, so the sessions survive. To end all sessions as well, revoke the refresh tokens in the database.
The database password
bash
sudo -u postgres psql -c "ALTER ROLE chatex PASSWORD 'NEW_PASSWORD'"Then replace the password in ConnectionStrings__Chatex with nano /etc/chatex/chatex.env and restart the API with systemctl restart chatex-api. Do the three steps one after another: between the first and the last, new database connections of the running API would fail.
The deploy key
Create a new key pair on your computer as in Access for GitHub, append the new public key to /root/.ssh/authorized_keys, replace the secret SSH_PRIVATE_KEY, and deploy once with Run workflow to prove that it works. Then remove the line of the old key from authorized_keys with nano. If the server itself was rebuilt, its host key changed: update SSH_KNOWN_HOSTS too, or the job stops at the host key check — as it should.
Step 6. Deploying by hand
Why. If GitHub Actions is unavailable, or you deploy to a test machine, the release can be built and activated without the workflow. The steps are the same as in the workflow, done by you — including the responsibility to deploy only a commit that passed CI.
Do. From the repository root on your computer, delete an old release/ folder first if there is one, then build the release:
powershell
dotnet publish Source/Chatex.Api -c Release -r linux-x64 --self-contained false -o release/apipowershell
dotnet tool restore && dotnet ef migrations bundle --project Source/Chatex.Data --configuration Release -r linux-x64 -o release/efbundle --forcepowershell
npm --prefix Source/Chatex.Web ci && npm --prefix Source/Chatex.Web run build && cp -r Source/Chatex.Web/dist/chatex/browser release/webA known mistake in the reference documentation
Docs/Deployment.md writes the bundle command with -c Release. For dotnet ef, -c is the short form of --context, not of --configuration, so that command stops with No DbContext named 'Release' was found. Use --configuration Release as above and as deploy.yml does.
The release name must be a commit hash; take the short form of the current commit:
powershell
git rev-parse --short HEADUpload the release into incoming/ and activate it:
powershell
scp -r release root@<your domain>:/opt/chatex/incoming/<commit>powershell
ssh root@<your domain> chatex-activate <commit>The output is the same as in the workflow log. A release built this way has no REVISION file; its folder name tells you the commit.