164 lines
5.2 KiB
Markdown
164 lines
5.2 KiB
Markdown
# Deployment
|
|
|
|
This service is intended to run on both webserver VMs, with only the active
|
|
webserver's broker container started. Both installations use the same MariaDB
|
|
Galera database, Gitea application, package registry, and secrets. The
|
|
containers use Linux host networking so `127.0.0.1:3306` reaches each
|
|
webserver's local HAProxy listener.
|
|
|
|
## 1. DNS and TLS
|
|
|
|
Create `tapm.scity.us` in PowerDNS and point it at the existing HAProxy
|
|
frontend. Install the certificate using the existing certificate automation.
|
|
The broker only listens on `127.0.0.1:8080`; Nginx terminates TLS.
|
|
|
|
An example virtual host is in
|
|
[`deploy/nginx/tapm.scity.us.conf`](../deploy/nginx/tapm.scity.us.conf).
|
|
Adjust only the certificate paths if the local convention differs.
|
|
|
|
## 2. MariaDB Galera
|
|
|
|
Create a dedicated database and application users limited to the two webserver
|
|
addresses. Run this once against the cluster as a database administrator,
|
|
replacing the password with the same locally generated value in both accounts:
|
|
|
|
```sql
|
|
CREATE DATABASE tapm_broker
|
|
CHARACTER SET utf8mb4
|
|
COLLATE utf8mb4_unicode_ci;
|
|
CREATE USER 'tapm'@'10.10.1.121' IDENTIFIED BY 'replace-with-a-random-password';
|
|
CREATE USER 'tapm'@'10.10.1.122' IDENTIFIED BY 'replace-with-a-random-password';
|
|
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, REFERENCES
|
|
ON tapm_broker.* TO 'tapm'@'10.10.1.121';
|
|
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, REFERENCES
|
|
ON tapm_broker.* TO 'tapm'@'10.10.1.122';
|
|
FLUSH PRIVILEGES;
|
|
```
|
|
|
|
The Keepalived address `10.10.1.120` is not included because outbound HAProxy
|
|
connections normally originate from the physical webserver address. Add a
|
|
third account for `10.10.1.120` only if a connection test or the HAProxy
|
|
configuration proves that it explicitly uses the virtual address as its source.
|
|
|
|
## 3. Gitea accounts and OAuth
|
|
|
|
Create these separately:
|
|
|
|
1. The `taiadmin` technician account used to sign in to the portal.
|
|
2. A `tapm-packages` service account with read access only to the private
|
|
`TAI/files` package owner. Create a personal access token for this account
|
|
and store it only in the broker environment file.
|
|
3. An OAuth2 application with callback URL
|
|
`https://tapm.scity.us/auth/callback`.
|
|
|
|
The OAuth client secret authenticates the portal to Gitea. The package token
|
|
is used only server-side while proxying an authorized file. Neither value is
|
|
returned to a technician or Proxmox host. The broker requests only Gitea's
|
|
`read:user` OAuth scope, which it uses to confirm the signed-in username
|
|
against `TAPM_ALLOWED_GITEA_USERS`.
|
|
|
|
## 4. Broker configuration
|
|
|
|
On each webserver:
|
|
|
|
```sh
|
|
git clone https://git.scity.us/TAI/TA-Deployment-Broker.git
|
|
cd TA-Deployment-Broker
|
|
cp .env.example .env
|
|
chmod 600 .env
|
|
```
|
|
|
|
Edit `.env` and use the same values on both webservers. Generate the cookie
|
|
secret with:
|
|
|
|
```sh
|
|
openssl rand -base64 48
|
|
```
|
|
|
|
The database DSN must retain `parseTime=true`, `multiStatements=true`, and the
|
|
encoded UTC `time_zone` setting from `.env.example`. Its address remains
|
|
`127.0.0.1:3306`, which uses the local HAProxy frontend rather than selecting a
|
|
single Galera member.
|
|
|
|
## 5. Build and migrate
|
|
|
|
Build the image on each webserver:
|
|
|
|
```sh
|
|
docker compose build
|
|
```
|
|
|
|
Apply migrations from only one webserver:
|
|
|
|
```sh
|
|
docker compose run --rm migrate
|
|
```
|
|
|
|
Migrations use a MariaDB advisory lock, so an accidental simultaneous run
|
|
cannot apply the same migration twice.
|
|
|
|
## 6. Start the active instance
|
|
|
|
On the active webserver:
|
|
|
|
```sh
|
|
docker compose up -d broker
|
|
curl --fail http://127.0.0.1:8080/health/ready
|
|
```
|
|
|
|
On the standby webserver, keep the image and `.env` current but leave the
|
|
broker stopped. The existing failover automation can run:
|
|
|
|
```sh
|
|
docker compose stop broker
|
|
```
|
|
|
|
on the old active VM, followed by:
|
|
|
|
```sh
|
|
docker compose up -d broker
|
|
```
|
|
|
|
on the new active VM. There is no local application state to transfer.
|
|
|
|
## 7. HAProxy health check
|
|
|
|
The load balancer should consider an origin healthy only when
|
|
`GET /health/ready` returns HTTP 200. `/health/live` checks only the process;
|
|
`/health/ready` also verifies Galera connectivity.
|
|
|
|
Only the active webserver should be eligible for traffic. Retaining a single
|
|
active origin avoids duplicate service management while Galera preserves
|
|
sessions and authorization state across failover.
|
|
|
|
## 8. Publish a protected package
|
|
|
|
Publish files to the private Gitea Generic Package Registry:
|
|
|
|
```sh
|
|
curl --user 'publisher:PACKAGE_WRITE_TOKEN' \
|
|
--upload-file ./SentinelAgent_linux_x86_64.deb \
|
|
'https://git.scity.us/api/packages/TAI/generic/sentinelone-linux/26.1.1.31/SentinelAgent_linux_x86_64.deb'
|
|
```
|
|
|
|
Calculate its checksum:
|
|
|
|
```sh
|
|
sha256sum SentinelAgent_linux_x86_64.deb
|
|
```
|
|
|
|
Sign in at `https://tapm.scity.us`, enter the matching registry package name,
|
|
version, filename, and SHA-256, then enable it. The broker intentionally does
|
|
not browse the registry with the technician's credentials.
|
|
|
|
## 9. Backup and rotation
|
|
|
|
- Include `tapm_broker` in the existing Galera backup policy.
|
|
- Keep `.env` outside Git and readable only by the service administrator.
|
|
- Rotate the Gitea package token and OAuth secret if either webserver is
|
|
compromised.
|
|
- Changing `TAPM_COOKIE_SECRET` signs out portal users; it does not invalidate
|
|
active deployment codes.
|
|
- Revoke active deployment authorizations from the portal when a deployment
|
|
ends early.
|