Self-host ZUI Cloud

This guide is for platform teams operating Config Store, PDF Export, and Scheduled Delivery in their own environment. It describes the deployment contract implemented by the current release, not an aspirational architecture.

All three modules ship in one Go process and share one SQLite database, authentication boundary, console, and API origin. PDF Export and PDF schedules also require the built web renderer and a compatible Chromium binary. You cannot currently install or scale the modules as independent services.

:::caution Paid pilot topology

The hosted ZUI Cloud paid pilot uses this same single-node topology: one active process, SQLite, and local artifact storage. Its Business and Platform limits are commercial capacity allowances, not a promise of active-active failover or horizontal scaling. The current pilot requires workload qualification and an agreed backup and restore target before production use.

:::

Supported production topology

Run one active ZUI Cloud process with its SQLite database and artifact directory on locally attached persistent storage. Put a TLS reverse proxy in front of the process and bind the process itself to loopback. If the proxy is on another host, bind to 0.0.0.0, firewall the port to that proxy, and retain loopback reachability: the internal PDF renderer loads http://127.0.0.1:{port}/render.html.

                         HTTPS
consumer backends ──▶ reverse proxy ──▶ ZUI Cloud :8790
                                           │
                          ┌────────────────┼────────────────┐
                          ▼                ▼                ▼
                      SQLite WAL       Chromium       artifact files
                    authoritative     one process     scheduled output
                                           │
                                           ▼
                                  optional Redis buffer
                                for Scheduled Delivery

The current database driver deliberately uses one SQLite connection. Do not place the database on NFS, mount it into multiple replicas, or assume that a load balancer plus Redis makes the service highly available. Redis carries identifier-only execution jobs; SQLite remains authoritative for accounts, keys, configs, schedules, runs, and attempts. The built-in artifact store is also local filesystem storage.

Use vertical scaling and a supervised single process for this release. An active-active deployment requires a different supported relational store and a shared ArtifactStore implementation in addition to Redis; those are code changes, not environment-only settings.

Production prerequisites

Requirement	Why it is required
Linux host or VM	The checked-in service unit and release layout target a conventional Linux host.
Node.js 22.13 or newer and pnpm	Builds the console and renderer; installs the pinned Playwright Chromium. Node is not needed to execute the Go binary after installation.
Go 1.25	Builds the server. `CGO_ENABLED=0` produces a portable Linux binary because the service uses a pure-Go SQLite driver.
SQLite CLI	Performs online backups and integrity checks. It is an operator tool; the running service uses its embedded pure-Go driver.
Chromium installed by Playwright	PDF Export and PDF schedules use Chrome DevTools Protocol. The process validates the executable during startup.
Local persistent disk	Stores the SQLite database and, by default, scheduled artifacts. Include both in backup and capacity plans.
TLS reverse proxy	Terminates HTTPS, limits body size, applies connection timeouts, and keeps port `8790` private.
Optional Redis	Provides a process-shared, restart-recoverable execution buffer for Scheduled Delivery. It is not the database or an HA switch.

Size the first host from measurements, not request counts alone. PDF memory and CPU depend on node count, charts, fonts, dataset size, page count, and ZUI_CLOUD_PDF_CONCURRENCY. Start with concurrency 2, run representative documents, and leave enough memory for two simultaneous Chromium contexts plus the browser, Go process, kernel, and backup operations.

Build an immutable release

From a clean checkout of the exact version you intend to operate:

corepack enable
pnpm install --frozen-lockfile
pnpm build
pnpm --filter @zuilib/cloud build:web

cd packages/cloud
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
  go build -trimpath -ldflags="-s -w" \
  -o dist/server/zui-cloud ./cmd/zui-cloud

Use arm64 instead of amd64 only when it matches the runtime host. Treat dist/server/zui-cloud and the complete dist/web directory as one release: the browser renderer imports the ZUI packages built from that same source tree.

Install Chromium and its distribution dependencies on the runtime host. Keep the browser path outside release directories so releases can be swapped without downloading Chromium each time:

PLAYWRIGHT_BROWSERS_PATH=/opt/zui/cloud/browsers \
  pnpm --filter @zuilib/cloud exec \
  playwright install --with-deps chromium

--with-deps may require root on Debian-family systems. If your image build cannot run package installation, install the dependencies in a privileged image stage and run ZUI Cloud as an unprivileged user in the final stage.

Before promotion, run the service tests and a real-browser render test:

pnpm --filter @zuilib/cloud test
pnpm --filter @zuilib/cloud build
pnpm --filter @zuilib/cloud test:e2e

Filesystem layout and ownership

The repository deployment uses this layout; the same separation works with any release mechanism:

/opt/zui/cloud/
├── current -> releases/20260906153000/app
├── releases/
│   └── 20260906153000/app/
│       └── dist/{server,web}
├── browsers/                       # Playwright browser cache
├── data/
│   ├── pro-services.sqlite         # plus transient -wal and -shm files
│   └── artifacts/                  # scheduled output, mode 0600
└── cloud.env                # secrets; mode 0600

Own the tree with a dedicated system user such as zui-cloud. Release files may be read-only. The service needs write access only to data, its temporary directory, and the Chromium runtime paths. Never put the database, artifact directory, or environment file inside an atomically replaced release.

Configure the process

Use an environment file readable only by the service account. Paths shown here are production examples.

Shared service and PDF Export

Variable	Default	Production guidance
`ZUI_CLOUD_HOST`	`127.0.0.1`	Keep loopback when the reverse proxy is on the same host. If a remote proxy requires `0.0.0.0`, restrict the port by firewall; binding only a non-loopback address breaks the internal renderer.
`ZUI_CLOUD_PORT`	`8790`	Expose only through the reverse proxy.
`ZUI_CLOUD_DATABASE_URL`	`./data/pro-services.sqlite`	Use an absolute path on persistent local disk.
`ZUI_CLOUD_WEB_DIR`	`./dist/web`	Point at the `dist/web` from the same release as the binary.
`ZUI_CLOUD_PDF_CHROMIUM_EXECUTABLE`	auto-discovered	Set an absolute executable path when the runtime cannot discover the Playwright cache.
`ZUI_CLOUD_SESSION_TTL_HOURS`	`720`	Choose a console-session lifetime consistent with your access policy. Existing sessions are checked against their stored expiry.
`ZUI_CLOUD_ALLOW_SIGNUP`	`true`	Bootstrap the first owner, then normally set `false` for a private deployment.
`ZUI_CLOUD_CORS_ORIGINS`	unset	Exact comma-separated browser origins. Server-to-server callers do not need CORS. `localhost` is a development convenience, not a production value.
`ZUI_CLOUD_DEMO_KEY`	unset	Creates or maintains a public demo organization. Leave unset unless a deliberately quota-limited public demo is required.
`ZUI_CLOUD_STRIPE_SECRET_KEY`	unset	Hosted paid-pilot only. Server-side Stripe credential for Checkout and billing-portal requests. Keep it in a secret manager.
`ZUI_CLOUD_STRIPE_WEBHOOK_SECRET`	unset	Hosted paid-pilot only. Verifies Stripe webhook signatures. Keep it separate from the API credential.
`ZUI_CLOUD_STRIPE_BUSINESS_MONTHLY_PRICE_ID`	unset	Stripe recurring Price whose lookup key is `zui_cloud_business_monthly`.
`ZUI_CLOUD_STRIPE_BUSINESS_ANNUAL_PRICE_ID`	unset	Stripe recurring Price whose lookup key is `zui_cloud_business_annual`.
`ZUI_CLOUD_STRIPE_PLATFORM_MONTHLY_PRICE_ID`	unset	Stripe recurring Price whose lookup key is `zui_cloud_platform_monthly`.
`ZUI_CLOUD_STRIPE_PLATFORM_ANNUAL_PRICE_ID`	unset	Stripe recurring Price whose lookup key is `zui_cloud_platform_annual`.
`ZUI_CLOUD_STRIPE_PORTAL_CONFIGURATION_ID`	unset	Optional explicit customer-portal `bpc_…` configuration; otherwise Stripe's account default is used.
`ZUI_CLOUD_BILLING_RETURN_URL`	`https://cloud.zuilib.com/`	Hosted paid-pilot only. Set the absolute HTTPS console URL used after Checkout and billing-portal actions explicitly.
`ZUI_CLOUD_PDF_CONCURRENCY`	`2`	Maximum active Chromium contexts. Raise only after load and memory testing.
`ZUI_CLOUD_PDF_QUEUE_LIMIT`	`50`	Maximum waiting renders; excess work receives `503 busy`. Bound it to protect latency and memory.
`ZUI_CLOUD_PDF_BODY_LIMIT_MB`	`25`	API body limit. Keep the reverse proxy limit slightly higher so the API can return its JSON `413` contract.
`ZUI_CLOUD_PDF_DEFAULT_MONTHLY_QUOTA`	`1000`	Quota assigned to newly created organizations. It is not a global capacity limit.
`ZUI_CLOUD_PDF_RENDER_TIMEOUT_MS`	`60000`	Per-render browser deadline. Set above measured p99 but below upstream request timeouts.

Scheduled Delivery

Variable	Default	Production guidance
`ZUI_CLOUD_SCHEDULE_ENABLED`	`true`	Starts sweep, recovery, and worker loops. The schedule API remains registered when this is `false`, but manual execution returns `503`.
`ZUI_CLOUD_SCHEDULE_REDIS_URL`	unset	Optional Redis execution buffer. Use TLS/auth where the Redis service supports it.
`ZUI_CLOUD_SCHEDULE_SWEEP_INTERVAL_MS`	`60000`	How often due schedules are materialized.
`ZUI_CLOUD_SCHEDULE_LOOKAHEAD_MS`	`120000`	How far ahead occurrences are materialized. Keep it larger than the sweep interval.
`ZUI_CLOUD_SCHEDULE_REAPER_INTERVAL_MS`	`300000`	How often durable state is inspected for work missing from the execution buffer.
`ZUI_CLOUD_SCHEDULE_STALE_THRESHOLD_MS`	`900000`	Claim age treated as orphaned. Keep above the p99 generation or provider latency.
`ZUI_CLOUD_SCHEDULE_JITTER_WINDOW_SECONDS`	`300`	Stable per-schedule spread after nominal execution time to avoid a thundering herd.
`ZUI_CLOUD_SCHEDULE_MAX_CONSECUTIVE_FAILURES`	`5`	Unsuccessful runs before the schedule is suspended.
`ZUI_CLOUD_SCHEDULE_ARTIFACT_DIR`	`./data/artifacts`	Use an absolute path on the backed-up persistent disk.
`ZUI_CLOUD_SCHEDULE_DELIVERY_MODE`	`simulate`	Keep `simulate` through acceptance testing; `live` enables external Email, SFTP, and Slack effects.
`ZUI_CLOUD_SCHEDULE_GENERATOR_ORIGINS`	`http://127.0.0.1:8081`	Exact origins permitted for `zui-view-url`; use HTTPS and the narrowest possible list.
`ZUI_CLOUD_SCHEDULE_PUBLIC_BASE_URL`	unset	Public HTTPS origin used to create signed artifact links.
`ZUI_CLOUD_SCHEDULE_SIGNING_SECRET`	unset	High-entropy HMAC secret required for signed links. Rotation invalidates links signed with the previous value.

Delivery and generator credentials use environment references such as env:FINANCE_REPORT_EMAIL. Their values are JSON secrets described in the Scheduled Delivery guide. Environment files are read when the process starts, so restart the service after rotating one.

An initial production file might be:

ZUI_CLOUD_HOST=127.0.0.1
ZUI_CLOUD_PORT=8790
ZUI_CLOUD_DATABASE_URL=/opt/zui/cloud/data/pro-services.sqlite
ZUI_CLOUD_WEB_DIR=/opt/zui/cloud/current/dist/web
ZUI_CLOUD_ALLOW_SIGNUP=false
ZUI_CLOUD_SESSION_TTL_HOURS=24

ZUI_CLOUD_PDF_CONCURRENCY=2
ZUI_CLOUD_PDF_QUEUE_LIMIT=50
ZUI_CLOUD_PDF_BODY_LIMIT_MB=25
ZUI_CLOUD_PDF_DEFAULT_MONTHLY_QUOTA=1000
ZUI_CLOUD_PDF_RENDER_TIMEOUT_MS=60000

ZUI_CLOUD_SCHEDULE_ENABLED=true
ZUI_CLOUD_SCHEDULE_DELIVERY_MODE=simulate
ZUI_CLOUD_SCHEDULE_ARTIFACT_DIR=/opt/zui/cloud/data/artifacts
ZUI_CLOUD_SCHEDULE_GENERATOR_ORIGINS=https://reports.internal.example
ZUI_CLOUD_SCHEDULE_PUBLIC_BASE_URL=https://cloud.example.com
ZUI_CLOUD_SCHEDULE_SIGNING_SECRET=replace-with-at-least-32-random-bytes

Do not commit a populated environment file. Prefer a secret manager that can render a root-owned file immediately before service start. Avoid putting API keys or provider credentials directly in a unit file, container manifest, or command line where process inspection can expose them.

Run under a supervisor

The repository includes a systemd unit at deploy/cloud/zui-cloud.service. Its important properties are a dedicated user, an explicit working directory and environment file, automatic restart, a private temporary directory, and NoNewPrivileges=true.

After installing the unit:

sudo systemctl daemon-reload
sudo systemctl enable --now zui-cloud
sudo systemctl status zui-cloud
sudo journalctl -u zui-cloud --since "10 minutes ago"

Additional systemd hardening must preserve write access to the data and Chromium temporary paths. Add restrictions incrementally and prove PDF Export, SFTP, Email, Slack, backup, and graceful shutdown in a staging environment.

Containers must preserve the same contract: run as a non-root UID, mount the database and artifacts on persistent local volumes, include Chromium runtime dependencies, set a writable temporary directory, and use exactly one replica. The repository does not currently publish a canonical container image or Kubernetes manifest.

Terminate TLS at a reverse proxy

The service speaks HTTP. A minimal nginx proxy is:

server {
    listen 443 ssl http2;
    server_name cloud.example.com;

    client_max_body_size 30m;

    location / {
        proxy_pass http://127.0.0.1:8790;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_connect_timeout 5s;
        proxy_read_timeout 120s;
        proxy_send_timeout 120s;
        proxy_buffering off;
    }
}

Set the proxy read timeout above ZUI_CLOUD_PDF_RENDER_TIMEOUT_MS plus queueing and network overhead. Rate-limit authentication endpoints and public signed artifact downloads at the edge, but do not retry POST requests automatically in the proxy. Restrict direct access to port 8790 with host and network firewalls.

Bootstrap an organization and API key

On a private installation, temporarily start with ZUI_CLOUD_ALLOW_SIGNUP=true, create the first owner, create an API key, then set sign-up to false and restart. The console at / performs the same calls as this API flow.

curl --fail-with-body https://cloud.example.com/v1/auth/signup \
  -H 'Content-Type: application/json' \
  --data '{
    "email": "platform-owner@example.com",
    "password": "use-a-long-random-password",
    "name": "Platform Owner",
    "organizationName": "Example Engineering"
  }' > signup-response.json

SESSION_TOKEN="$(jq -r .token signup-response.json)"

curl --fail-with-body https://cloud.example.com/v1/api-keys \
  -H "Authorization: Bearer ${SESSION_TOKEN}" \
  -H 'Content-Type: application/json' \
  --data '{"name":"production-backend"}' > api-key-response.json

The full API-key secret appears only in api-key-response.json. Import it into your secret manager immediately, restrict the file, and remove the local copy through your approved secure-deletion workflow. API-key listing returns only a hint. Use separate keys for separate applications or environments so one can be revoked without a coordinated outage.

Acceptance tests before live traffic

Run these checks through the public TLS endpoint:

GET /health returns the expected version and scheduler state.
Sign-up is closed after bootstrap if the deployment is private.
An organization key can create and resolve a disposable Config Store document, including a deliberate 409 version_conflict test.
A representative PDF renders, has the expected page count and fonts, and exposes non-empty X-Request-Id and X-Render-Ms headers.
A schedule in simulate mode reaches completed, creates an artifact, and retains delivery attempt evidence after a process restart.
CORS permits only intended browser origins. Server-to-server calls work independently of CORS.
A revoked test key receives 401 api_key_invalid.
Backup and restore are proven in an isolated host before the first production release.

The module guides provide complete requests for Config Store, PDF Export, and Scheduled Delivery.

Health checks and observability

GET /health is an unauthenticated process liveness endpoint:

{
  "ok": true,
  "version": "0.1.0",
  "pending": 0,
  "scheduledDelivery": true
}

pending is the number of active plus waiting PDF renders. The health route does not execute a database query, start Chromium, verify artifact storage, connect to Redis, or contact delivery providers. Do not treat it as a complete readiness test.

Use three layers of monitoring:

Layer	Check	Alert signal
Process	Poll `/health`; supervise the process; collect stdout/stderr logs.	Endpoint failure, restart loop, unexpected version, scheduler disabled.
Module	Periodically resolve a canary config and render a small canary PDF with a dedicated low-privilege key.	Contract failure, render latency, non-zero render issues, quota exhaustion.
Workflow	Create or monitor a non-production schedule and inspect run/delivery history.	Runs remain non-terminal beyond the stale threshold, repeated failures, suspension, missing artifacts.

The current process emits structured key/value logs through Go slog, but it does not expose a Prometheus endpoint or distributed traces. Retain logs at the platform layer. Correlate PDF failures with X-Request-Id; correlate scheduled work with schedule, run, delivery, and attempt IDs returned by the API.

Define service-level objectives around the contract clients observe: API availability, config read/write latency and conflicts, PDF success/latency and queue rejection, scheduled occurrence delay, delivery completion, and artifact availability. Never classify an expected 409 or a permanent provider rejection as infrastructure unavailability.

Backup and restore

The minimum recoverable set is:

pro-services.sqlite, captured with a SQLite-aware backup;
the complete ZUI_CLOUD_SCHEDULE_ARTIFACT_DIR when Scheduled Delivery is used;
the environment and secret-manager configuration needed to reconstruct the process, stored in your approved secret backup system;
the exact release artifact or source revision.

SQLite WAL files are not a backup strategy. Do not copy only the main database file while the process is writing. For a database-only hot backup, use the SQLite online backup command. For a point-in-time backup consistent with local artifacts, prefer a short maintenance window:

sudo systemctl stop zui-cloud
sudo -u zui-cloud sqlite3 \
  /opt/zui/cloud/data/pro-services.sqlite \
  ".backup '/opt/zui/cloud/data/cloud.backup.sqlite'"
sudo systemctl start zui-cloud

Copy the backup database and artifact tree to encrypted off-host storage after the process restarts. Record checksums, encrypt in transit and at rest, and apply a retention policy consistent with the reports' data classification. The service does not currently prune render request history, schedule history, or artifact files automatically; monitor both database and artifact growth.

To restore, stop the service, preserve the failed state for forensics, restore the database and matching artifact snapshot into empty paths, apply the service user ownership and restrictive modes, select the compatible application release, and start the process. Then verify:

sqlite3 /opt/zui/cloud/data/pro-services.sqlite \
  'PRAGMA integrity_check; PRAGMA foreign_key_check;'
curl --fail-with-body http://127.0.0.1:8790/health

Finish with one authenticated config read, a synthetic PDF, an artifact download, and inspection of the next scheduled occurrences. Run a restore exercise on a fixed cadence; an untested backup is not a recovery plan.

Upgrade and rollback

Database migrations run automatically and transactionally at process startup. They are append-only, but a newer schema is not promised to be readable by an older binary.

Use this sequence:

Read release notes and test the new release against a sanitized copy of the production database and representative reports.
Capture a database-plus-artifact backup and record the current release.
Build and stage a new immutable release without changing current.
Stop the process, atomically switch current, and start it.
Gate promotion on /health, authenticated config access, a canary PDF, and schedule/runtime checks.
Keep at least the previous release and the pre-upgrade backup until the observation window closes.

If startup or acceptance fails before a migration, point current back to the previous release. After a migration, the safest rollback is the previous binary and its paired pre-upgrade database/artifact backup. Do not improvise a schema downgrade in production.

Graceful shutdown allows 15 seconds for HTTP requests after scheduler workers are stopped. Align the supervisor's stop timeout above that value. Durable schedule state is reaped on restart, but clients should still use normal timeouts and retry only according to each module's documented semantics.

Security baseline

Expose only HTTPS and keep the process port private.
Keep organization keys in consumer backends. CORS is not an authorization control and does not make a browser-held API key safe.
Disable open sign-up after bootstrap unless it is an intentional product feature. Use a dedicated owner account and long random password.
Issue keys per application and environment, audit lastUsedAt, rotate by creating a replacement before revoking the old key, and test revocation.
Treat X-ZUI-User-ID and X-ZUI-Group-IDs as signed assertions from a trusted backend. Never forward arbitrary client-supplied values.
Store the database, artifacts, backups, render payloads, and logs according to the highest classification of data they may contain.
Keep credentials out of AppSpec, ViewSpec, schedule target config, and generator request bodies. Use env: references for provider and generator secrets.
Restrict egress to required provider, SFTP, Slack, and generator endpoints. PDF render tabs block network access other than their own renderer origin; application data must arrive in the payload.
Keep ZUI_CLOUD_SCHEDULE_GENERATOR_ORIGINS exact. Validate authorization and current data again in every generator request.
Patch the host, Chromium, Go build, Node build toolchain, and reverse proxy on a controlled cadence. Re-run PDF visual regression tests after Chromium changes.

Incident runbooks

Symptom	First checks	Safe response
Process will not start	Journal, database path/permissions, web directory, Chromium discovery, environment syntax.	Restore the previous immutable release; restore the paired database only if a migration occurred.
`503 busy` from PDF Export	`/health.pending`, CPU, memory, render p95/p99, client burst rate.	Shed/retry with jitter at clients; lower incoming concurrency. Raise service concurrency only after headroom testing.
PDF timeouts	Request history and ID, dataset/body size, page count, custom fonts, `settleMs`, host pressure.	Fix or split the report; increase timeout only when measured normal work legitimately exceeds it.
Schedules stop advancing	Scheduler flag, process logs, Redis reachability, sweep/reaper timing, `nextRunAt`.	Restore Redis or restart the supervised process; durable SQLite state lets the reaper reconstruct missing jobs.
Delivery repeatedly fails	Run attempt evidence, provider status, secret availability, egress/DNS, SFTP host-key pin.	Correct the target or secret, restart after env rotation, then use the explicit retry endpoint. Do not rewrite attempt history.
Artifact is missing	`artifactKey` in run state, artifact volume mount, backup/restore alignment, file permissions.	Restore the matching artifact snapshot; a database-only restore cannot recreate every historical artifact.
Suspected API-key exposure	Key ID/hint and `lastUsedAt`, edge logs, affected organization.	Create and deploy a replacement, revoke the exposed key, investigate caller logs, and rotate any downstream credentials included in exposed payloads.
SQLite integrity or disk failure	Disk health/capacity, `PRAGMA integrity_check`, service logs.	Stop writes, preserve evidence, and restore the latest verified off-host database-plus-artifact backup.

Production readiness checklist

The checked-in GCE/systemd deployment under deploy/cloud implements this single-node model and can be used as a concrete reference. Adapt its provider-specific release transport without changing the runtime invariants above.

Supported production topology​

Production prerequisites​

Build an immutable release​

Filesystem layout and ownership​

Configure the process​

Shared service and PDF Export​

Scheduled Delivery​

Run under a supervisor​

Terminate TLS at a reverse proxy​

Bootstrap an organization and API key​

Acceptance tests before live traffic​

Health checks and observability​

Backup and restore​

Upgrade and rollback​

Security baseline​

Incident runbooks​

Production readiness checklist​