Operations
Day-to-day checks. Deeper tooling (K9s, scaling, DB ops) is in Infrastructure → Operations.
Health checks
| Check | How |
|---|---|
| Services | kubectl get pods -n nx-backend - all Running / Ready? |
| Queues (BullMQ) | Kafka UI / Redis - is the confirmation/scheduler backlog draining? |
| Database | pooler reachable; connection count not saturated (data-layer) |
| Cache (Redis) | reachable; memory not at maxmemory |
| Search (Typesense) | collections healthy; CDC connector (Debezium) lag low |
| WebSocket (signal) | clients connecting; Redis pub/sub flowing |
| Versions | match across environments - Platform Facts → versions (alpha vs staging drift is flagged there) |
Routine tasks
- Restart a service:
kubectl rollout restart deploy/<svc> -n nx-backend. - Scale:
kubectl scale deploy/<svc> --replicas=<n>- guidance in infra → Scaling. - Logs: infra → Logs (K9s /
kubectl logs). - DB operations: infra → Database Operations.
Backup & restore
Verify the backup story
Staging currently has no off-cluster DB backup while the deploy docs imply CNPG is running - reconcile this before relying on restore. Target RPO ≤ 15 min (see SLOs).
Restore SOP (once backups are confirmed): stop writers → restore the latest base backup + WAL to the target point → re-point the pooler → verify → resume. Practise it before you need it.