Operations

1. Deployment

Property	Value
Dev port	1190 ⚠️ (siblings use 310x0)
Local nginx upstream	invoice_upstream → 127.0.0.1:31140 (packages/gateway/local/nginx.conf)
Base path	/v1/api (no per-service segment)
Run modes	api, worker (BullMQ issuance + claim-expiry), migrate
Migration	bun run migrate:dev (seeds VN admin data, configs, webhooks, permissions)
Snowflake ID range	⚠️ unset (APP_ENV_..._WORKER_ID not configured)
Service code	SVC-00150-INVOICE (in core ServiceCodes); Kafka id defaults to SVC-00150-INVOICE_CONSUMER

⚠️ Pre-production reconciliation (from package AGENTS): assign a unique SVC code, a snowflake worker id, and a 310x0 port; resolve the SVC-00050 collision with inventory before any production deploy.

Build & run

bun run rebuild      # clean + build (deps: core, iiapi, t-van, mq-pay)
bun run lint:fix     # ESLint + Prettier
bun run server:dev   # run with .env.development
bun run migrate:dev  # migrations / seeds

2. Observability

Signal	Source	Where to look
Logs	stdout (structured key-value)	kubectl logs <pod> / Loki
Issuance audit	invoice.InvoiceAuditTracing	per-invoice event trail (eventType/outcome/before-after)
Health	IGNIS default health route	gateway portal

Key log markers

Marker	Meaning
[handlePaymentSuccess]	payment-driven issuance decision
[handleMerchantCDC] op=u tax unchanged, SKIP	CDC diff short-circuit
[enqueueIssuance] ... partition: %s	partition routing
[_handleIssuanceFailure] Scheduling retry	transient retry with backoff
[DLQ] invoice-issuance	job attempts exhausted → invoice FAILED + audit

3. Security

Concern	Mitigation
AuthN	JWT (ES256, JWKS from identity); VerifierApplication
AuthZ	Permission-gated controllers + AuthorizationService (merchant-level)
Provider credentials	AES-256-GCM, key = exactly 32 bytes / 64 hex (APP_ENV_INVOICE_CREDENTIALS_KEY)
Webhook trust	Two HMAC secrets: WEBHOOK_SECRET (merchant→platform), WEBHOOK_INTERNAL_SECRET (iiapi/commerce→platform, HMAC-SHA256 verified)
Buyer claim token	random UUID, partial-unique, time-boxed (claimDeadline)
Secrets	env / Configuration table (encrypted); never in code
Soft-delete	deletedAt — no hard-delete by default

4. Runbook

4.1 Alert classes

Alert	Trigger	Check	Fix	Escalate
Issuance failures spike	InvoiceAuditTracing FAILED rate ↑	logs [_handleIssuanceFailure] / provider 4xx	inspect provider response payload; fix config/credentials	invoice-team
BullMQ DLQ growth	[DLQ] invoice-issuance logs	Redis queue depth per partition	re-enqueue after fixing root cause	on-call SRE
CDC TaxInfo drift	FE shows stale merchant tax	compare metadata.tax vs TaxInfo	trigger merchant update (re-CDC); verify diff logic	invoice-team
Webhook rejects	401 on /webhooks	signature mismatch	rotate/realign WEBHOOK_* secret	invoice-team

4.2 Common operations

Operation	How
Tail logs	kubectl logs -n <ns> -f deploy/invoice
Inspect invoice state	query invoice.Invoice + InvoiceIssuance + InvoiceAuditTracing by sourceId
Replay a stuck issuance	re-enqueue via issuance flow (jobId = orderId); idempotent on active invoice
Re-sync merchant tax	re-emit/replay Merchant CDC; op=u diff-skips if unchanged
Inspect partition for an order	partition = getPartitionByKey(orderId) (hashCode mod 3)

4.3 Known gotchas / tech debt

Item	Detail
No single transaction	Invoice + InvoiceRequest creation not atomic (payment-success can precede request)
Config snapshot immutable	provider config copied at invoice creation; mid-lifecycle config edits do not apply
Service identity drift	unset code/worker-id/port (see Deployment ⚠️)

Cross-service incidents: see /runbook/.

5. Related Pages

• Configuration
• Architecture
• Decisions