Workload assumptions
The future envelope is 1M+ simultaneous users. Connected sessions, active users, API requests and running AI jobs have separate planning assumptions. Qualify traffic, storage, queues, inference capacity, failover and costs in stages before each expansion.
Architecture stages
A stage is entered only after its previous workload envelope meets latency, error, correctness and recovery criteria. Add partitioning or read replicas only when measured query and write patterns justify them. Avoid giving every browser a database connection. PostgreSQL pools need a host-wide connection budget across all replicas and workers.
AI and execution capacity
Use durable admission queues, per-tenant concurrency/token budgets, queue deadlines, cancellation and backpressure. Size local inference from measured tokens/second, prompt/context length, output length, model memory and review multiplier. Size sandbox workers from measured case runtime and resource envelopes. Approximate required busy inference slots as admitted jobs/second × measured service seconds/job, then add bounded utilization and redundancy headroom. No useful GPU count or monthly price can be asserted without that workload and hardware benchmark. The current local-only model adapter does not become a remote GPU fleet by configuration. Secure distributed inference, workload identity, provider qualification and data-processing agreements need a separately reviewed adapter. Preserve model-digest checks and separate review/approval boundaries while scaling.
Viral bursts, cost and recovery
Serve immutable public assets through a CDN; reserve private API paths from public caching. Apply quotas, request bounds, tenant fairness and queue admission before expensive work. Autoscaling must have an approved maximum, dependency limits and rollback/backpressure behavior. Database saturation, GPU queues, logging/egress costs and third-party quotas are practical blockers. Budget alerts are not spending caps. One million 1MB initial downloads is approximately 1TB of transfer before repeat requests. Large-scale operation requires funded capacity, current regional pricing and workload-based estimates. Existing-host operation and future paid expansion are separate stages. See performance qualification, cost boundaries and reliability. References: PostgreSQL connection resources, FastAPI deployment concepts.