Most connection-pooling advice was written for a world of big, long-lived application servers: size your pool, warm it at boot, tune it quarterly. A scale-to-zero microservice platform inverts every assumption in that paragraph. Instances are small, appear on demand, multiply under load and evaporate minutes later. Meanwhile the databases they connect to are shared, stateful and — in the case of our always-on graph database — have connection limits that don't care how clever your architecture diagram is.
The serverless pooling paradox
Here's the tension. Pooling exists to amortize expensive connection setup across many requests. Serverless platforms spawn and kill instances so aggressively that a pool might live for only a few minutes — yet within those minutes an instance may serve thousands of requests, each of which would otherwise pay a TLS handshake and an auth round-trip. So pooling still pays for itself many times over; what changes is where the pool lives and how big it dares to be. The unit of pooling is the instance, the pools must be born lazily, and the sum of all pools across all instances must stay under what the database will tolerate.
One driver per process, one session per request
The pattern that holds everything together is boring and load-bearing: every external client is a module-scope singleton, created on first use — never at import time, for reasons covered in the cold-start entry — and reused for the life of the process. The graph database driver is the textbook case: the driver object holds the actual connection pool and is expensive to construct, so it's built exactly once per instance; the per-request unit of work is a lightweight session borrowed from that pool and returned immediately. Confusing those two lifetimes — building a driver per request — is the classic mistake, and it doesn't show up in local testing where "per request" and "per process" feel identical. It shows up in production, as connection exhaustion at the worst possible hour.
Tokens are connections too
The subtler version of the same lesson: authentication tokens. Service-to-service calls here are authenticated with short-lived identity tokens fetched from the platform's metadata service. Fetching one is a network call — cheap, but not free, and absurd to repeat per request when the token is valid for the better part of an hour. Same medicine: a module-scope cache keyed by audience, refreshed shortly before expiry. Anything that costs a round-trip and lives longer than a request — tokens, JWKS key sets, remote configuration — earns a process-lifetime cache with an expiry-aware getter.
The wake-up stampede
The failure mode unique to scale-to-zero: a burst of traffic lands on a sleeping service, the platform obligingly starts several instances at once, and every one of them initiates its database connections simultaneously — a self-inflicted thundering herd. The mitigations are unglamorous: keep per- instance pool sizes modest (an instance capped at a few dozen concurrent requests does not need a hundred connections — it needs single digits), cap maximum instance counts per service at numbers the database's connection budget can absorb even in the worst case, and let retries back off with jitter so a failed first connection doesn't synchronize into waves. None of this is exotic; all of it had to be actually done, service by service, with the math written down.
Know what your SDK already does
Half the pooling mistakes I've seen — including mine — come from not knowing which layer already owns the problem. The document database's client library multiplexes requests over a small set of long-lived channels entirely on its own; wrapping it in a hand-rolled "pool" of clients doesn't add throughput, it adds file descriptors and confusion. The graph driver, by contrast, genuinely needs its lifecycle managed by the application. Before writing any pooling code, read what the SDK does out of the box: the best connection management is quite often the code you delete.
Rules I now follow
- Clients are singletons; work units are per-request. Know which is which for every library you use.
- Connect lazily, cache aggressively, expire gracefully. First use pays; everyone after rides free.
- Budget connections globally, not per instance: max instances × pool size must fit the database's ceiling with room to spare.
- Jitter every retry. Synchronized failure is worse than failure.
- Log connection creation. If the logs show connections being built in a hot path, something regressed — the pattern is only as strong as its least careful call site.
The deeper story of what those connections talk to — and why there are two databases on the other end in the first place — is the subject of documents and graphs.