A waiting room, at the edge, for about four minutes twice a year.
A series release puts fifty to a hundred times normal traffic through a website in the time it takes to read a push notification. The honest answer is not more servers. It is a queue that admits people in order, tells them where they are, and holds the rest on a static page that cannot fall over.
The queue, specified.
Where it sits
- At the edge
- Before any application serverA queue that runs on the thing it is protecting is not a queue.
- On a separate origin
- Static, cached, and independently deployedThe waiting room page must survive the failure it exists for.
- Engaged by
- A concurrency threshold, automaticallyNot by somebody watching a graph at eleven at night.
What it tells you
- Your position
- A number, and it only goes downA position that jumps around destroys trust in about four seconds.
- An estimated wait
- In minutes, and deliberately pessimisticBeing admitted early is a pleasant surprise. The reverse is not.
- What you are waiting for
- Named, plainlyA queue with no explanation reads as a broken website.
- What is available now
- The content pages, which are not queued at allReading a chapter does not require a place in the queue.
The rules
- First come, first served
- StrictlyNo priority lane, no paid skip and no member fast track for the queue itself.
- Holding a place
- Survives a refresh, for ten minutesBecause everybody refreshes and punishing it produces a stampede.
- One place per person
- Enforced at the edgeSix tabs does not mean six places.
- Reading is never queued
- Only writes and checkoutThe queue exists to protect a stock service, not an archive.
Why not simply buy more servers.
Because the spike is four minutes long and happens twice a year, and provisioning for it means paying for a hundred times the capacity for three hundred and sixty-four days in order to use it on one. Autoscaling helps and does not solve it — the scale-up takes minutes and the event is minutes.
And because the thing that saturates is not usually compute. It is a stock service, a payment gateway or a database connection pool, and none of those gets better by adding web servers in front of it. Ten thousand people arriving at a checkout that can handle four hundred concurrent sessions is a queueing problem, and queueing problems have a queue-shaped answer.
The queue is also the honest answer. It tells ten thousand people that there are ten thousand of them and roughly how long it will be, which is information they can act on. The alternative tells them the site is broken.
What is queued, and what never is.
| Queued | Why | |
|---|---|---|
| Reading any page | No | Static and cached. It has no capacity limit worth defending |
| The chapter archive | No | The same. It is the thing most people came for |
| The cameras | No | A separate origin with its own capacity and its own failure mode |
| Search | No | A static index file loaded by the browser |
| Adding to a basket | Yes, above the threshold | It touches the stock service, which is the contended resource |
| Checkout | Yes, always, above the threshold | It touches everything, and it is the thing that must not be oversold |
| Creating an account | Yes | A write, and it can wait four minutes |
| The newsletter | No — queued server-side instead | One row, replayed later. Nobody should wait in a queue to give you an email address |
Four minutes, in order.
A notification goes out
And traffic begins climbing before the first paragraph of it has been read.
The threshold is crossed
Concurrency, not requests per second. It is sessions in a checkout that matter, not page views.
The waiting room engages
Static page, separate origin, position and estimate. Reading the site carries on entirely unaffected.
People are admitted in order
At the rate the slowest downstream service can actually take, measured rather than guessed.
It drains
Usually within twenty minutes, and the threshold releases on its own.
There is no queue on this build, because there is nothing to queue for
This site takes no payment, holds no stock and has no release night. The waiting room described here is a design. The one state you can actually see is Queueing, which is one of the fourteen named system states this build does render.
The queue, questioned.
No. Member windows exist and they open earlier, which is a different mechanism entirely. Once the queue is running it is strictly first come, first served, because a visible priority lane in a public queue is the fastest way to make people furious.
Your place is held for ten minutes. Punishing a refresh produces a stampede of people refreshing faster.
Because pages are static and cached and have no capacity limit worth defending. The queue exists to protect a stock service and a payment gateway, not an archive.
And how the whole thing is actually built.
Python generators, plain PHP arrays, and a theme that provisions itself.
