Request hedging is a second GET, not a bigger pool

The festival sale goes live and a thousand people open the same homepage inside a minute: a pair of size-8 white sneakers. Every tap is one request to my API.

The API reads Redis, then the database. Three things go wrong on that path, and the fix for the third one makes the second one worse.

Opening a socket per page

The first version opened a database connection, read the row, and closed it.

Opening is not free. TCP will not carry a query until a connection exists. That takes three packets: the API sends SYN (“I want to start”), the database replies SYN-ACK (“yes”), the API sends ACK. Only then does the query leave.

Closing is four packets: FIN and ACK both ways. When that is over, the next request starts from nothing.

Time diagram from the API to the database: SYN, SYN-ACK, ACK, then the query for the sneakers and the row, then FIN, ACK and FIN, then ACK. — Figure 1. Handshake, then the query, then teardown. Every request.

One buyer, one handshake, one query, one teardown. A thousand taps means a thousand of those. Closed sockets also linger in TIME-WAIT so leftover packets do not hit a new connection on the same port. The query was a few bytes. The ceremony was the load.

Four connections that never hang up

So the process opens four connections at startup and keeps them. Those four pay the handshake once. That set is the connection pool.

A request borrows one, runs the query, and hands it back. No FIN, no new SYN.

A thousand buyers share those four. If all four are busy, the fifth waits. That wait is the point: at most four queries at once, so a spike cannot become a thousand simultaneous reads.

The pool does not make the database faster. It only removes the handshake from the tap.

At process start the API does four handshakes and holds a pool. Each GET checks out, queries the sneakers, and returns the connection. No FIN and no new SYN. — Figure 2. A thousand users reuse four handshakes.

Redis has never heard of the sneakers

The page is still slow if every request hits the database. The API looks in Redis first, under sneakers:8-white. Hit: answer from memory, pool untouched. Miss: Redis returns nil, the API reads the row and writes it back.

The sale flips the homepage. Redis has no sneakers:8-white. A thousand requests arrive.

Every one asks Redis, gets nil, borrows a pool connection, runs the same query. The pool fills with copies of one read. Other products wait behind the sneakers.

The pool saved the handshake. It did nothing about a thousand identical reads.

Many users send GET sneakers to the API. Redis returns nil. The pool and database run the same row many times and pages get slow. — Figure 3. Redis had nothing. Every GET became a database read.

The fix is a lock. The first request to see the miss claims the right to fill sneakers:8-white. It reads the row, writes Redis, lets go. The others wait, then ask Redis again. One database read. Three pool connections stay free.

A lock for the sneakers key. Worker w1 holds it and fills the cache. Workers w2, w3, and w4 wait and then get a cache hit. The pool has one connection busy on sneakers and three free for other keys. — Figure 4. The lock is for the fill. The pool is for the query.

Skip the lock and the page stays slow. A slow page is when the third idea looks like courage.

Sending the same request twice

Hedging: the user tapped once. The API is waiting on the database. Fifty milliseconds later it fires a second copy of the same read. First answer wins. The loser should be cancelled.

That helps when one path is sick: a bad host, a bad disk. It pulls in the slowest few percent, the tail.

On this flood both copies see the same nil. Both borrow from the same pool. One tap holds two of four connections. A thousand slow taps become two thousand borrowings.

The first request was slow because the key was empty. The hedge doubled the pile.

One user click. At t=0 the API GETs the sneakers, Redis is nil, goes to the database. At 50ms it sends the same GET again. First answer wins. Both copies sit on the pool. — Figure 5. Hedging is a second GET. Redis is still empty for both.

Hedging only helps if the second attempt can land somewhere different, and if the loser actually gives the connection back. Neither was true here. And a hedge never writes Redis, which is what the page needed.

The handshake is why the pool exists. The empty key is why the lock exists. Hedging is a second GET. It does not fill the key.

I still size the pool by user count instead of by how many queries the database can run at once. I still let the losing hedge keep running.

More API boxes do not turn those four sockets into a safe MySQL budget. I wrote that when the shop outgrew one process. A Redis lock that fills sneakers:8-white is still not the row. I wrote that when two people booked the same seat.

If this is useful, wrong, or incomplete, write to me.