KGS Sequential Generation

Who fills the pool — and how

The key pool in Redis needs to be kept topped up. Something has to generate keys, verify they haven't been used, and push them into Redis before the pool runs dry. That something is the Key Generation Service — and it turns out to be the simplest piece in the entire system.

What the KGS is¶

The Key Generation Service (KGS) is a background worker. Not a managed cloud service. Not a third-party tool. Something you build yourself — a simple script or small service that runs on one cheap server.

Its entire job:

1. Check if the Redis pool is running low
2. Generate a batch of keys
3. Push them into Redis
4. Sleep until the pool needs topping up again

That's it. The KGS handles zero user traffic. It has no scaling pressure — it's not in the request path at all. It runs quietly in the background, wakes up every few hours, fills the pool, goes back to sleep.

The service itself is maybe 50 lines of code.

Why NOT UUID or Snowflake¶

UUID and Snowflake IDs exist to solve one specific problem: generating unique IDs across multiple machines simultaneously, without coordination.

When you have 20 app servers all creating records at the same time, each machine needs to generate an ID that won't collide with any ID generated by the other 19 machines. UUID uses randomness across 122 bits to make collision statistically impossible. Snowflake uses a combination of timestamp + machine ID + sequence number to guarantee uniqueness across machines without any coordination.

The KGS has none of this problem. It is a single process running on one machine. There is no distributed coordination problem to solve. No other process is generating keys at the same time. UUID and Snowflake are solutions to a problem the KGS simply does not have.

Using UUID in the KGS would mean generating 128 bits of randomness, encoding it, trimming it to 6 characters — and reintroducing the exact collision problem you were trying to escape. You'd be back to checking whether the trimmed UUID is already taken.

The KGS doesn't need UUID. It doesn't need Snowflake. It just needs to count.

Sequential base62 generation¶

The KGS walks through the base62 space in order:

000000 → 000001 → 000002 → ... → 000009 → 00000a → 00000b → ... → zzzzzz

Every code is unique by definition — you never generate the same code twice because you never go backwards. No collision check needed within the KGS itself. No randomness needed. No coordination needed. Just increment a counter.

This also means the KGS can resume exactly where it left off after a restart. Store the last generated code in a file or a DB row. On startup, read it and continue from there.

The math — does base62 with 6 chars actually cover 50 billion?¶

The estimation said the system needs to handle 50 billion URLs over 10 years. Let's verify that 6-char base62 is sufficient.

Base62 uses 62 characters: digits 0-9 (10) + lowercase a-z (26) + uppercase A-Z (26) = 62.

Step by step — how many unique codes does each length give?

1 char  → 62^1 = 62
2 chars → 62^2 = 62 × 62 = 3,844
3 chars → 62^3 = 3,844 × 62 = 238,328
4 chars → 62^4 = 238,328 × 62 = 14,776,336        (~14.7 million)
5 chars → 62^5 = 14,776,336 × 62 = 916,132,832    (~916 million)
6 chars → 62^6 = 916,132,832 × 62 = 56,800,235,584 (~56.8 billion)

56.8 billion unique codes from 6 characters.

The requirement is 50 billion. 56.8 billion covers it with 6.8 billion codes to spare — about 13.6% headroom.

Requirement:  50,000,000,000
Available:    56,800,235,584
Headroom:      6,800,235,584  (13.6% spare capacity)

6 characters is exactly the right length. 5 characters gives only 916 million — nowhere near enough. 6 characters clears the 50 billion requirement comfortably.

How the KGS keeps the pool topped up¶

The pool holds 100 million keys. App servers burn through them at 1k creations/sec:

100,000,000 keys / 1,000 per sec = 100,000 seconds ≈ 27 hours

The pool holds more than a full day of capacity. The KGS can refill in large batches — say 50 million keys at a time — running every 12 hours or when the pool drops below a threshold.

KGS schedule:
→ Pool drops below 20M keys → wake up
→ Generate next 50M sequential codes
→ LPUSH into Redis pool
→ Sleep until pool drops again

The KGS never races with itself (single process) and never races with app servers (app servers only LPOP, KGS only LPUSH — opposite ends of the list if needed, or separate lists with atomic swap).

Interview framing

"KGS is a simple background worker — 50 lines of code, one cheap server, not in the request path. No UUID or Snowflake needed because there's no distributed coordination problem — it's a single process. It just counts sequentially through base62 space: 000000 to zzzzzz. Math: 62^6 = 56.8 billion — covers the 50B requirement with 13.6% headroom. 5 chars gives only 916M, not enough. 6 chars is exactly right."