Single-Flight Pattern • luminary.blog

A traditional way to refresh the access token is to make UI smart enough that when it gets a 401 it will refresh the access token calling the backend.

I decided to refresh the access token in the backend instead of bothering the frontend. Initially it didn’t work because many UI components call the backend during page load, which, as you guessed, created a race condition.

The Problem

When multiple requests arrive with expired access tokens, only one out of three requests succeeds.

1
  Timeline:
2
  T=0ms:  Request A arrives → Token expired → Start refresh with token "ABC123"
3
  T=1ms:  Request B arrives → Token expired → Start refresh with token "ABC123"
4
  T=2ms:  Request C arrives → Token expired → Start refresh with token "ABC123"
5

6
  T=10ms: Request A's refresh completes successfully
7
          - Gets new tokens: access="NEW_A", refresh="XYZ789"
8
          - Old refresh token "ABC123" is now INVALID (rotated)
9

10
  T=15ms: Request B's refresh attempts to use "ABC123"
11
          ❌ FAILS: "Invalid or expired refresh token"
12

13
  T=18ms: Request C's refresh attempts to use "ABC123"
14
          ❌ FAILS: "Invalid or expired refresh token"

The Solution

I decided to create a new class named RefreshLock that uses a Map to track in-progress refresh operations:

1
class RefreshLock {
2
private locks: Map<string, Promise<AuthTokenResponse>> = new Map();
3
//                 ^^^^^^  ^^^^^^^^^^^^^^^^^^^^^^^^^
4
//                  key     shared promise
5
}

The RefreshLock class prevents the race condition by implementing a “single-flight” pattern. All concurrent requests have the same old refresh token (“ABC123”) before any refresh completes. I use this token as the lock key.

1
Timeline:
2
T=0ms:  Request A calls refreshLock.refresh("ABC123", refreshFn)
3
        1. Check map: locks.get("ABC123") → undefined
4
        2. Start refresh operation (call refreshFn)
5
        3. Store promise: locks.set("ABC123", promise)
6
        4. Return promise to Request A
7

8
T=1ms:  Request B calls refreshLock.refresh("ABC123", refreshFn)
9
        1. Check map: locks.get("ABC123") → EXISTS!
10
        2. Log: "Refresh already in progress, waiting for result..."
11
        3. Return SAME promise (no new refresh started)
12
        4. Request B waits for Request A's refresh
13

14
T=2ms:  Request C calls refreshLock.refresh("ABC123", refreshFn)
15
        1. Check map: locks.get("ABC123") → EXISTS!
16
        2. Return SAME promise
17
        3. Request C waits for Request A's refresh
18

19
T=10ms: Request A's refresh completes
20
        - Gets new tokens: access="NEW", refresh="XYZ789"
21
        - Promise resolves with these tokens
22
        - Cleanup: locks.delete("ABC123")
23

24
        All three requests receive the SAME result simultaneously:
25
        - Request A: Got new tokens from its refresh
26
        - Request B: Got new tokens from Request A's refresh (waited)
27
        - Request C: Got new tokens from Request A's refresh (waited)

Result: All 3 requests succeed with only 1 actual refresh operation. Less load on the database, responsive web page, happy customers.

Full Code

1
class RefreshLock {
2
  private locks: Map<string, Promise<AuthTokenResponse>> = new Map();
3

4
  async refresh(
5
    refreshToken: RefreshToken,
6
    refreshFn: () => Promise<AuthTokenResponse>
7
  ): Promise<AuthTokenResponse> {
8
    const key = refreshToken;
9

10
    const existing = this.locks.get(key);
11
    if (existing) {
12
      return existing;
13
    }
14

15
    const refreshPromise = refreshFn()
16
      .finally(() => {
17
        this.locks.delete(key);
18
      });
19

20
    this.locks.set(key, refreshPromise);
21

22
    return refreshPromise;
23
  }
24
}

How This Works

Single-Flight Guarantee: Subsequent requests don’t start a new refresh; they wait for the in-progress one.

1
const existing = this.locks.get(key);
2
if (existing) return existing; // Return shared promise

Shared Result: All waiting requests get the same Promise object, so when it resolves, they all receive the same new tokens.
Token-Based Key: All concurrent requests with the same expired token are automatically grouped together.

1
const key = refreshToken;

Automatic Cleanup: Whether the refresh succeeds or fails, the lock is removed so future refreshes aren’t blocked.

1
.finally(() => this.locks.delete(key))

Complete Flow Diagram

1
1. Request arrives
2
    ↓
3
2. Middleware: Check access token
4
    → Expired/Missing
5
    ↓
6
3. Check for refresh token
7
    → Found
8
    ↓
9
4. RefreshLock: Check if refresh in progress
10
    → No → Start refresh
11
    → Yes → Wait for result
12
    ↓
13
5. AuthService.refreshToken():
14
    a. Hash refresh token
15
    b. Lookup session
16
    c. Validate session (not revoked/expired)
17
    d. Get user roles
18
    e. Rotate refresh token (old one invalid now)
19
    f. Generate new access token JWT
20
    g. Hash and save new refresh token
21
    h. Return new tokens + user data
22
    ↓
23
6. Middleware:
24
    a. Format cookies with new tokens
25
    b. Set Set-Cookie headers
26
    c. Validate new access token
27
    d. Set user context (userId, tenantId, etc.)
28
    e. Continue to route handler
29
    ↓
30
7. Route handler executes
31
    ↓
32
8. Response sent with new cookies
33
    ↓
34
9. Browser stores new cookies
35
    ↓
36
10. Next request uses fresh access token

Key Security Considerations

There are a few things to keep in mind relevant to access and refresh tokens.

HTTP-Only Cookies: Tokens never accessible via JavaScript (XSS protection)
Token Rotation: Old refresh token invalidated immediately after use
Hashed Storage: Refresh tokens hashed before storage (database breach protection)
Single-Flight: Prevents race conditions from concurrent refreshes
Session Validation: Checks revocation and expiration
Short-Lived Access: 15 minutes forces frequent refresh
SameSite=Strict: CSRF protection

Key Benefits of Single-Flight Pattern

While my use case was unique, the single-flight pattern fits many situations because it offers the benefits below:

Reduce Resource Usage: Prevents thundering herds by deduplicating expensive queries, API calls, or computations.
Lower Latency: Waiting requests piggyback on in-memory results instead of performing redundant work.
Prevent Race conditions: Ensures non-idempotent operations that must run once actually run once.

The Question

Always answer this question while you are building a system.

What happens if a hundred requests ask for the same resource at once?

← Time and Space Complexity Refresher

We Need More Datacenters →