Go's singleflight package and why it's awesome for concurrent requests

I’m still pretty new to Go, but thanks to my awesome crew at Pomerium, I’ve been learning fast by jumping into real-world code. One thing that clicked recently came from a pull request (PR) by my teammate Caleb. He introduced me to singleflight, a super handy tool that helps avoid duplicate work storms in concurrent code. singleflight incoming idp token session creation #5491 calebdoxsey posted on Feb 20, 2025 Summary If a burst of requests come in with the same IdP tokens we would attempt to validate and create sessions for those tokens many times. This PR update the code to wrap session creation in a singleflight so that the token has to be validated once. Related issues ENG-2025 Checklist [x] reference any related issues [ ] updated unit tests [x] add appropriate label (enhancement, bug, breaking, dependencies, ci) [x] ready for review View on GitHub Here’s the situation. Imagine a burst of requests all come in with the same IdP token. Without coordination, each request would validate and create a session separately — wasting resources and hammering downstream systems. In the pull request, Caleb fixed that by wrapping session creation in singleflight, so only one validation runs and all callers share the result. Problem solved — efficiently. At first, I thought this was memoization, but singleflight is built for concurrent deduplication, not long-term caching. What is the singleflight utility? singleflight prevents duplicate work when many goroutines make the same request at once. Think of it like short-lived memoization: only concurrent calls are deduplicated. Once done, the result is shared and forgotten — no caching needed. It’s super useful when: You want to avoid request storms (like validating the same token 100 times). You need to reduce load on databases or APIs. You want consistent results during concurrency spikes. How it works Deduplicates concurrent calls: Multiple goroutines calling Do() with the same key → only one runs. Shares results: All callers get the same result (including errors). Cleans up after: Once the work is done, the entry is removed. Isolated by key: Different keys don’t block each other. Real-world example In Caleb's PR, singleflight makes sure only one session gets created when a lot of requests hit at once with the same token. res, err, _ := c.singleflight.Do(sessionID, func() (any, error) { // Try to get existing session s, err := c.getSession(ctx, sessionID) if err == nil { return s, nil // Found existing session } else if !storage.IsNotFound(err) { return nil, err } // No existing session → create new one // Verify token and make API calls... err = c.putSessionAndUser(ctx, s, u) if err != nil { return nil, fmt.Errorf("error saving session and user: %w", err) } return s, nil }) No matter how many concurrent requests come in, only one runs this logic. The rest just get the same result. Why use singleflight Prevent verification storms: One verification instead of dozens. Reduce database/API load: No redundant work. Return consistent results: All callers see the same outcome. singleflight vs memoization Memoization caches results across time. singleflight only deduplicates concurrent calls. Memoization singleflight Long-term cache Only during concurrent calls Needs eviction/TTL No expiry needed Speeds up future calls Avoids duplicate work right now Final thoughts singleflight is one of those elegant Go utilities that quietly solves a tough problem. When you’re dealing with concurrency and don’t want the hassle of caching or locking, it’s the perfect tool. I’m still early in my Go journey, so running into patterns like this feels exciting. If you know other handy concurrency tricks (especially for someone coming from JS/C#/React land), drop them in the comments — always down to learn

May 4, 2025 - 06:02

I’m still pretty new to Go, but thanks to my awesome crew at Pomerium, I’ve been learning fast by jumping into real-world code.

One thing that clicked recently came from a pull request (PR) by my teammate Caleb. He introduced me to singleflight, a super handy tool that helps avoid duplicate work storms in concurrent code.

singleflight incoming idp token session creation #5491

calebdoxsey posted on Feb 20, 2025

Summary

If a burst of requests come in with the same IdP tokens we would attempt to validate and create sessions for those tokens many times. This PR update the code to wrap session creation in a singleflight so that the token has to be validated once.

Related issues

ENG-2025

Checklist

[x] reference any related issues
[ ] updated unit tests
[x] add appropriate label (enhancement, bug, breaking, dependencies, ci)
[x] ready for review

View on GitHub

Here’s the situation. Imagine a burst of requests all come in with the same IdP token. Without coordination, each request would validate and create a session separately — wasting resources and hammering downstream systems.

In the pull request, Caleb fixed that by wrapping session creation in singleflight, so only one validation runs and all callers share the result. Problem solved — efficiently.

At first, I thought this was memoization, but singleflight is built for concurrent deduplication, not long-term caching.

What is the singleflight utility?

singleflight prevents duplicate work when many goroutines make the same request at once. Think of it like short-lived memoization: only concurrent calls are deduplicated. Once done, the result is shared and forgotten — no caching needed.

It’s super useful when:

You want to avoid request storms (like validating the same token 100 times).
You need to reduce load on databases or APIs.
You want consistent results during concurrency spikes.

How it works

Deduplicates concurrent calls: Multiple goroutines calling Do() with the same key → only one runs.
Shares results: All callers get the same result (including errors).
Cleans up after: Once the work is done, the entry is removed.
Isolated by key: Different keys don’t block each other.

Real-world example

In Caleb's PR, singleflight makes sure only one session gets created when a lot of requests hit at once with the same token.

res, err, _ := c.singleflight.Do(sessionID, func() (any, error) {
    // Try to get existing session
    s, err := c.getSession(ctx, sessionID)
    if err == nil {
        return s, nil // Found existing session
    } else if !storage.IsNotFound(err) {
        return nil, err
    }

    // No existing session → create new one
    // Verify token and make API calls...

    err = c.putSessionAndUser(ctx, s, u)
    if err != nil {
        return nil, fmt.Errorf("error saving session and user: %w", err)
    }

    return s, nil
})

No matter how many concurrent requests come in, only one runs this logic. The rest just get the same result.

Why use `singleflight`

Prevent verification storms: One verification instead of dozens.
Reduce database/API load: No redundant work.
Return consistent results: All callers see the same outcome.

singleflight vs memoization

Memoization caches results across time. singleflight only deduplicates concurrent calls.

Memoization	`singleflight`
Long-term cache	Only during concurrent calls
Needs eviction/TTL	No expiry needed
Speeds up future calls	Avoids duplicate work right now

Final thoughts

singleflight is one of those elegant Go utilities that quietly solves a tough problem. When you’re dealing with concurrency and don’t want the hassle of caching or locking, it’s the perfect tool.

I’m still early in my Go journey, so running into patterns like this feels exciting. If you know other handy concurrency tricks (especially for someone coming from JS/C#/React land), drop them in the comments — always down to learn