Singleflight: Reduce Costs and Improve Performance in Go Applications

As software engineers, we should always consider how to improve the performance of our applications while also reducing costs. Achieving both of these goals simultaneously is not always easy; often, we are forced to choose between being cheaper or faster. In this article, I want to show you how we can achieve both goals by using singleflight. What is Singleflight? Singleflight is a standard Go library (golang.org/x/sync/singleflight) that prevents redundant calls to the same function. It achieves this by grouping concurrent requests to the same resource and returning the first result to all subsequent requests. Use cases examples Queries to external APIs charged per request. Read operations in databases. Prevention of rate limiting in APIs with request limits. Example: Product Price Search System An e-commerce platform where users search for product prices. The price is obtained from an external API that charges per request. Multiple users can search for the price of the same product simultaneously. Code without singleflight package main import ( "encoding/json" "log" "net/http" "time" ) var cost float64 // Simulates a call to an external API that returns the price of a product func fetchProductPrice(productID string) (float64, error) { log.Printf("[COST: $0.01] Calling external API for product: %s\n", productID) time.Sleep(2 * time.Second) // Simulates latency cost += 0.01 return 99.99, nil // Fictitious price } // Handler to fetch the price of a product func getProductPriceHandler(w http.ResponseWriter, r http.Request) { // Extract the product ID from the URL productID := r.URL.Query().Get("id") // Fetches the product price from the external API price, err := fetchProductPrice(productID) if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } // Create a response map response := map[string]interface{}{ "product_id": productID, "price": price, } // Convert the response to JSON w.Header().Set("Content-Type", "application/json") if err := json.NewEncoder(w).Encode(response); err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } } func getCost(w http.ResponseWriter, r http.Request) { // Create a response map response := map[string]interface{}{ "total_cost": fmt.Sprintf("%.2f", cost), } // Convert the response to JSON w.Header().Set("Content-Type", "application/json") if err := json.NewEncoder(w).Encode(response); err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } } func clearCosts(w http.ResponseWriter, r http.Request) { cost = 0 } func main() { // Create a new ServeMux mux := http.NewServeMux() // Register the handler for the /products/:id/price route mux.HandleFunc("/products/{id}/price", getProductPriceHandler) mux.HandleFunc("/costs", getCost) mux.HandleFunc("/clear-costs", clearCosts) // Start the server log.Println("API running without singleflight on port :8080...") if err := http.ListenAndServe(":8080", mux); err != nil { log.Fatalf("Could not start server: %s\n", err.Error()) } } Problem: If 100 users search for the price of the same product at the same time, the system will make 100 calls to the external API, generating a cost of $1.00 and increasing latency. Code with singleflight package main import ( "encoding/json" "log" "net/http" "time" "golang.org/x/sync/singleflight" ) var ( group singleflight.Group cost float64 ) // Simulates a call to an external API that returns the price of a product func fetchProductPrice(productID string) (float64, error) { log.Printf("[COST: $0.01] Calling external API for product: %s\n", productID) time.Sleep(2 time.Second) // Simulates latency cost += 0.01 return 99.99, nil // Fictitious price } // Handler to fetch the price of a product func getProductPriceHandler(w http.ResponseWriter, r *http.Request) { // Extract the product ID from the URL productID := r.URL.Query().Get("id") // Uses singleflight to avoid redundant calls result, err, _ := group.Do(productID, func() (interface{}, error) { return fetchProductPrice(productID) }) if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } // Convert the result to float64 price := result.(float64) // Create a response map response := map[string]interface{}{ "product_id": productID, "price": price, } // Convert the response to JSON w.Header().Set("Content-Type", "application/json") if err := json.NewEncoder(w).Encode(response); err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return

Mar 4, 2025 - 22:45

Singleflight: Reduce Costs and Improve Performance in Go Applications

As software engineers, we should always consider how to improve the performance of our applications while also reducing costs. Achieving both of these goals simultaneously is not always easy; often, we are forced to choose between being cheaper or faster.

In this article, I want to show you how we can achieve both goals by using singleflight.

What is Singleflight?

Singleflight is a standard Go library (golang.org/x/sync/singleflight) that prevents redundant calls to the same function. It achieves this by grouping concurrent requests to the same resource and returning the first result to all subsequent requests.

Use cases examples

Queries to external APIs charged per request.
Read operations in databases.
Prevention of rate limiting in APIs with request limits.

Example: Product Price Search System

An e-commerce platform where users search for product prices.
The price is obtained from an external API that charges per request.
Multiple users can search for the price of the same product simultaneously.

Code without singleflight

package main

import (
    "encoding/json"
    "log"
    "net/http"
    "time"
)

var cost float64

// Simulates a call to an external API that returns the price of a product
func fetchProductPrice(productID string) (float64, error) {
    log.Printf("[COST: $0.01] Calling external API for product: %s\n", productID)
    time.Sleep(2 * time.Second) // Simulates latency
    cost += 0.01
    return 99.99, nil // Fictitious price
}

// Handler to fetch the price of a product
func getProductPriceHandler(w http.ResponseWriter, r *http.Request) {
    // Extract the product ID from the URL
    productID := r.URL.Query().Get("id")

    // Fetches the product price from the external API
    price, err := fetchProductPrice(productID)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    // Create a response map
    response := map[string]interface{}{
        "product_id": productID,
        "price":      price,
    }

    // Convert the response to JSON
    w.Header().Set("Content-Type", "application/json")
    if err := json.NewEncoder(w).Encode(response); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
}

func getCost(w http.ResponseWriter, r *http.Request) {
    // Create a response map
    response := map[string]interface{}{
        "total_cost": fmt.Sprintf("%.2f", cost),
    }

    // Convert the response to JSON
    w.Header().Set("Content-Type", "application/json")
    if err := json.NewEncoder(w).Encode(response); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
}

func clearCosts(w http.ResponseWriter, r *http.Request) {
    cost = 0
}

func main() {
    // Create a new ServeMux
    mux := http.NewServeMux()

    // Register the handler for the /products/:id/price route
    mux.HandleFunc("/products/{id}/price", getProductPriceHandler)
    mux.HandleFunc("/costs", getCost)
    mux.HandleFunc("/clear-costs", clearCosts)

    // Start the server
    log.Println("API running without singleflight on port :8080...")
    if err := http.ListenAndServe(":8080", mux); err != nil {
        log.Fatalf("Could not start server: %s\n", err.Error())
    }
}

Problem: If 100 users search for the price of the same product at the same time, the system will make 100 calls to the external API, generating a cost of $1.00 and increasing latency.

Code with singleflight

package main

import (
    "encoding/json"
    "log"
    "net/http"
    "time"

    "golang.org/x/sync/singleflight"
)

var (
    group singleflight.Group
    cost  float64
)

// Simulates a call to an external API that returns the price of a product
func fetchProductPrice(productID string) (float64, error) {
    log.Printf("[COST: $0.01] Calling external API for product: %s\n", productID)
    time.Sleep(2 * time.Second) // Simulates latency
    cost += 0.01
    return 99.99, nil // Fictitious price
}

// Handler to fetch the price of a product
func getProductPriceHandler(w http.ResponseWriter, r *http.Request) {
    // Extract the product ID from the URL
    productID := r.URL.Query().Get("id")

    // Uses singleflight to avoid redundant calls
    result, err, _ := group.Do(productID, func() (interface{}, error) {
        return fetchProductPrice(productID)
    })
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    // Convert the result to float64
    price := result.(float64)

    // Create a response map
    response := map[string]interface{}{
        "product_id": productID,
        "price":      price,
    }

    // Convert the response to JSON
    w.Header().Set("Content-Type", "application/json")
    if err := json.NewEncoder(w).Encode(response); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
}

func getCost(w http.ResponseWriter, r *http.Request) {
    // Create a response map
    response := map[string]interface{}{
        "total_cost": fmt.Sprintf("%.2f", cost),
    }
    // Convert the response to JSON
    w.Header().Set("Content-Type", "application/json")
    if err := json.NewEncoder(w).Encode(response); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
}

func clearCosts(w http.ResponseWriter, r *http.Request) {
    cost = 0
}

func main() {
    // Create a new ServeMux
    mux := http.NewServeMux()

    // Register the handler for the /products/price route
    mux.HandleFunc("/products/{id}/price", getProductPriceHandler)
    mux.HandleFunc("/costs", getCost)
    mux.HandleFunc("/clear-costs", clearCosts)

    // Start the server
    log.Println("API running with singleflight on port :8081...")
    if err := http.ListenAndServe(":8081", mux); err != nil {
        log.Fatalf("Could not start server: %s\n", err.Error())
    }
}

Improvement: Only one call is made to the external API, regardless of how many users are searching for the price of the same product. This reduces the cost to $0.01 and decreases latency.

Tests using Vegeta

Without Singleflight

vegeta attack -duration=2s -rate=200 -workers=200 -targets=targets_without_singleflight.txt | vegeta report

Requests      [total, rate, throughput]         400, 200.47, 100.08
Duration      [total, attack, wait]             3.997s, 1.995s, 2.001s
Latencies     [min, mean, 50, 90, 95, 99, max]  2.001s, 2.002s, 2.002s, 2.002s, 2.002s, 2.002s, 2.003s
Bytes In      [total, mean]                     12800, 32.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:400  
Error Set:

Latency:
- All requests took ~2 seconds to complete.
- This happens because each request made an independent call to the external API, which has a simulated latency of 2 seconds.
Throughput:
- The throughput was 100.08 requests per second.
- This indicates that the system was able to process about 100 requests per second, but with high latency.
Cost:
- Since each request made a call to the external API, the cost would be 4.00(400 requests × 0.01 per call).
- GET http://localhost:8080/costs
```
{"total_cost":"4.00"}
```

With singleflight

vegeta attack -duration=2s -rate=200 -workers=200 -targets=targets_with_singleflight.txt | vegeta report  
Requests      [total, rate, throughput]         400, 200.46, 198.32
Duration      [total, attack, wait]             2.017s, 1.995s, 21.452ms
Latencies     [min, mean, 50, 90, 95, 99, max]  15.337ms, 1.014s, 1.019s, 1.811s, 1.911s, 1.991s, 2.009s
Bytes In      [total, mean]                     12800, 32.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:400  
Error Set:

Latency:
- The minimum latency was 15.337ms, and the average was 1.014s.
- This happens because only one call was made to the external API (taking ~2 seconds), and the other requests received the result in less than 22ms.
Throughput:
- The throughput increased to 198.32 requests per second.
- This indicates that the system was able to process almost double the number of requests per second, thanks to the reduced latency for most requests.
Cost:
- Since only one call was made to the external API, the cost would be 0.01 (1 request × 0.01 per call).
- GET http://localhost:8081/costs
```
{"total_cost":"0.01"}
```

Direct Comparison

Metric	Without `singleflight`	With `singleflight`	Gain
Average Latency	2.002s	1.014s	~49%
Throughput	100.08 req/s	198.32 req/s	~98%
Cost	$4.00	$0.01	99.75%

Latency Reduction:
- Singleflight reduced the average latency from 2.002s to 1.014s, a gain of ~49%.
- This happens because most requests were resolved in less than 22ms, while only the first request took ~2 seconds.
Throughput Increase:
- The throughput increased from 100.08 req/s to 198.32 req/s, a gain of ~98%.
- This indicates that the system was able to process almost double the number of requests per second, thanks to the reduced latency.
Cost Reduction:
- The cost dropped from 4.00 to 0.01, a savings of 99.75%.
- This happens because only one call was made to the external API, instead of 400.

When to Use (and When Not to Use)

Use:
- For read operations (queries to APIs or databases).
- When the resource is idempotent (does not change the system state).
Do not use:
- For write operations (creation, update, deletion).
- When the result may vary between calls.

Conclusion

Singleflight is a powerful tool to optimize Go applications, reducing costs and improving performance in high-concurrency scenarios. Try implementing it in your projects and see the difference!

Try it out: Add singleflight to your next project and share the results!!