When building robust applications in Go, proper timeout handling is essential. Timeouts protect your systems from hanging indefinitely, conserve resources, and help maintain responsiveness even under challenging conditions. This guide walks through timeout concepts, implementation patterns, and best practices to make your Go applications more resilient.
Go's concurrency model makes it excellent for building networked services, but without proper timeout handling, these services can degrade or fail completely when external dependencies slow down. Whether you're making HTTP requests, querying databases, or processing background tasks, timeouts are your safety net.
Without proper timeouts, a slow database query or unresponsive API can cascade into system-wide failures. This article will show you how to implement timeouts effectively using Go's built-in concurrency primitives.
Understanding timeouts in Go
Go's approach to timeouts builds on its core concurrency primitives: goroutines,
channels, and the select statement. These elements work together to create
clean, readable timeout patterns.
A timeout in Go typically follows this conceptual pattern:
- Start an operation in a separate goroutine.
- Create a timer or deadline for completion.
- Use
selectto wait for either completion or timeout. - Clean up resources regardless of outcome.
The beauty of Go's approach is that it makes timeout logic explicit in your code, rather than hiding it in configuration parameters.
The context package
At the heart of Go's timeout handling is the context package. Introduced in Go
1.7, it provides a standard way to carry deadlines, cancellation signals, and
request-scoped values across API boundaries.
Let's look at how to create and use contexts with timeouts:
package main
import (
"context"
"fmt"
"time"
)
func main() {
// Create a context that will timeout after 2 seconds
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel() // Always call cancel to release resources
// Simulate work that takes longer than our timeout
go func() {
time.Sleep(3 * time.Second)
fmt.Println("Work finished - but too late!")
}()
// Wait for the context to timeout
select {
case <-ctx.Done():
fmt.Println("Operation timed out:", ctx.Err())
}
// Give the goroutine time to print its message
time.Sleep(2 * time.Second)
}
When you run this program, you'll see:
Operation timed out: context deadline exceeded
Work finished - but too late!
The context.WithTimeout() function creates a context that will automatically
cancel after the specified duration. The cancel() function should always be
deferred to prevent resource leaks, even if the operation completes before the
timeout.
Another similar function is context.WithDeadline(), which allows you to
specify an absolute time rather than a duration:
deadline := time.Now().Add(5 * time.Second)
ctx, cancel := context.WithDeadline(context.Background(), deadline)
defer cancel()
HTTP client timeouts
One of the most common places to implement timeouts is in HTTP clients. Go's
standard http package provides several timeout options:
[http_client.go]
package main
import (
"context"
"fmt"
"io"
"net/http"
"time"
)
func main() {
// Create a client with various timeout settings
client := &http.Client{
Timeout: 10 * time.Second, // Overall timeout for the request
Transport: &http.Transport{
DialContext: (&net.Dialer{
Timeout: 3 * time.Second, // Connection timeout
KeepAlive: 30 * time.Second,
}).DialContext,
TLSHandshakeTimeout: 3 * time.Second, // TLS handshake timeout
ResponseHeaderTimeout: 5 * time.Second, // Wait for response headers
ExpectContinueTimeout: 1 * time.Second, // Wait for 100 Continue
IdleConnTimeout: 90 * time.Second, // Idle connection timeout
},
}
// Create a context for this specific request
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
// Create a request with the context
req, err := http.NewRequestWithContext(ctx, "GET", "https://example.com", nil)
if err != nil {
fmt.Println("Error creating request:", err)
return
}
// Send the request
resp, err := client.Do(req)
if err != nil {
fmt.Println("Request failed:", err)
return
}
defer resp.Body.Close()
// Read and process the response
body, err := io.ReadAll(resp.Body)
if err != nil {
fmt.Println("Error reading response:", err)
return
}
fmt.Printf("Response status: %s, Body length: %d bytes\n",
resp.Status, len(body))
}
This example demonstrates several different timeout layers:
client.Timeout- The overall timeout for the entire request/response cycle.DialContexttimeout - How long to wait for the TCP connection to establish.TLSHandshakeTimeout- Maximum time for TLS handshake completion.ResponseHeaderTimeout- How long to wait for the server's response headers.ExpectContinueTimeout- Time to wait for a 100 Continue response.- Request-specific context timeout - Can be shorter than the client timeout.
The layered approach gives you fine-grained control over timeouts at different stages of the HTTP request lifecycle.
Database query timeouts
Database operations are another critical area for timeout handling. Most database drivers for Go support context-based timeouts:
[database.go]
package main
import (
"context"
"database/sql"
"fmt"
"time"
_ "github.com/lib/pq" // PostgreSQL driver
)
func queryWithTimeout(db *sql.DB) error {
// Create a context with a 3-second timeout
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
// Execute query with the timeout context
rows, err := db.QueryContext(ctx,
"SELECT * FROM large_table WHERE complex_condition = true")
if err != nil {
return fmt.Errorf("query failed: %w", err)
}
defer rows.Close()
// Process results
count := 0
for rows.Next() {
// Handle each row...
count++
// Check if context has been canceled during processing
if ctx.Err() != nil {
return fmt.Errorf("processing interrupted: %w", ctx.Err())
}
}
if err := rows.Err(); err != nil {
return fmt.Errorf("error during row iteration: %w", err)
}
fmt.Printf("Processed %d rows\n", count)
return nil
}
func main() {
// Connect to database
db, err := sql.Open("postgres", "postgres://user:password@localhost/dbname?sslmode=disable")
if err != nil {
fmt.Println("Failed to connect:", err)
return
}
defer db.Close()
// Set connection pool timeouts
db.SetConnMaxLifetime(5 * time.Minute)
db.SetConnMaxIdleTime(5 * time.Minute)
// Execute query with timeout
err = queryWithTimeout(db)
if err != nil {
fmt.Println(err)
}
}
This example demonstrates how to apply a timeout to a database query using
QueryContext. The same pattern works for all database operations in Go's
standard database/sql package, including ExecContext, PrepareContext, and
Conn.QueryContext.
Implementing timeouts in web servers
Web servers need timeouts too. Without them, slow clients or resource-intensive requests can tie up your server indefinitely. Go's HTTP server provides several timeout settings:
[http_server.go]
package main
import (
"context"
"fmt"
"net/http"
"time"
)
func slowHandler(w http.ResponseWriter, r *http.Request) {
// Get the request context
ctx := r.Context()
// Simulate work with a cancelable operation
select {
case <-time.After(5 * time.Second):
fmt.Fprintln(w, "Work completed successfully!")
case <-ctx.Done():
// The client canceled or the server timed out
fmt.Println("Handler canceled:", ctx.Err())
return
}
}
func main() {
// Register our handler
http.HandleFunc("/slow", slowHandler)
// Create a server with various timeout settings
server := &http.Server{
Addr: ":8080",
// Maximum duration for reading the entire request
ReadTimeout: 5 * time.Second,
// Maximum duration for writing the response
WriteTimeout: 10 * time.Second,
// Maximum duration for reading the request headers
ReadHeaderTimeout: 2 * time.Second,
// Maximum amount of time to wait for the next request when keep-alives are enabled
IdleTimeout: 120 * time.Second,
// Handler to use for incoming requests
Handler: nil, // Use the DefaultServeMux
}
// Start the server with a graceful shutdown capability
go func() {
fmt.Println("Server starting on :8080")
if err := server.ListenAndServe(); err != http.ErrServerClosed {
fmt.Printf("HTTP server error: %v\n", err)
}
}()
// Wait for a while, then shut down
time.Sleep(60 * time.Second)
// Create a shutdown context
shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// Attempt graceful shutdown
if err := server.Shutdown(shutdownCtx); err != nil {
fmt.Printf("Server shutdown error: %v\n", err)
}
fmt.Println("Server stopped gracefully")
}
The HTTP server example demonstrates several important timeout settings:
ReadTimeout- Maximum time to read the entire requestWriteTimeout- Maximum time to write the responseReadHeaderTimeout- Maximum time to read request headersIdleTimeout- Maximum time to wait for the next request on keep-alive connections
The handler also shows how to use the request's context to detect when a client disconnects or a timeout occurs.
Using go-resty/resty for HTTP timeouts and retries
Instead of implementing your own timeout and retry logic for HTTP clients, the go-resty/resty package provides an elegant, fluent API with built-in support for timeouts, retries, and other advanced features. Let's explore how to use resty to handle timeouts effectively:
package main
import (
"fmt"
"time"
"github.com/go-resty/resty/v2"
)
func main() {
// Create a new resty client
client := resty.New()
// Configure timeouts
client.SetTimeout(10 * time.Second) // Total request timeout
client.SetDialTimeout(3 * time.Second) // TCP connection timeout
client.SetTLSHandshakeTimeout(3 * time.Second) // TLS handshake timeout
// Configure retry behavior
client.
// Retry up to 3 times
SetRetryCount(3).
// Retry on specific HTTP status codes
SetRetryWaitTime(100 * time.Millisecond). // Base wait time
SetRetryMaxWaitTime(2 * time.Second). // Maximum wait time
// Implement exponential backoff with jitter automatically
SetRetryAfter(func(client *resty.Client, resp *resty.Response) (time.Duration, error) {
// Log retry attempts
fmt.Printf("Retry attempt %d for %s\n",
client.RetryCount+1, resp.Request.URL)
return 0, nil // Use default backoff
})
// Make a request with all the configured settings
resp, err := client.R().
// Set request-specific timeout (overrides client timeout)
SetTimeout(5 * time.Second).
// Enable automatic retry for specific status codes
AddRetryCondition(func(r *resty.Response, err error) bool {
return r.StatusCode() >= 500 || err != nil
}).
Get("https://api.example.com/data")
if err != nil {
fmt.Printf("Request failed after retries: %v\n", err)
return
}
fmt.Printf("Response Status: %d, Body: %s\n",
resp.StatusCode(), resp.String())
}
Understanding resty's timeout features
The resty package provides several timeout configuration options:
SetTimeout: Sets the overall timeout for the entire request lifecycle, including connection, request sending, and response reading. This is similar tohttp.Client.Timeout.SetDialTimeout: Controls how long to wait for establishing a TCP connection, equivalent to theDialContext.Timeoutin the standard library.SetTLSHandshakeTimeout: Sets the maximum time allowed for TLS handshake completion.
These can be set at the client level (affecting all requests) or on individual requests to override the client defaults:
// Client-level timeout (default for all requests)
client := resty.New().SetTimeout(10 * time.Second)
// Request-specific timeout (overrides client timeout)
resp, err := client.R().
SetTimeout(5 * time.Second).
Get("https://api.example.com/endpoint")
The ability to set timeouts at different levels gives you fine-grained control over timing constraints based on the specific needs of each API endpoint you're calling.
Resty's automatic retry system
One of resty's most powerful features is its built-in retry system, which eliminates the need to implement complex retry logic manually:
[resty_retry.go]
client := resty.New()
// Configure retry behavior
client.
// Retry up to 5 times
SetRetryCount(5).
// Start with 100ms wait time
SetRetryWaitTime(100 * time.Millisecond).
// Cap wait time at 2 seconds
SetRetryMaxWaitTime(2 * time.Second).
// Retry on these HTTP status codes
SetRetryHook(func(resp *resty.Response, err error) {
fmt.Printf("Retrying request to %s (attempt %d) due to: %v\n",
resp.Request.URL, resp.Request.Attempt, err)
})
The retry system automatically implements exponential backoff with jitter, which helps prevent overwhelming services during recovery periods. This approach gradually increases wait times between retry attempts, with some randomness added to prevent all clients from retrying simultaneously.
Customizing retry conditions
By default, resty will retry on connection errors, but you can customize
exactly when retries should occur:
[resty_retry_conditions.go]
// Add custom retry conditions
client.AddRetryCondition(func(r *resty.Response, err error) bool {
// Retry on any 5xx server error
return r.StatusCode() >= 500 || err != nil
})
// Or add more specific conditions
client.AddRetryCondition(func(r *resty.Response, err error) bool {
// Retry on rate limiting (429) or service unavailable (503)
return r.StatusCode() == 429 || r.StatusCode() == 503
})
This flexibility allows you to implement sophisticated retry strategies based on your application's specific needs and the APIs you're interacting with.
Context support in resty
Resty also supports Go's context package, enabling you to control timeouts and cancellation from outside the request:
[resty_context.go]
package main
import (
"context"
"fmt"
"time"
"github.com/go-resty/resty/v2"
)
func main() {
client := resty.New()
// Create a context with timeout
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
// Use the context with the request
resp, err := client.R().
SetContext(ctx).
Get("https://api.example.com/data")
if err != nil {
if ctx.Err() != nil {
fmt.Println("Request canceled by context:", ctx.Err())
} else {
fmt.Println("Request failed:", err)
}
return
}
fmt.Printf("Response received: %d\n", resp.StatusCode())
}
This integration with context allows for more advanced patterns like cascading timeouts, where a parent operation allocates portions of its total timeout budget to different sub-operations.
Common mistakes and best practices
When implementing timeouts in Go, watch out for these common pitfalls:
1. Forgetting to call cancel functions
Always call the cancel function returned by context.WithTimeout or
context.WithDeadline, typically using defer:
// Good practice
ctx, cancel := context.WithTimeout(parentCtx, 5*time.Second)
defer cancel() // Will be called when the function returns
2. Not propagating contexts
Pass contexts through your call stack to ensure timeout signals propagate:
// Bad practice
func processSomething() {
// Creating a new context here loses the parent timeout
ctx := context.Background()
// ...
}
// Good practice
func processSomething(ctx context.Context) {
// Using the provided context preserves timeout/cancellation
// ...
}
3. Blocking operations
Avoid blocking operations that can't be interrupted by context cancellation:
// Bad practice - can't be interrupted
func badFunction(ctx context.Context) {
time.Sleep(10 * time.Second) // Ignores context cancellation
}
// Good practice - respects cancellation
func goodFunction(ctx context.Context) error {
select {
case <-time.After(10 * time.Second):
return nil
case <-ctx.Done():
return ctx.Err()
}
}
4. Improper error handling
Always check and handle timeout errors:
// Check for specific timeout errors
if err != nil {
if errors.Is(err, context.DeadlineExceeded) {
// Handle timeout specifically
log.Println("Operation timed out")
} else {
// Handle other errors
log.Printf("Operation failed: %v", err)
}
}
Testing timeout code
Testing timeout behavior is essential. Here's how to test timeout scenarios effectively:
package timeout
import (
"context"
"errors"
"testing"
"time"
)
func TestOperationWithTimeout(t *testing.T) {
// Test that operation completes before timeout
t.Run("SuccessBeforeTimeout", func(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
// Fast operation should succeed
result, err := fastOperation(ctx)
if err != nil {
t.Fatalf("Expected success, got error: %v", err)
}
if result != "success" {
t.Fatalf("Expected 'success', got %q", result)
}
})
// Test that operation times out appropriately
t.Run("TimeoutTriggered", func(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel()
// Slow operation should timeout
_, err := slowOperation(ctx)
if !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("Expected deadline exceeded error, got: %v", err)
}
})
// Test that cancellation is respected
t.Run("CancellationRespected", func(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
// Cancel after 10ms
go func() {
time.Sleep(10 * time.Millisecond)
cancel()
}()
// Operation should be canceled
_, err := slowOperation(ctx)
if !errors.Is(err, context.Canceled) {
t.Fatalf("Expected canceled error, got: %v", err)
}
})
}
// Example operation implementations for testing
func fastOperation(ctx context.Context) (string, error) {
select {
case <-time.After(10 * time.Millisecond):
return "success", nil
case <-ctx.Done():
return "", ctx.Err()
}
}
func slowOperation(ctx context.Context) (string, error) {
select {
case <-time.After(1 * time.Second):
return "success", nil
case <-ctx.Done():
return "", ctx.Err()
}
}
Final thoughts
Proper timeout handling is essential for building robust Go applications. By leveraging Go's context package and concurrency primitives, you can implement clean, effective timeout patterns throughout your codebase.
For HTTP clients specifically, the resty package offers a powerful, high-level alternative to manual timeout implementation, with built-in support for retries and exponential backoff.
With these practices, your Go applications will be more resilient to network issues, resource constraints, and other real-world challenges.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.