Redis Caching in Go: A Beginner's Guide

Redis is a versatile in-memory data store commonly used for caching, session management, pub/sub, and much more. Its flexibility and wide range of use cases have made it a popular choice for personal and commercial projects.

This article will offer an accessible introduction to using Redis as a cache for Go programs, exploring its most prevalent application. You will learn how to connect to the Redis server in your Go applications and perform essential database operations, harnessing its capabilities to enhance performance and reduce database load.

Let's get started!

Prerequisites

To follow along with this article, ensure you have the latest version of Go installed on your machine. If you are missing Go, you can find the installation instructions here.

Step 1 — Installing and configuring Redis

Please follow the instructions here to install the latest stable release of Redis for your operating system (v7.x at the time of writing).

redis-server --version

Redis server v=7.0.12 sha=00000000:0 malloc=jemalloc-5.2.1 bits=64 build=d706905cc5f560c1

Once installed, confirm that Redis is running by executing the command below:

sudo systemctl status redis

● redis-server.service - Advanced key-value store
     Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-08-09 08:23:32 UTC; 5min ago
       Docs: http://redis.io/documentation,
             man:redis-server(1)
   Main PID: 3099 (redis-server)
     Status: "Ready to accept connections"
      Tasks: 5 (limit: 1025)
     Memory: 2.8M
        CPU: 259ms
     CGroup: /system.slice/redis-server.service
             └─3099 "/usr/bin/redis-server 127.0.0.1:6379" "" "" "" "" "" "" ""

You can now connect to your Redis server via the Redis CLI and test that it is working.

redis-cli

127.0.0.1:6379> PING

PONG

Receiving the PONG output above confirms that your Redis server is configured correctly.

One additional but optional step is to configure Redis' Access Control List (ACL) feature to require authentication for the default user (and any other users). This step is not required to complete this tutorial, but it is highly recommended for production environments. Please see the documentation for more details.

Step 2 — Setting up the demo repository

In this section, you will clone the repository containing all the examples presented in this tutorial and install the necessary dependencies.

Execute each of the following commands to get set up:

git clone https://github.com/betterstack-community/go-redis.git

cd go-redis/

go mod download

Locate the .env.example file in the project root and rename it to .env:

mv .env.example .env

Open the file in your text editor and change the connection details to match that of your local Redis instance. If you set up a password for your Redis user, include it here. Otherwise, you can leave it blank.

nano .env

ADDRESS=localhost:6379
PASSWORD=
DATABASE=0

Step 3 — Connecting to the Redis server

Similar to conventional databases, Redis utilizes databases to facilitate data segregation, enabling the creation of dedicated databases for individual projects. Unlike SQL databases, Redis does not provide a CREATE DATABASE equivalent. Instead, Redis includes a pre-defined set of 16 databases numbered from 0 to 15. The number of databases can be modified by adjusting the databases directive in the redis.conf configuration file.

In this section, you will learn how to connect to the Redis database specified using the DATABASE key in your .env file using Go. We will be using the official Redis package for Go applications throughout this tutorial.

Assuming you're already in your project directory, use the following command to install the Go Redis package:

go get github.com/redis/go-redis/v9

Once installed, open the cmd/connect/connect.go file in your text editor and observe the highlighted lines below:

package main

import (
    "context"
    "fmt"
    "github.com/redis/go-redis/v9"
    "github.com/woojiahao/go_redis/internal/utility"
    "log"
)

func main() {
    ctx := context.Background()
    // Ensure that you have Redis running on your system
    rdb := redis.NewClient(&redis.Options{
        Addr:     utility.Address(),
        Password: utility.Password(), // no password set
        DB:       utility.Database(), // use default DB
    })
    // Ensure that the connection is properly closed gracefully
    defer rdb.Close()

    // Perform basic diagnostic to check if the connection is working
    // Expected result > ping: PONG
    // If Redis is not running, error case is taken instead
    status, err := rdb.Ping(ctx).Result()
    if err != nil {
        log.Fatalln("Redis connection was refused")
    }
    fmt.Println(status)
}

The Redis connection details are loaded from the .env file through the utility package. The Redis server address, database name, and default password (if any) are loaded from the .env file, and these details are used to set up a new Redis client (rdb).

After creating a new Redis client, it's necessary to confirm that the configuration details are correct using the Ping() method. You will receive an error if the Redis server is not running at the provided address or if any other options are misconfigured. Otherwise, a PONG response will be observed, like when we used the redis-cli earlier.

go run cmd/connect/connect.go

PONG

If the Redis server is not running or the connection string is invalid, you will receive the following response instead, and the application will terminate:

2023/07/15 15:23:13 Redis connection was refused

Once you've successfully connected to your Redis server and confirmed its functionality, you can begin utilizing Redis as a cache with Go Redis. Just like many other data stores, you can perform the fundamental CRUD (Create, Read, Update, Delete) operations in your Redis cache, and that's what we'll be demonstrating throughout this article.

Step 4 — Adding data to the cache

You can add data to the cache via the SET command in Redis:

127.0.0.1:6379> SET FOO "BAR"

OK

Doing so assigns the value "BAR" to the key FOO.

In Go, you can add data to the connected database using the Set() method with the Redis client:

// Package imports

type Person struct {
    Name string `redis:"name"`
    Age  int    `redis:"age"`
}

func main() {
    // Redis connection...
    _, err := rdb.Set(ctx, "FOO", "BAR", 0).Result()
    if err != nil {
        fmt.Println("Failed to add FOO <> BAR key-value pair")
        return
    }
    rdb.Set(ctx, "INT", 5, 0)
    rdb.Set(ctx, "FLOAT", 5.5, 0)

    rdb.Set(ctx, "EXPIRING", 15, 30*time.Minute)

    rdb.HSet(ctx, "STRUCT", Person{"John Doe", 15})
}

The highlighted portion of the file describes how to store various data types in the Redis cache. Here is the signature of the Set() method:

func (c Client) Set(ctx context.Context, key string, value interface{}, expiration time.Duration) *StatusCmd

Note that while Set() allows you to specify any type in its third argument, the actual data stored in the Redis cache will be in the form of strings. However, the go-redis package offers a convenient wrapper around the supported data types in Redis, providing utility functions to parse the Redis string into the corresponding data types. We will delve into these utility functions in the next section.

Much like the Ping() method, Set() also returns a wrapper around the result of the query so that you can explicitly handle errors, as seen in the first Set() call.

It's worth noting the last parameter in the Set() method call, which specifies the expiration time for the key in the cache. In most examples above, a value of 0 is used, indicating that the key will not expire automatically and must be manually removed from the cache when desired. An example of a key-value pair that automatically expires in 30 minutes has also been included.

If you wish to store a struct in Redis, you can use the HSet() method which stores your struct as a Redis hash .

The above example can be executed using the command below:

go run cmd/set/set.go

Now that we've stored some values in the cache let's try accessing it in the next section.

Step 5 — Reading data from the cache

After storing a value in the Redis database, you may retrieve it using the GET command. You need to pass the Redis key as an argument to this command:

127.0.0.1:6379> GET FOO

"BAR"

In Go, the same can be achieved with the Get() method:

// Package imports

type Person struct {
    Name string `redis:"name"`
    Age  int    `redis:"age"`
}

func main() {
    // Redis connection...
    result, err := rdb.Get(ctx, "FOO").Result()
    if err != nil {
        fmt.Println("Key FOO not found in Redis cache")
    } else {
        fmt.Printf("FOO has value %s\n", result)
    }

    intValue, err := rdb.Get(ctx, "INT").Int()
    if err != nil {
        fmt.Println("Key INT not found in Redis cache")
    } else {
        fmt.Printf("INT has value %d\n", intValue)
    }

    var person Person
    err = rdb.HGetAll(ctx, "STRUCT").Scan(&person)
    if err != nil {
        fmt.Println("Key STRUCT not found in Redis cache")
    } else {
        fmt.Printf("STRUCT has value %+v\n", person)
    }

    result, err = rdb.Get(ctx, "BAZ").Result()
    if err != nil {
        fmt.Println("Key BAZ not found in Redis cache")
    } else {
        fmt.Printf("BAZ has value %s\n", result)
    }
}

You can run the example in the demo repository:

go run cmd/get/get.go

You should observe the following result:

FOO has value BAR
INT has value 5
STRUCT has value {Name:John Doe Age:15}
Key BAZ not found in Redis cache

As mentioned earlier, Go Redis employs wrappers to encapsulate the actual results obtained from Redis. When using the Get() function, a *StringCmd wrapper is returned. These wrappers prove particularly useful when working with data types other than strings, as they offer utility methods for parsing values into the appropriate data types.

For example, in the previous case, the key FOO was associated with the string value "BAR", which can be retrieved using the Result() method.

However, in the case of the key INT, which was associated with the integer value 5 in the previous example, Go Redis provides the utility method Int() to parse the Redis string into an int type, considering that the value was stored as a Redis string. A Float32() method also exists for parsing floats.

For a list of the utility methods for the StringCmd type, please refer to the documentation.

When retrieving hash values with HGetAll(), the resulting MapStringStringCmd type provides a Scan() method that receives a pointer reference to the intended struct and automatically populates the fields of that struct appropriately.

In situations where the provided key does not exist in the cache, an error is returned, allowing you to handle the situation accordingly.

In the next section, we will consider how to modify the values stored in the cache.

Step 6 — Updating data in the cache

Instead of having a dedicated command like UPDATE or EDIT, Redis utilizes the SET command for updating data. When SET is used, Redis performs a check to see if the specified key already exists in the cache. A new key-value pair in the cache if the key is not found. However, if the key already exists, Redis updates the existing key-value pair with the new value. This approach allows Redis to handle both the creation and updating of key-value pairs in a unified manner.

127.0.0.1:6379> SET FOO 5

OK

127.0.0.1:6379> GET FOO

"5"

In Go, the behavior follows, and the Set() method is used to update values in the database:

// Package imports
func main() {
    // Redis connection...
    // Set "FOO" to be associated with "BAR"
    rdb.Set(ctx, "FOO", "BAR", 0)
    result, err := rdb.Get(ctx, "FOO").Result()
    if err != nil {
        fmt.Println("FOO not found")
    } else {
        fmt.Printf("FOO has value %s\n", result)
    }

    // Update "FOO" to be associated with 5
    rdb.Set(ctx, "FOO", 5, 0)
    intResult, err := rdb.Get(ctx, "FOO").Int()
    if err != nil {
        fmt.Println("FOO not found")
    } else {
        fmt.Printf("FOO has value %d\n", intResult)
    }
}

Just like the Redis CLI example, the code snippet first associates the key FOO with the string value "BAR". Then, the value associated with the key FOO is updated to the integer value 5 immediately afterward.

If you execute the provided example, you can expect to observe the following results:

go run cmd/update/update.go

FOO has value BAR
FOO has value 5

As expected, the initial value held by FOO is "BAR" and after updating, the value becomes 5.

Step 7 — Deleting data from the cache

Rounding out CRUD operations in Redis is the delete operation, which can be achieved using the DEL command in the Redis CLI:

127.0.0.1:6379> DEL FOO

(integer) 1

To delete cached data from your Go program, you can similarly use the Del() method:

// Package imports
func main() {
    // Redis connection...
    // Set "FOO" to be associated with "BAR"
    rdb.Set(ctx, "FOO", "BAR", 0)
    result, err := rdb.Get(ctx, "FOO").Result()
    if err != nil {
        fmt.Println("FOO not found")
    } else {
        fmt.Printf("FOO has value %s\n", result)
    }

    // Deleting the key "FOO" and its associated value
    rdb.Del(ctx, "FOO")
    result, err = rdb.Get(ctx, "FOO").Result()
    if err != nil {
        fmt.Println("FOO not found")
    } else {
        fmt.Printf("FOO has value %s\n", result)
    }
}

Running the code in the demo repository would yield the following results:

go run cmd/delete/delete.go

FOO has value BAR
FOO not found

Once the value associated with the key FOO is set and retrieved, the program deletes the key-value pair using the Del() method. Subsequently, any attempt to access the value associated with the key FOO would result in an error, as the key-value pair has been removed from the cache.

Now that we've explored all the essential operations on a Redis cache, let us examine how Redis can be effectively utilized alongside a database and application server to deliver efficient and dependable caching functionality.

Step 8 — Putting it all together

In this section, you will improve the performance of a time-consuming database query by storing the retrieved data in a Redis cache and reusing it for subsequent requests. Although the database query is simulated, the conceptual thinking and implementation remain consistent.

To provide a concise overview, the demonstration aims to make an expensive data request to the (simulated) database three times. There will be two separate types of requests made. The initial type adheres to the system architecture devoid of a cache, leading to all three requests querying the database and awaiting its response. In contrast, the second type caches the database response after the first query, so subsequent requests will encounter significantly reduced processing durations.

To grasp the impact of caching on system efficiency, the execution time for each function is incorporated, showcasing the favorable ramifications of caching.

// Package imports
func main() {
    fmt.Println("Without caching...")
    start := time.Now()
    getDataExpensive()
    elapsed := time.Since(start)
    fmt.Printf("Without caching took %s\n\n", elapsed)

    fmt.Println("With caching...")
    start = time.Now()
    getDataCached()
    elapsed = time.Since(start)
    fmt.Printf("With caching took %s\n", elapsed)
}

func getDataExpensive() {
    for i := 0; i < 3; i++ {
        fmt.Println("\tBefore query")
        result := databaseQuery()
        fmt.Printf("\tAfter query with result %s\n", result)
    }
}

func getDataCached() {
    ctx := context.Background()
    rdb := redis.NewClient(&redis.Options{
        Addr:     utility.Address(),
        Password: utility.Password(), // no password set
        DB:       utility.Database(), // use default DB
    })
    // Ensure that the connection is properly closed gracefully
    defer rdb.Close()

    for i := 0; i < 3; i++ {
        fmt.Println("\tBefore query")
        val, err := rdb.Get(ctx, "query").Result()
        if err != nil {
            // Database query was not cached yet
            // Make database call and cache the value
            val = databaseQuery()
            rdb.Set(ctx, "query", val, 0)
        }
        fmt.Printf("\tAfter query with result %s\n", val)
    }
}

func databaseQuery() string {
    fmt.Println("\tDatabase queried")
    // Intentionally sleep for 5 seconds to simulate a long database query
    time.Sleep(5 * time.Second)
    return "bar"
}

When you run this program, you will observe the following results:

go run cmd/demo/demo.go

Without caching...
        Before query
        Database queried
        After query with result bar
        Before query
        Database queried
        After query with result bar
        Before query
        Database queried
        After query with result bar
Without caching took 15.003013s

With caching...
        Before query
        Database queried
        After query with result bar
        Before query
        After query with result bar
        Before query
        After query with result bar
With caching took 5.0340745s

As evident from the results, caching effectively reduces the number of database queries to one, aligning with our expectations.

In the scenario without caching, the database is immediately queried and required to process the request repeatedly, regardless of the processing time, for each request made by the server. With three data requests, the database is also queried thrice, each one taking five seconds to complete. Consequently, the total execution time for the three requests accumulates to approximately 15 seconds.

Conversely, when caching is implemented, we initially check the cache for the value associated with the query key. A database request is only made if the associated value does not exist in the cache, and its response is immediately stored in the cache. This results in subsequent data requests being served from the cache rather than the database. As a result, the execution time for all three requests is reduced to approximately five seconds, accounting for the five-second delay caused by the initial database query.

This example effectively emphasizes the significance of caching in real-time applications, as it significantly diminishes the execution time of processes.

For this example, we have decided to simplify the data request process by mocking the database call, but you can most certainly replace the body of databaseQuery() with an actual database call using the database/sql package and rename the cached query key to something more meaningful.

For instance, if the database query retrieves the computed profits stored in the database, the cached query key could be something like computed-profits instead of query so that it is more meaningful for your business logic. The remainder of the caching logic will remain the same, and you would have effectively migrated from a mock database call to a real database call.

Note that when storing data in the cache, an appropriate invalidation policy (such as time-based expiration) must also be in place so that your application isn't serving outdated responses to user requests.

Final thoughts

This article briefly introduced using Redis as a caching service and an in-memory database for Go applications. While it only scratches the surface of what Redis can achieve, we hope you are encouraged to explore this topic further.

Thanks for reading, and happy coding!

Further reading:

• Redis caching in Node.js
• JSON in Golang
• Top 8 Go Logging Libraries

Article by

Woo Jia Hao

Woo Jia Hao is a software developer from Singapore. He is an avid learner who loves solving and talking about complex and interesting problems. Lately, he has been working with Go and Elixir!

Got an article suggestion? Let us know

Next article

A Comprehensive Guide to Using JSON in Go

→

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.