Sorted Sets in Redis from CLI, Python and Golang
In this post, we will see a demo of sorted sets in redis. I just learned about them and I think they are really cool! This post shows how we can play with sorted sets first via the redis-cli, then from Python and Golang.
┌────────────┐
.───────────────. │ │ .─────────────.
( Redis CLI ) ───▶ │ Redis │ ◀───── ( Golang )
`───────────────' │ │ `─────────────'
└────────────┘
▲
│
│
.─────────────.
( Python )
`─────────────'
We will first need a local redis server running. We will see how we do so on Fedora Linux next. If you are running another operating system, please see the download page.
Installation and server setup on Fedora Linux
We can install redis server using dnf, like so:
$ sudo dnf install redis
..
$ redis-server --version
Redis server v=4.0.6 sha=00000000:0 malloc=jemalloc-4.5.0 bits=64 build=427484a80e1b4515
Let’s start the server:
$ sudo systemctl start redis
$ sudo systemctl status redis
● redis.service - Redis persistent key-value database
Loaded: loaded (/usr/lib/systemd/system/redis.service; disabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/redis.service.d
└─limit.conf
Active: active (running) since Sun 2018-02-18 00:08:22 AEDT; 4s ago
Main PID: 29944 (redis-server)
Tasks: 4 (limit: 4915)
CGroup: /system.slice/redis.service
└─29944 /usr/bin/redis-server 127.0.0.1:6379
Feb 18 00:08:22 fedora.home systemd[1]: Starting Redis persistent key-value database...
Feb 18 00:08:22 fedora.home systemd[1]: Started Redis persistent key-value database.
..
Check if Redis is alive
Once the server has started, let’s check if our server is up and running:
$ redis-cli ping
PONG
Sorted Sets
Redis’ sorted set is a set data structure but each element in the set is also associated with a score. It is a
hash map but with an interesting property - the set is ordered based on this score value.
This allows us to perform the following operations easily:
- Retrieve the top or bottom 10 keys based on the score
- Find the rank/position of a key in the set
- The score of a key can be updated anytime while the set will be adjusted (if needed) based on the new score
The section on sorted sets here and here in the Redis docs has more details on this.
Example scenario: Top tags
We will now create a sorted set called tags. This set will store tags for posts in a blog or some other content
system where entries can have one or more tags associated with them. At any given point of time, we would like to
know what are the top 5 tags in our system.
redis-cli demo
We will first add a few tags to our sorted set tags using the ZADD command:
127.0.0.1:6379> ZADD tags 1 "python"
(integer) 1
127.0.0.1:6379> ZADD tags 1 "golang"
(integer) 1
127.0.0.1:6379> ZADD tags 1 "redis"
(integer) 1
127.0.0.1:6379> ZADD tags 1 "flask"
(integer) 1
127.0.0.1:6379> ZADD tags 1 "rust"
(integer) 1
127.0.0.1:6379> ZADD tags 2 "rust"
(integer) 0
127.0.0.1:6379> ZADD tags 3 "python"
(integer) 0
127.0.0.1:6379> ZADD tags 1 "docker"
(integer) 1
127.0.0.1:6379> ZADD tags 1 "linux"
(integer) 1
127.0.0.1:6379> ZADD tags 1 "c"
(integer) 1
127.0.0.1:6379> ZADD tags 1 "software"
(integer) 1
1
127.0.0.1:6379> ZADD tags 1 "memcache"
(integer) 1
Above, I used the command to update the score of rust and python twice to be 2 and 3 respectively. I could have used
ZINCRBY as well. Now, I will list all the keys using the zrange command:
127.0.0.1:6379> zrange tags 0 -1
1) "c"
2) "docker"
3) "flask"
4) "golang"
5) "linux"
6) "memcache"
7) "redis"
8) "software"
9) "rust"
10) "python"
Note how the last two keys are rust and python - as they have the highest scores (2 and 3 respectively). The others are
sorted lexicographically.
To reverse the order, we will use the zrevrange command:
127.0.0.1:6379> ZREVRANGE tags 0 -1 withscores
1) "python"
2) "3"
3) "rust"
4) "2"
5) "redis"
6) "1"
7) "golang"
8) "1"
9) "flask"
10) "1"
Above, we can see how with the withscores command, we also get the scores back.
Now, to get the top 5 tags, we will do the following:
127.0.0.1:6379> ZREVRANGE tags 0 4 withscores
1) "python"
2) "3"
3) "rust"
4) "2"
5) "software"
6) "1"
7) "redis"
8) "1"
9) "memcache"
10) "1"
1
Python demo
We will use the redis-py package to talk to redis and perform the above operations. The Python client looks as follows:
import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
tags_scores = {
'rust': 2,
'python': 3,
'golang': 1,
'redis': 1,
'docker': 1,
'linux': 1,
'software': 1,
'c': 1,
'memcache': 1,
'flask': 1,
}
# Add the keys with scores
for tag, score in tags_scores.items():
r.zadd('tags', score, tag)
# Retrieve the top 5 keys
for key, score in r.zrevrange('tags', 0, 4, 'withscores'):
print(key, score)
Running the above (How?) will give us the output:
b'python' 3.0
b'rust' 2.0
b'software' 1.0
b'redis' 1.0
b'memcache' 1.0
Note above how the syntax for the Python wrappers are almost the same as the corresponding redis CLI command.
Golang demo
We will use the go-redis package to interact with redis. The following program shows how we can achieve the above in Go:
package main
import (
"log"
"github.com/go-redis/redis"
)
func main() {
client := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Password: "", // no password set
DB: 0, // use default DB
})
tags := map[string]float64{
"python": 3,
"memcache": 1,
"rust": 2,
"c": 1,
"redis": 1,
"software": 1,
"docker": 1,
"go": 1,
"linux": 1,
"flask": 1,
}
for tag, score := range tags {
_, err := client.ZAdd("tags", redis.Z{score, tag}).Result()
if err != nil {
log.Fatalf("Error adding %s", tag)
}
}
result, err := client.ZRevRangeWithScores("tags", 0, 4).Result()
if err != nil {
log.Fatalf("Error retrieving top 5 keys: %v", err)
}
for _, zItem := range result {
log.Printf("%v\n", zItem)
}
}
When we run the program after having done the necessary setup, we will see the following output:
$ go run sorted_sets.go
2018/02/18 23:28:41 {3 python}
2018/02/18 23:28:41 {2 rust}
2018/02/18 23:28:41 {1 software}
2018/02/18 23:28:41 {1 redis}
2018/02/18 23:28:41 {1 memcache}
And that’s all for this post.