How many requests per second can I get out of Redis?

Warning: This is not a very interesting post. I'm toying around with the Redis benchmarking tool. What would be significantly more interesting would be to toy around with the Lua API in Redis, which I'll do in a subsequent post.

In this post, I'll try to squeeze as many get/set requests out of Redis as I can. I'll use the redis-benchmark tool to test just the set and get commands. This is not meant to be a benchmark, but a learning experience to see "what works".

I'm testing the current stable version of Redis: 2.6.15.

Basic testing approach

First, compile Redis from source (it should "just work") and place the binaries somewhere useful. Next, start Redis server (I use port 7777 for no specific reason):

redis-server --port 7777

To test (set and get):

redis-benchmark -p 7777 -t set,get -q

You should use the redis-benchmark tool to benchmark Redis, for exactly the reasons mentioned in the pitfalls and misconception section on the Redis benchmarking page. The primary reason is that the tool uses multiple connections, and easily enables commands to be pipelined.

This command above uses the -p switch to set the port, uses the -t to limit the commands we test, and finally the -q switch to limit the output to just the throughput.

Additional notes

Redis is a single-threaded server. Unfortunately it does not seems possible to use the benchmark tool to load-balance over several Redis instances, say running on different ports on the same machine. I guess nothing is keeping me from using consistent hashing (or another partitioning technique) with Redis, but the benchmarking tool does not seem to support any kind of partitioning.

Antirez has a blog post about using a Redis proxy called Twemproxy for doing partitioning with Redis. It can potentially increase the throughput. Unfortunately the Proxy uses the epoll system call in Linux, which does not exist on Mac OS X (where kqueue is used instead), so I can not try it.

All in all, I'll be evaluating Redis in a purely single-node setup, using a TCP loopback connection to the Redis server running on my laptop.

A further thing that is noted on the benchmarking page is that:

Finally, when very efficient servers are benchmarked (and stores like Redis or memcached definitely fall in this category), it may be difficult to saturate the server. Sometimes, the performance bottleneck is on client side, and not server-side. In that case, the client (i.e. the benchmark program itself) must be fixed, or perhaps scaled out, in order to reach the maximum throughput.

Another reason that Redis may not be saturated by the benchmark is that Redis throughput may is limited by the network well before being limited by the CPU. As I'm running on a local machine, I'm assuming that this is not the case, but I'm not entirely sure that there are not other bottlenecks in the OS regarding communication between the benchmark process and the redis-server process. As noted on the benchmarking page: When client and server run on the same box, the CPU is the limiting factor with redis-benchmark.

Let's keep all that in mind.

1: Running Redis server on my slightly old Macbook Pro

This is the 100% lazy installation. I compiled Redis from source on my laptop, using all defaults, and simply started it.

Hardware: 2.66 GHz Intel Core 2 Duo, 4 GB 1067 Mhz DDR3
OS: Mac OS X 10.6.8 (Snow Leopard)

The result is 37K and 38K requests per second for set and get respectively:

$ redis-benchmark -p 7777 -t set,get -q
SET: 37174.72 requests per second
GET: 37313.43 requests per second

The standard test uses just a single key. To increase the number of expected cache misses, I'll run the same test using a million random keys (using the -r switch to set number of keys) to see if it makes a huge difference:

redis-benchmark -p 7777 -t set,get -r 1000000 -q

The difference is roughly 2.8% for set and 3% for get. Nothing dramatic. The performance overall is however not great for this initial setup running unmodified on my laptop.

2: Using pipelining

Now I'll read the fucking manual. Maybe it helps. Redis has a page about benchmarking Redis. The first suggestion is to use pipelining. It is enabled by using the -P switch with an argument of the number of commands to bunch together in each request. I'll try 16 as suggested on the page.

$ redis-benchmark -p 7777 -t set,get -P 16 -q
SET: 222222.22 requests per second
GET: 256410.25 requests per second

Actually the throughput varies a lot between different runs of this test, much more than the non-pipelined test. With that in mind, it seems that using a pipeline level of 100 is better than 16, about 30% higher throughput:

$ redis-benchmark -p 7777 -t set,get -P 100 -q
SET: 312500.00 requests per second
GET: 333333.34 requests per second

But using a pipeline level of 1000 is worse. Again there is a lot of variance, so I'd need to do a proper statistical analysis. Here I'm doing a rough estimation, and using pipelining of 100 dominates 1000, and that is all I care about.

Bottom line is that you can get 1 order of magnitude improvement to throughput by using pipelining, at least on my old Macbook Pro. Maybe it will be more or less on a "proper" server.

The question is, can we do better?

3: Using lua scripting

Redis supports Lua scripts that are evaluated server side. This can improve the throughput in the situation where a read is followed by a computation follow by say a write. Without scripting, even if using pipelining, there would be a roundtrip following the initial read in order to do the computation. The benefits of scripting are really application specific, and I'll not explore that further.

4: Various potential optimizations

• Use another memory allocation library. Default is libc. Unlikely to have any dramatic effect on the test
• Other things to consider?

I have not tried any of these optimizations.

5: Givin'er all she's got!

On the Redis page there are results posted for a high-end server, using TCP loopback (like I am) and without pipelining.

Here are the results for a 2 x Intel X5670 @ 2.93 GHz (without pipelining):

SET: 142653.36 requests per second
GET: 142450.14 requests per second

For Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (with pipelining):

SET: 552028.75 requests per second
GET: 707463.75 requests per second

Note that these are not the same machines.

That is roughly 3.8x increase in throughput (compared to my laptop), in the non-pipelining case (run on the high-end server) and roughly 2x in the pipelining case (run on the not-quite-as-high-end server). Again, take the numbers with a big grain of salt. They essentially say nothing wildly interesting. The main conclusion is that pipelining and perhaps Lua scripting is a good idea. Also that partitioning may improve throughput, in which case you could try the Twemproxy code if you're on Linux.

Conclusion and next steps

Using a single-node instance of Redis running on my laptop I managed to get 300K requests per second (both get and set). This is achieved only if using pipelining (100 commands at a time). On a high-end machine someone got 700K get requests per second using pipelining, i.e. a bit more than twice the throughput.

My goal is to squeeze 1 million get requests per second out of Redis, for a "realistic workload". For this I'll use a partitioning approach. The approach is to use Twemproxy running on a multi-core Linux machine with several Redis instances. The exact setup will take some experimenting to get right.

Out of the box, both pipelining and Lua scripting are good avenues for improving performance with Redis server. I saw 1 order of magnitude improvement to throughput when using pipelining. Both approaches are quite application specific, perhaps Lua scripting more so than pipelining. I did not experiment with lua scripting. That would also be very interesting to try.