Using the Nginx Memcached module with Passenger

Nginx, everyone's favorite speedy web server has a module to hook in directly to memcached. For those of us running Ruby servers behind nginx we can avoid hitting our running Ruby processes completely on a cache hit. I'll avoid the details and point you instead to Ilya Grigorik's coverage of the Nginx + Memcached goodness.

My trouble was that I couldn't find a good example of using Passenger and the Nginx memcached module together. The module documentation shows how to proxy cache misses to your running web servers (like Mongrel), but with Passenger you don't know what processes are running. This is because Passenger automatically spawns web server processes for you. Well, after a little trial and error I figured out a config that works.

I'll show the config at the bottom of this post, but first let's look at some performance numbers. For my use case this goes beyond simple page caching. I'm using HTTP services to serve up data. So it's entirely possible that I can achieve a high cache hit ratio and make things really speedy. To check performance I set up two tests. The first returns a small json object and the second returns a collection of objects. The test Passenger application is a simple Sinatra service that looks like this:

require 'rubygems'
require 'json'
require 'sinatra'

get '/api/v1/comments/users/:id/no_cache' do
 ([{:id => 2, 
   :user_id => "paul", 
   :body => "this is a test comment to see how this thing works."}
  ] * 25).to_json
end

get '/api/v1/comments/:id/no_cache' do
 {:id => 2, 
  :user_id => "paul", 
  :body => "this is a test comment to see how this thing works."}.to_json
end

The setup was a single small EC2 instance running nginx and another small instance in the same availability zone running Apache Bench against the server. The first run is for cache misses that fall through to Passenger. The second run is against cache hits that go from nginx straight to memcached. Here's a small section of the results:

# single json object cache miss (passenger)
ab -n 10000 -c 50 http://ec2-some-ip.compute-1.amazonaws.com/api/v1/comments/1/no_cache
Requests per second: 310.08 [#/sec] (mean)
Time per request: 161.249 [ms] (mean)
Time per request: 3.225 [ms] (mean, across all concurrent requests)
Transfer rate: 94.17 [Kbytes/sec] received

# single json object cache hit (nginx + memcached)
ab -n 10000 -c 50 http://ec2-some-ip.compute-1.amazonaws.com/api/v1/comments/1
Requests per second: 2496.64 [#/sec] (mean)
Time per request: 20.027 [ms] (mean)
Time per request: 0.401 [ms] (mean, across all concurrent requests)
Transfer rate: 556.12 [Kbytes/sec] received

# collection of 25 json objects cache miss (passenger)
ab -n 10000 -c 50 http://ec2-some-ip.compute-1.amazonaws.com/api/v1/comments/users/1/no_cache
Requests per second: 225.98 [#/sec] (mean)
Time per request: 221.256 [ms] (mean)
Time per request: 4.425 [ms] (mean, across all concurrent requests)
Transfer rate: 530.31 [Kbytes/sec] received

# collection of 25 json objects cache hit (nginx + memcached)
Requests per second: 2385.22 [#/sec] (mean)
Time per request: 20.962 [ms] (mean)
Time per request: 0.419 [ms] (mean, across all concurrent requests)
Transfer rate: 5434.29 [Kbytes/sec] received

As you can see the time per request and the number of concurrent requests is significantly higher for the cache hits. The performance bump this reflects is actually better than the 400% Ilya mentioned in his blog post (of course, your mileage may vary). If you're interested, I've posted the full ab output in this gist.

And finally, without any more delay, here's the nginx.conf I used.

user nobody nogroup;
worker_processes 4;

error_log logs/error.log;

events {
 worker_connections 1024;
}

http {
 passenger_root /usr/lib/ruby/gems/1.8/gems/passenger-2.2.5;
 passenger_ruby /usr/bin/ruby1.8;

 include mime.types;
 default_type application/octet-stream;
 tcp_nopush on;
 tcp_nodelay on;
 gzip on;

 sendfile on;

 keepalive_timeout 65;

 server {
 listen 80;
 server_name localhost; # put in your IP in place of localhost
 location / {
 set $memcached_key $uri;
 memcached_pass 127.0.0.1:11211;
 default_type application/json;
 error_page 404 502 = @fallback;
 }

 location @fallback {
 root /var/www/public;
 passenger_enabled on;
 }
 }
}