« First NYC Machine Learning Meetup | Main | Detect and Parse Atom and RSS Feed URLs from HTML with TruffleHog »

October 08, 2009


Pete DeLaurentis

Hey Paul,

Thanks for writing this up - it's a big help. We're using passenger + nginx, and I'm planning to use your technique for caching some of our RESTful API calls.

Here's one question, which I thought you might have some insight on:

I want the 400-1000% speed boost of nginx directly serving the memcached content as you've done here, but I still need to authenticate users before they can see that data. (Fastest idea I've come up with so far is to hit Ruby for authentication, then send a redirect back to nginx to serve up the content. But it seems wasteful to hit Ruby soley for authentication.)

Have you tried your technique in combination with any nginx authentication plugins?

Pete DeLaurentis

Paul Dix

Hey Pete,
I haven't tried that before. As for handing it off to ruby and then going over to nginx, it seems like you wouldn't get the speed improvement. Once you've hit the Ruby process you've already lost.

I'm sure you could keep it inside an nginx module. The easiest way I can think of (although probably not totally secure) is to store the session in memcached with some sensible timeout. That session id would get passed around in the cookie. That way when a request comes in you can check to see if its there inside nginx and if it is then hit memcached again for the resource.

Just an idea, not sure if that will work. I'd be interested in hearing what you come up with.

Pete DeLaurentis

Thanks for the response on this Paul. As I moved forward with the design, an additional requirement was introduced: selectively allowing access to content based on permissions.

The design I ended up with: authentication and permission data stored in memcached, and retrieved by Ruby with each request. The content is stored in memcached too, so for most reads, the database doesn't need to be accessed.

I think authentication and selective permissions are a very common need, and if we can't get the performance we're looking for, I might have together a custom nginx module. The trick would be figuring out a good general way to represent it that others could hook into.

The comments to this entry are closed.

My Photo



Twitter / pauldix