toolbox: webdev unicorns: servers (2/4)


This is kind of a big topic because several components are so interconnected.

Exposing Code to the Web

I like cherrypy’s approach of navigating an object hierarchy slightly better than django’s approach of evaluating a list of regular expressions. The problem, of course, is how to deal with URIs that are illegal python symbols, such as foo~bar or A regex would make this trivial, but cherrypy makes it a little tedious: you need to specify a default method, which is called anytime a URI isn’t resolved against the symbols provided in the object tree. This can lead to a proliferation of really ugly methods where most work is done in default methods instead of in the nice and clean object hierarchy. Maybe regex is better?

Performance, Deployment, and Resource Pooling

I also like how cherrypy easily allows access to thread data, and server data. This makes things like persistent resources very clear and easy to support, such as database connections, etc…. In addition, cherrypy makes it easy to run a local copy for development purposes, and provide alternate means for running behind fastcgi (using flup or, or jonpy) with its wsgiserver. However, I question the performance. I suspect twisted is much faster, but haven’t yet investigated any comparisons. Django is able to be run directly under mod_python or fastcgi. I believe running under fastcgi is comparable in speed to mod_python, but has the huge benefit of being much much more portable. I think deplying django in this fashion also probably results in better application performance and less overhead than putting cherrypy behind fastcgi. I haven’t yet looked at how twisted can be deployed.

Apache vs lighttpd vs ?

One thing I have been doing is comparing apache with lighttpd. Unfortunately, I can’t share my results because I did the comparison using company resources. I did two tests: one was between two different implementations using different technologies, the other was between two identical implementations and the only difference was lighttpd and apache.

In the first test, the first implementation used mod_perl with apache to execute a cgi script that accessed the database. This was legacy code that I was rewriting, and I needed to answer the simple question of which performs better: my new code or the old code? My new code was a turbogears app that gave the same exact output, but ran behind lighttpd via fastcgi. lighttpd+fastcgi+turbogears+kid was faster than apache+mod_perl by a factor of almost 4, with a lower overall system impact (lower load, etc…)! A 4x speed gain is nice, but which components are responsible? I did another test with a friend’s implementation.

In the second test, my friend had written a service in C that was deployed behind fastcgi. We compared apache+mod_fcgid vs lighttpd. In this test, we saw a HUGE difference. lighttpd was able to handle 2000 successful requests per second, while apache handled about 11. There are some things we should have tweaked to make sure it was an apples-apples comparison, (such as looking at load to make sure it came down enough), but the difference in performance is too large for some of the factors to make a difference. I should probably do this test again, just to see if we get different results, but for now, I’m pretty keen on lighttpd. The problem with cherrypy, of course, is that it limits the performance of the application to whatever cherrypy is able to handle, which is significantly lower than the max of either apache or lighttpd. However, at least lighttpd will do a good job of managing that pipeline so that your applicaton won’t just die, as can happen with apache.

With all this in mind, I’m keen to take a look at twisted, to see where and how it compares. Besides twisted, are there any other mechanisms to expose code to the web that I should be looking at? How does twisted fit in?

This is the third in a five part series discussing the web, tools to develop it, and why its important, in the context of python, code, web servers, templating, and xml technologies.

intro (0/4)
The first post, in which I explain what I’m doing.
templates (1/4)
The second post, in which I discuss templating technologies.
servers (2/4)
Third post, in which I discuss server technology, and methods I like to expose code to the web, and some performance implications.
components (3/4)
In which I discuss XML technologies for python.
conclusion (4/4)
In which I briefly mention why I’m passionate about good technology.

Post a Comment

Required fields are marked *

%d bloggers like this: