Crossroads: a very good load balancing solution
Recently Martin Aspeli published a very nice buildout recipe that implements a complete setup for a production Zope/Plone server. One of the key components in that setup is the load balancer, which is implemented using the load balancer capabilities of nginx. Nginx might be a good load balancer, but it has a problem with the way Zope/Plone works: the first page loads, for a newly restarted Plone instance are very slow (especially if the site or the catalogs are big). It would be nice to have a way of telling nginx to take out one of the Zope instances from the cluster and then add it back once it's restarted and has its caches warmed. Sure, you could probably script something that would change the buildout options and reload nginx, but why bother when there's a load balancer that offers this and much more?
Crossroads (in its second version, at the moment 2.41) might lack the popularity when compared to perlball, pound or haproxy and may not even have the same features or performances (it's good enough for my needs and I haven't cross-benchmarked), but it's fast, easy to configure and, best of all, it can be configured to provide a web configuration/administration interface. This way, when a Zope server needs to be updated, I can take out the ZEO client from the balancer, restart it in supervisor, warm up the caches by calling the zope sites directly and then add it back to the cluster. The only faults that I could find to it is that the compilation mechanism is not very well tested (I couldn't complete an install with a specific destination prefix. Just checkout the Crossroads trunk or a tag and do a "make local", then you can use the xr binary from that folder). On the other hand, it's pretty well featured: it has sticky sessions, HTTP awarness (although its documentation recommends treating the http servers with the TCP algorithm), can use an external program to determine backend states (and could be plugged this way with Zope's ICP support), etc.
Below is a screenshot of the Crossroads administration interface, with Crossroads started with the following script:
#!/bin/sh XR=/usr/sbin/xr SERVER="-s http:127.0.0.1:7000" BACKENDS="-b 127.0.0.1:9080 -b 127.0.0.1:9081" ALGORITHM="-dl" HTTP_FLAGS="-x -X" TIMEOUTS="-t 10" CHECK_CALLS="-c 10" WEB_INTERFACE="-W 127.0.0.1:7020" #DEBUGGING="-v" DEBUGGING="" $XR $SERVER $BACKENDS $ALGORITHM $HTTP_FLAGS $TIMEOUTS $CHECK_CALLS $WEB_INTERFACE $DEBUGGING
