Playing with Lighttpd, Litespeed, FCGI and Mongrel

We’ve been big pushers (not advocates but downright pushers) of alternative web servers, primarily ones that at least support fastcgi and have proxying/load balancing capabilities. We do this despite being big fans and long time users of Apache (SVN and DAV in Apache is primarily what we still use it for. Of course the WebDAV implementation in the Joyent application is entirely in ruby but that’s a discussion for another time).

There’s lighttpd (endorsed and supported here), litespeed. We’ve been using litespeed since we got all the little things “fixed” in it to support ruby-fcgi back in October 2005. Fixed is in quotes because it’s not litespeed’s fault, the problem with it was that litespeed (unlike Apache or lighttpd) supports persistent fastcgi. Now I know what you’re asking, “I thought that fastcgi was always persistent?” Yes that’s correct, but the connection between the web server and the fastcgi processes are not. It’s the same as the difference between pconnect and connect in PHP with MySQl (which is a persistent server process). Litespeed happens to be the only one that also does persistent connections between itself and the fastcgi processes (Can you say “firehose”?). The easier solution was to introduce an option in litespeed to turn persistent fastcgi connections off (we’ll fix the ruby fastcgi connector later).

Then there’s mongrel, Zed Shaw’s ruby (and there rails) as HTTP.

So I thought I’d take two Dell 2850s, 4GB RAM, Dual 3.6 Ghz, 73GB SCSI drives, over a trunked 2x gigabit connection.

We’ll just look at Lighttpd with FCGI, Litespeed with FCGI and Lighttpd with Mongrel (Litespeed + Mongrel was comparable). I also won’t take it across the entire concurrency spectrum but will save that for another time.

Lighttpd-fcgi (mod_fastcgi)

ApacheBench


$ ab -n 1000 -c 10 http://private.textdrive.com/
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.121.2.12 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking private.textdrive.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests

Server Software: lighttpd/1.4.9
Server Hostname: private.textdrive.com
Server Port: 80

Document Path: /
Document Length: 11047 bytes

Concurrency Level: 10
Time taken for tests: 6.318620 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 11228000 bytes
HTML transferred: 11047000 bytes
Requests per second: 158.26 [#/sec] (mean)
Time per request: 63.186 [ms] (mean)
Time per request: 6.319 [ms] (mean, across all concurrent requests)
Transfer rate: 1735.19 [Kbytes/sec] received

Connection Times (ms)

min mean[+/-sd] median max
Connect: 0 0 1.5 1 14
Processing: 15 61 32.0 55 356
Waiting: 14 58 31.9 52 353
Total: 15 62 32.1 56 356

Percentage of the requests served within a certain time (ms)

50% 56

66% 67

75% 74

80% 80

90% 98

95% 116

98% 133

99% 196

100% 356 (longest request)

Siege (light)


Lifting the server siege... done. Transactions: 2834 hits
Availability: 100.00 %
Elapsed time: 60.19 secs
Data transferred: 29.86 MB
Response time: 0.03 secs
Transaction rate: 47.09 trans/sec
Throughput: 0.50 MB/sec
Concurrency: 1.43
Successful transactions: 2834
Failed transactions: 0
Longest transaction: 0.15
Shortest transaction: 0.01

Lighttpd-mongrel (via mod_proxy)

ApacheBench


$ ab -n 1000 -c 10 http://private.mongrel.textdrive.com/
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.121.2.12 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking private.mongrel.textdrive.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests

Server Software: lighttpd/1.4.9
Server Hostname: private.mongrel.textdrive.com
Server Port: 80

Document Path: /
Document Length: 11047 bytes

Concurrency Level: 10
Time taken for tests: 6.760181 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 11228000 bytes
HTML transferred: 11047000 bytes
Requests per second: 147.93 [#/sec] (mean)
Time per request: 67.602 [ms] (mean)
Time per request: 6.760 [ms] (mean, across all concurrent requests)
Transfer rate: 1621.85 [Kbytes/sec] received

Connection Times (ms)

min mean[+/-sd] median max
Connect: 0 0 1.5 0 9
Processing: 19 65 33.9 57 398
Waiting: 15 62 33.3 54 394
Total: 19 66 33.9 58 398

Percentage of the requests served within a certain time (ms)

50% 58

66% 71

75% 79

80% 85

90% 106

95% 124

98% 157

99% 177

100% 398 (longest request)

Siege


Lifting the server siege... done. Transactions: 2839 hits
Availability: 100.00 %
Elapsed time: 60.30 secs
Data transferred: 29.91 MB
Response time: 0.03 secs
Transaction rate: 47.08 trans/sec
Throughput: 0.50 MB/sec
Concurrency: 1.41
Successful transactions: 2839
Failed transactions: 0
Longest transaction: 0.13
Shortest transaction: 0.01

Litespeed-fcgi (non-persistent)

ApacheBench


$ ab -n 1000 -c 10 http://private.lite.textdrive.com/
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.121.2.12 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking private.lite.textdrive.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests

Server Software: LiteSpeed
Server Hostname: private.lite.textdrive.com
Server Port: 80

Document Path: /
Document Length: 11047 bytes

Concurrency Level: 10
Time taken for tests: 5.987303 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 11193896 bytes
HTML transferred: 11049752 bytes
Requests per second: 167.02 [#/sec] (mean)
Time per request: 59.873 [ms] (mean)
Time per request: 5.987 [ms] (mean, across all concurrent requests)
Transfer rate: 1825.70 [Kbytes/sec] received

Connection Times (ms)

min mean[+/-sd] median max
Connect: 0 0 1.3 0 6
Processing: 20 58 23.2 54 164
Waiting: 14 51 20.5 47 141
Total: 22 59 23.2 55 164

Percentage of the requests served within a certain time (ms)

50% 55

66% 63

75% 68

80% 74

90% 91

95% 105

98% 126

99% 137

100% 164 (longest request)

Mild Siege


Lifting the server siege... done. Transactions: 3051 hits
Availability: 100.00 %
Elapsed time: 60.21 secs
Data transferred: 32.14 MB
Response time: 0.03 secs
Transaction rate: 50.67 trans/sec
Throughput: 0.53 MB/sec
Concurrency: 1.46
Successful transactions: 3051
Failed transactions: 0
Longest transaction: 0.16
Shortest transaction: 0.01

Final thoughts

Then I nuked them with a high concurrency and alternating bursts of requests (using a very fun config for …) and while litespeed did in fact crank out at a high concurrency, lighttpd 1.4.9 went into a “D state” and it’s processes become unkillable, so the server machine had to be rebooted. It’s likely something that’s fixed in 1.4.10 (which at first release didn’t compile on FreeBSD or Solaris and I didn’t fix that compile).

Mongrel as a proxy performs just as well as fcgi does, either with lighttpd or litespeed, and could easily become an Adios FCGI thing.

Litespeed didn’t make me have to reboot a server (but it was the paid enterprise version) and combined with a few other features that I’ll cover later did make me do


$ curl -I http://textdrive.com/
HTTP/1.1 200 OK
Server: LiteSpeed
Date: Sat, 18 Feb 2006 20:01:53 GMT
Connection: close
Content-Type: text/html
Cache-Control: no-cache

You’ll also notice that Wordpress.com uses Litespeed

And dude, static files. I love ‘em


Server Software: LiteSpeed
Server Hostname: private.lite.textdrive.com
Server Port: 80

Document Path: /images/hero/2006-02.jpg
Document Length: 12973 bytes

Concurrency Level: 200
Time taken for tests: 6.289341 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 132886064 bytes
HTML transferred: 129753247 bytes
Requests per second: 1589.99 [#/sec] (mean)
Time per request: 125.787 [ms] (mean)
Time per request: 0.629 [ms] (mean, across all concurrent requests)
Transfer rate: 20633.48 [Kbytes/sec] received

In the next post, I’ll show you how Litespeed on Solaris 10 using both devpoll and sendfile() on a ZFS file system absolutely cranks.

Cranks

·:· Posted 18 February 2006, 20:03 by Jason Hoffman to Web servers  |  

  1. I really wish Zeus would wake up and give away their web server, offering up support contracts for those who want it. Cause I’d rather be running Zeus than LiteSpeed or lighttpd. It would be interesting to see how well Zeus would stack up in your performance tests!

    That being said, I haven’t looked at LiteSpeed in a long time. Perhaps it’s time for another peek. LiteSpeed’s Apache compatibility is certainly attractive compared to lighttpd or Zeus.

    — Mark Mayo    19 February 2006, 01:41    #
  2. Hmm … I have a Zeus license. Let me see if it does ruby fcgi OK.

    — jason Hoffman    19 February 2006, 02:57    #