Lighttpd and cherokee benchmark

I've repeated Pylons + server tests for cherokee and lighttpd under SCGI. Results show that Cherokee can handle more requests on high load. In my old benchmark under Arch Lighttpd crashed on high load - in this test lighttpd didn't crash even when I pushed 1000cons/sec. :) NOTE that this is a simple test and may not reflect production server/application performance.

CHEROKEE 0.5.4

100:
Reply time [ms]: response 4.7
CPU time [s]: total 53.8%

300:
Reply time [ms]: response 2245.3
Reply rate [replies/s]: avg 208.2
Errors: total 2355 client-timo 58

400:
Reply time [ms]: response 2232.8
Reply rate [replies/s]: avg 183.9
Errors: total 4270 client-timo 167

500:
Reply rate [replies/s]: avg 192.3
Reply time [ms]: response 2661.7
Errors: total 5102 client-timo 3

600:
Reply rate [replies/s]: avg 191.1
Reply time [ms]: response 2480.6
Errors: total 5842 client-timo 22

700:
Reply rate [replies/s]: avg 164.9
Reply time [ms]: response 2470.7
Errors: total 6527 client-timo 92

1000:
Reply rate [replies/s]: avg 166.6
Reply time [ms]: response 2441.7
Errors: total 7383 client-timo 120

LIGHTTPD 1.4.11

Loaded Modules: mod_acess, mod_scgi, mod_accesslog, no PHP

100:
CPU time [s]: total 54.8%
Reply time [ms]: response 4.6

300:
Reply rate [replies/s]: avg 174.0
Reply time [ms]: response 2628.8
Errors: total 3235 client-timo 0

400:
Reply rate [replies/s]: avg 148.1
Reply time [ms]: response 3003.9
Errors: total 4982 client-timo 0

500:
Reply rate [replies/s]: avg 142.3
Reply time [ms]: response 3255.9
Errors: total 6120 client-timo 80

600:
Reply rate [replies/s]: avg 140.0
Reply time [ms]: response 3427.6
Errors: total 6306 client-timo 16

700:
Reply rate [replies/s]: avg 111.2
Reply time [ms]: response 3488.0
Errors: total 6763 client-timo 0

1000:
Reply rate [replies/s]: avg 108.2
Reply time [ms]: response 3591.8
Errors: total 7535 client-timo 48

LIGHTTPD all modules, without rrdtool

Bit lower performance:
500 | Reply rate avg: 140.9
600 | Reply rate avg: 131.2

Benchmark various servers

I've did a quick benchmark of various pylons-servers configurations:
- Paste "developement" server (pylons 0.9.2)
- Nginx as a reverse proxy for 2 running paste servers (0.4.2)
- Nginx FastCGI
- Cherokee SCGI (0.5.4)

I've used a simple pylons page which returned a small myghty template (a search form actually) no database, no external operations. Tested on Acer Aspire 5002 WLMi AMD Turion ML-30, 512MB RAM, AMD64 Gentoo Linux. The test was divided into two parts - no errors load of 100 requests/second with 10000 total requests and critical load: 300 and 400 requests/sec with 40000 requests send. httperf was used to make the benchmark, and the test was split because occurring errors may false the results and the results doesn't have good repeatability.

This is a simple benchmark so it may not reflect real behaviour of tested software in production usage

100 requests / second

Total CPU Time (%)

Reply Response Time (ms)SCGI/Cherokee uses less CPU time, but it has the slowes reply response time (only 0,6 ms of difference). Reverse proxy got second place two times.

Plain Paste/Pylons server

Connection time [ms]: avg 4.4
CPU time [s]: total 58.3%
Reply time [ms]: response 4.3

Nginx reverse proxy

Connection time [ms]: avg 5.0
CPU time [s]: total 52.5%
Reply time [ms]: response 4.9

Nginx FastCGI

Connection time [ms]: avg 4.7
CPU time [s]: total 54.3%
Reply time [ms]: response 4.7

Cherokee SCGI

Connection time [ms]: avg 5.1
CPU time [s]: total 50.6%
Reply time [ms]: response 5.0

Critical Load

Reply Rate [replies/s]

Reply Response Time (ms) Cherokee wins - has the shortest response time and has the biggest reply rate. Nginx good name is saved only by good FastCGI reply rate :)

Plain Paste/Pylons server

300 requests/s: Reply rate [replies/s]: avg 192.3
Reply time [ms]: response 2679.1
Errors: total 13391 client-timo 571

400 requests/s: Reply rate [replies/s]: avg 180.0
Reply time [ms]: response 2615.7
Errors: total 22115 client-timo 2154

Nginx reverse proxy

300 requests/s: Reply rate [replies/s]: avg 180.6
Reply time [ms]: response 2749.3
Errors: total 15031 client-timo 1764

400 requests/s: Reply rate [replies/s]: avg 183.9
Reply time [ms]: response 2725.1
Errors: total 21746 client-timo 2142

Nginx FastCGI

300 requests/s: Reply rate [replies/s]: avg 204.6
Reply time [ms]: response 2738.2
Errors: total 12187 client-timo 627

400 requests/s: Reply rate [replies/s]: avg 205.5
Reply time [ms]: response 2843.7
Errors: total 18694 client-timo 328

Cherokee SCGI

300 requests/s: Reply rate [replies/s]: avg 209.7
Reply time [ms]: response 2548.0
Errors: total 11533 client-timo 294

400 requests/s: Reply rate [replies/s]: avg 207.3
Reply time [ms]: response 2576.3
Errors: total 18325

http://www.rkblog.rk.edu.pl/w/p/pylons-benchmark-various-servers/