Benchmark various servers

I've did a quick benchmark of various pylons-servers configurations:
- Paste "developement" server (pylons 0.9.2)
- Nginx as a reverse proxy for 2 running paste servers (0.4.2)
- Nginx FastCGI
- Cherokee SCGI (0.5.4)

I've used a simple pylons page which returned a small myghty template (a search form actually) no database, no external operations. Tested on Acer Aspire 5002 WLMi AMD Turion ML-30, 512MB RAM, AMD64 Gentoo Linux. The test was divided into two parts - no errors load of 100 requests/second with 10000 total requests and critical load: 300 and 400 requests/sec with 40000 requests send. httperf was used to make the benchmark, and the test was split because occurring errors may false the results and the results doesn't have good repeatability.

This is a simple benchmark so it may not reflect real behaviour of tested software in production usage

100 requests / second

Total CPU Time (%)

Reply Response Time (ms)SCGI/Cherokee uses less CPU time, but it has the slowes reply response time (only 0,6 ms of difference). Reverse proxy got second place two times.

Plain Paste/Pylons server

Connection time [ms]: avg 4.4
CPU time [s]: total 58.3%
Reply time [ms]: response 4.3

Nginx reverse proxy

Connection time [ms]: avg 5.0
CPU time [s]: total 52.5%
Reply time [ms]: response 4.9

Nginx FastCGI

Connection time [ms]: avg 4.7
CPU time [s]: total 54.3%
Reply time [ms]: response 4.7

Cherokee SCGI

Connection time [ms]: avg 5.1
CPU time [s]: total 50.6%
Reply time [ms]: response 5.0

Critical Load

Reply Rate [replies/s]

Reply Response Time (ms) Cherokee wins - has the shortest response time and has the biggest reply rate. Nginx good name is saved only by good FastCGI reply rate :)

Plain Paste/Pylons server

300 requests/s: Reply rate [replies/s]: avg 192.3
Reply time [ms]: response 2679.1
Errors: total 13391 client-timo 571

400 requests/s: Reply rate [replies/s]: avg 180.0
Reply time [ms]: response 2615.7
Errors: total 22115 client-timo 2154

Nginx reverse proxy

300 requests/s: Reply rate [replies/s]: avg 180.6
Reply time [ms]: response 2749.3
Errors: total 15031 client-timo 1764

400 requests/s: Reply rate [replies/s]: avg 183.9
Reply time [ms]: response 2725.1
Errors: total 21746 client-timo 2142

Nginx FastCGI

300 requests/s: Reply rate [replies/s]: avg 204.6
Reply time [ms]: response 2738.2
Errors: total 12187 client-timo 627

400 requests/s: Reply rate [replies/s]: avg 205.5
Reply time [ms]: response 2843.7
Errors: total 18694 client-timo 328

Cherokee SCGI

300 requests/s: Reply rate [replies/s]: avg 209.7
Reply time [ms]: response 2548.0
Errors: total 11533 client-timo 294

400 requests/s: Reply rate [replies/s]: avg 207.3
Reply time [ms]: response 2576.3
Errors: total 18325

http://www.rkblog.rk.edu.pl/w/p/pylons-benchmark-various-servers/