Friday, September 10, 2010

Cherokee web server

Cherokee is really really fast. The speed at which any web server can serve requests for content is both directly tied to and limited by the I/O speed of underlying hardware and operating system. In this regard, a web server's performance is measured by the latency incurred after an I/O request to the underlying system has completed.
The primary objective of the Cherokee project is to reduce to zero the latency incurred between the time a dynamic or static system I/O request has completed and the time the resulting content is served to the requesting client. Admittedly, a very tough goal to reach.

In fact, it might be impossible. But achieving that which was once seen as impossible is what drives innovation. And it's innovation that drives the ongoing development of the Cherokee project, bringing us closer to the impossible with each new release.

To show the project's progression towards the ultimate goal, whenever a benchmark is performed it will be published right here. Older releases had impressive but not very thorough benchmarks. Whenever it is possible the conditions of the benchmark will be provided so that anyone can replicate the results. That is, after all, an essential basis of the Scientific method.

The benchmark consisted on half a million requests of a 1.7KiB static file, with 20 concurrent clients, using a 1Gbit/s local network. The results (fastest to slowest) were:
Cherokee:

Server Software:        Cherokee/0.8.1
Server Hostname:        10.0.0.102
Server Port:            80

Document Path:          /index.html
Document Length:        1795 bytes

Concurrency Level:      20
Time taken for tests:   17.819725 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    500000
Total transferred:      999007442 bytes
HTML transferred:       897506630 bytes
Requests per second:    28058.79 [#/sec] (mean)
Time per request:       0.713 [ms] (mean)
Time per request:       0.036 [ms] (mean, across all concurrent requests)
Transfer rate:          54747.93 [Kbytes/sec] received

Lighttpd:
Server Software:        lighttpd/1.4.19
Server Hostname:        10.0.0.102
Server Port:            80

Document Path:          /index.html
Document Length:        1795 bytes

Concurrency Level:      20
Time taken for tests:   21.248000 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    470598
Total transferred:      991856958 bytes
HTML transferred:       897503590 bytes
Requests per second:    23531.63 [#/sec] (mean)
Time per request:       0.850 [ms] (mean)
Time per request:       0.042 [ms] (mean, across all concurrent requests)
Transfer rate:          45585.94 [Kbytes/sec] received

NginX:
Server Software:        nginx/0.5.33
Server Hostname:        10.0.0.102
Server Port:            80

Document Path:          /index.html
Document Length:        1795 bytes

Concurrency Level:      20
Time taken for tests:   23.741872 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    500000
Total transferred:      1006000217 bytes
HTML transferred:       897500000 bytes
Requests per second:    21059.84 [#/sec] (mean)
Time per request:       0.950 [ms] (mean)
Time per request:       0.047 [ms] (mean, across all concurrent requests)
Transfer rate:          41379.30 [Kbytes/sec] received

Apache2.2:
Server Software:        Apache/2.2.8
Server Hostname:        10.0.0.102
Server Port:            80

Document Path:          /index.html
Document Length:        1795 bytes

Concurrency Level:      20
Time taken for tests:   35.438605 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    495064
Total transferred:      1043777896 bytes
HTML transferred:       897500000 bytes
Requests per second:    14108.91 [#/sec] (mean)
Time per request:       1.418 [ms] (mean)
Time per request:       0.071 [ms] (mean, across all concurrent requests)
Transfer rate:          28762.81 [Kbytes/sec] received
For the record: I did my best configuring all the servers in the very same way. In all the cases I removed unnecessary rules that could have slowed down the server (checks for htpasswd files and so on). And all the binaries came from the Debian repository, except for Cherokee 0.8.1 that hasn't been packaged yet.
Anyway, this benchmark has been just a quick test. It is not certainly representing the result that these servers would have handling real traffic though. So, in the following days I will try to do a new a more accurate benchmark with static and dynamic content, compression, redirections, etc. I'm pretty sure the results will be even better.

Cherokee + Apache + Lighttpd Benchmark

This benchmark was performed by Brian Rosner with Cherokee 0.6.0 beta2.

Software

  • cherokee 0.6.0 beta2
  • apache 2.0.59
  • lighttpd 1.4.16

Hardware

  • 733 MHz PIII
  • 256 MB RAM
  • 80GB 7200RPM IDE HD
  • Debian GNU/Linux 4.0

Background

I installed a fresh installation of Debian on the server hardware. Right after you login you will need to get sudo to perform root commands from your account: 
su
apt-get install sudo
Then add yourself to the /etc/sudoers file by running visudo and adding yourself in the user section. I just followed the root entry as this does not need to be a very secure server since it will not be running publicly. Now make sure you get back to your account and do: 
sudo apt-get install gcc make automake autoconf libtool
mkdir src ; cd src
sudo mkdir /usr/local/cherokee
sudo mkdir /usr/local/lighttpd
The installed version of gcc is 4.1.2

Cherokee Setup Details

The following is what I executed to build Cherokee: 
wget http://www.cherokee-project.com/download/0.6/0.6.0/cherokee-0.6.0b863.tar.gz
tar zxvf cherokee-0.6.0b863.tar.gz
cd cherokee-0.6.0b863
./configure --prefix=/usr/local/cherokee/0.6.0b863
make
sudo make install
Here is the configuration for cherokee: 
server!port = 80
server!timeout = 60
server!keepalive = 1
server!keepalive_max_requests = 500
server!pid_file = /var/run/cherokee.pid
server!server_tokens = full
server!encoder!gzip!allow = html,html,txt
server!panic_action = /usr/local/cherokee/0.6.0b863/bin/cherokee-panic
server!mime_files = /usr/local/cherokee/0.6.0b863/etc/cherokee/mime.types

vserver!default!document_root = /usr/local/cherokee/0.6.0b863/var/www
vserver!default!directory_index = index.html

vserver!default!directory!/!handler = common
vserver!default!directory!/!handler!iocache = 1
vserver!default!directory!/!priority = 1
To run the web server I used: 
cd /usr/local/cherokee/0.6.0b863
sudo sbin/cherokee -C etc/cherokee/cherokee.conf

Apache Setup Details

The following is what I executed to build Apache: 
wget http://apache.oregonstate.edu/httpd/httpd-2.0.59.tar.gz
tar zxvf httpd-2.0.59.tar.gz
cd httpd-2.0.59
./configure --prefix=/usr/local/apache/2.0.59
make
sudo make install
I used the supplied highperformance.conf configuration file. I started the server with: 
cd /usr/local/apache/2.0.59
sudo bin/httpd -k start -f conf/highperformance.conf
The server ran using prefork.

Lighttpd Setup Details

The following is what I executed to build lighttpd: 
wget http://www.lighttpd.net/download/lighttpd-1.4.16.tar.gz
tar zxvf lighttpd-1.4.16.tar.gz
cd lighttpd-1.4.16
./configure --prefix=/usr/local/lighttpd/1.4.16
make
sudo make install
The configuration I used looked like this: 
server.modules = (
    "mod_access",
    "mod_accesslog"

)

server.document-root = "/var/www"

mimetype.assign = (
    ".html" => "text/html",
    ".txt" => "text/plain"

)
I started the server with: 
cd /usr/local/lighttpd/1.4.16
sudo sbin/lighttpd -f sbin/lighttpd.conf

Benchmark

I will perform several different benchmarks on each webserver. This is to help gauge what type of performance each server can handle in the different conditions. Each test will have SSL turned on and turned off.

small static file test

  • filesize: 99 bytes
  • command: ab -c 2 -t 2 -k http://localhost/index0.html

large static file test

  • filesize: 1.5MB
  • command: ab -c 2 -t 2 -k http://localhost/static.txt

Results

I have included cherokee with both iocaching on and off. The out of the box setting is that iocache is turned on.

small static file test w/ keepalive

  • cherokee 0.6.0b863 w/ iocache - 7816 reqs./sec.
  • cherokee 0.6.0b863 w/o iocache - 5761 reqs./sec.
  • lighttpd 1.4.16 - 4884 reqs./sec.
  • apache 2.0.59 - 2924 reqs./sec.

small static file test w/o keepalive

  • cherokee 0.6.0b863 w/ iocache - 2182 reqs./sec.
  • cherokee 0.6.0b863 w/o iocache - 1874 reqs./sec.
  • lighttpd 1.4.16 - 2255 reqs./sec.
  • apache 2.0.59 - 1250 reqs./sec.

large static file test w/ keepalive

  • cherokee 0.6.0b863 w/ iocache - 108 reqs./sec.
  • cherokee 0.6.0b863 w/o iocache - 107 reqs./sec.
  • lighttpd 1.4.16 - 106 reqs./sec.
  • apache 2.0.59 - 94 reqs./sec.

large static file test w/o keepalive

  • cherokee 0.6.0b863 w/ iocache - 88 reqs./sec.
  • cherokee 0.6.0b863 w/o iocache - 88 reqs./sec.
  • lighttpd 1.4.16 - 92 reqs./sec.
  • apache 2.0.59 - 118 reqs./sec.
Courtesy : http://www.cherokee-project.com/benchmarks.html
Posted by at
Labels: , , , , ,