Changeset 1894

Show
Ignore:
Timestamp:
08/23/08 15:27:14 (3 months ago)
Author:
taher
Message:

--

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • web/benchmarks.php

    r1855 r1894  
    4242<div class="clearfix"><br /><br /><br /><br /><br /><br /><br /><br /></div> 
    4343 
     44<!-- Benchmark marker --> 
     45 
     46 
     47 
     48 
     49<h2>Cherokee 0.8.1 benchmark</h2> 
     50This is just a quick benchmark made by <a href="http://www.alobbs.com/1345/Cherokee_0_8_1_benchmark.html">Alvaro</a> to have something to show on the release date. Better, more representative benchmarks should be added shortly. 
     51 
     52<hr> 
     53 
     54    <p>Today, after releasing <a href="http://lists.octality.com/pipermail/cherokee/2008-August/008749.html">Cherokee 0.8.1</a> I decided it was time to test again how it was doing about performance. In the last months we have got a great new I/O cache layer and a couple of structural changes that were supposed to have a positive impact on the general server performance, although nobody checked whether those changes were as effective as we aimed them to be.</p> 
     55 
     56<p>So, once again, here I am bringing good news. We have done it. Cherokee is actually the fastest web server among a set of the most common servers nowadays: Apache, Cherokee, Lighttpd and Nginx!!</p> 
     57 
     58<div align="center"><img src="http://www.alobbs.com/images/web-servers-benchmark-20080819.jpg" /></div> 
     59 
     60<p>The benchmark consisted on half a million requests of a 1.7KiB static file, with 20 concurrent clients, using a 1Gbit/s local network. The results (fastest to slowest) were:</p> 
     61 
     62<p>Cherokee:</p> 
     63<pre class="shell">Server Software:        Cherokee/0.8.1 
     64Server Hostname:        10.0.0.102 
     65Server Port:            80 
     66 
     67Document Path:          /index.html 
     68Document Length:        1795 bytes 
     69 
     70Concurrency Level:      20 
     71Time taken for tests:   17.819725 seconds 
     72Complete requests:      500000 
     73Failed requests:        0 
     74Write errors:           0 
     75Keep-Alive requests:    500000 
     76Total transferred:      999007442 bytes 
     77HTML transferred:       897506630 bytes 
     78Requests per second:    28058.79 [#/sec] (mean) 
     79Time per request:       0.713 [ms] (mean) 
     80Time per request:       0.036 [ms] (mean, across all concurrent requests) 
     81Transfer rate:          54747.93 [Kbytes/sec] received</pre> 
     82 
     83<p>Lighttpd:</p> 
     84<pre class="shell">Server Software:        lighttpd/1.4.19 
     85Server Hostname:        10.0.0.102 
     86Server Port:            80 
     87 
     88Document Path:          /index.html 
     89Document Length:        1795 bytes 
     90 
     91Concurrency Level:      20 
     92Time taken for tests:   21.248000 seconds 
     93Complete requests:      500000 
     94Failed requests:        0 
     95Write errors:           0 
     96Keep-Alive requests:    470598 
     97Total transferred:      991856958 bytes 
     98HTML transferred:       897503590 bytes 
     99Requests per second:    23531.63 [#/sec] (mean) 
     100Time per request:       0.850 [ms] (mean) 
     101Time per request:       0.042 [ms] (mean, across all concurrent requests) 
     102Transfer rate:          45585.94 [Kbytes/sec] received</pre> 
     103 
     104<p>NginX:</p> 
     105<pre class="shell">Server Software:        nginx/0.5.33 
     106Server Hostname:        10.0.0.102 
     107Server Port:            80 
     108 
     109Document Path:          /index.html 
     110Document Length:        1795 bytes 
     111 
     112Concurrency Level:      20 
     113Time taken for tests:   23.741872 seconds 
     114Complete requests:      500000 
     115Failed requests:        0 
     116Write errors:           0 
     117Keep-Alive requests:    500000 
     118Total transferred:      1006000217 bytes 
     119HTML transferred:       897500000 bytes 
     120Requests per second:    21059.84 [#/sec] (mean) 
     121Time per request:       0.950 [ms] (mean) 
     122Time per request:       0.047 [ms] (mean, across all concurrent requests) 
     123Transfer rate:          41379.30 [Kbytes/sec] received</pre> 
     124 
     125<p>Apache2.2:</p> 
     126<pre class="shell">Server Software:        Apache/2.2.8 
     127Server Hostname:        10.0.0.102 
     128Server Port:            80 
     129 
     130Document Path:          /index.html 
     131Document Length:        1795 bytes 
     132 
     133Concurrency Level:      20 
     134Time taken for tests:   35.438605 seconds 
     135Complete requests:      500000 
     136Failed requests:        0 
     137Write errors:           0 
     138Keep-Alive requests:    495064 
     139Total transferred:      1043777896 bytes 
     140HTML transferred:       897500000 bytes 
     141Requests per second:    14108.91 [#/sec] (mean) 
     142Time per request:       1.418 [ms] (mean) 
     143Time per request:       0.071 [ms] (mean, across all concurrent requests) 
     144Transfer rate:          28762.81 [Kbytes/sec] received</pre> 
     145 
     146<p>For the record: I did my best configuring all the servers in the very same way. In all the cases I removed unnecessary rules that could have slowed down the server (checks for htpasswd files and so on). And all the binaries came from the Debian repository, except for Cherokee 0.8.1 that hasn't been packaged yet.</p> 
     147 
     148<p>Anyway, this benchmark has been just a quick test. It is not certainly representing the result that these servers would have handling real traffic though. So, in the following days I will try to do a new a more accurate benchmark with static and dynamic content, compression, redirections, etc.  I'm pretty sure the results will be even better.</p> 
     149</div> 
     150 
     151 
     152 
     153 
     154 
     155<!-- Benchmark marker --> 
     156<hr> 
     157 
    44158<h2>Cherokee 0.6</h2> 
    45159<p>This benchmark was performed by <a href="http://www.ajo.es">Miguel Angel Ajo Pelayo</a> with Django running on Cherokee 0.6.0 and Apache 2.0.40. It shows Cherokee as a clear winner serving dynamic content. Specifically three times as much! And you haven't yet seen static results. The advantage is a lot bigger.</p> 
    …… 
    152266</pre> 
    153267 
    154 <h3>Other 0.6 benchmarks</h3> 
    155 <ul> 
    156 <li><a href="http://code.google.com/p/cherokee/wiki/CherokeeBenchmarkBrian">Cherokee + Apache + Lighttpd Benchmark</a></li> 
    157 </ul> 
     268 
     269 
     270 
     271<!-- Benchmark marker --> 
     272<hr> 
     273 
     274<h2>Cherokee + Apache + Lighttpd Benchmark</h2> 
     275<p>This benchmark was performed by <a href="http://oebfare.com">Brian Rosner</a> with Cherokee 0.6.0 beta2.</p> 
     276<h3>Software</h3><ul><li>cherokee 0.6.0 beta2 </li><li>apache 2.0.59 </li><li>lighttpd 1.4.16 </li></ul><h2>Hardware</h2><ul><li>733 MHz PIII </li><li>256 MB RAM </li><li>80GB 7200RPM IDE HD </li><li>Debian GNU/Linux 4.0 </li></ul><h2>Background</h2><p>I installed a fresh installation of Debian on the server hardware. Right after you login you will need to get sudo to perform root commands from your account: </p><pre class="shell">su 
     277apt-get install sudo</pre><p>Then add yourself to the <tt>/etc/sudoers</tt> file by running <tt>visudo</tt> and adding yourself in the user section. I just followed the root entry as this does not need to be a very secure server since it will not be running publicly. Now make sure you get back to your account and do: </p><pre class="shell">sudo apt-get install gcc make automake autoconf libtool 
     278mkdir src ; cd src 
     279sudo mkdir /usr/local/cherokee 
     280sudo mkdir /usr/local/lighttpd</pre><p>The installed version of gcc is 4.1.2 </p><h2>Cherokee Setup Details</h2><p>The following is what I executed to build Cherokee: </p><pre class="shell">wget http://www.cherokee-project.com/download/0.6/0.6.0/cherokee-0.6.0b863.tar.gz 
     281tar zxvf cherokee-0.6.0b863.tar.gz 
     282cd cherokee-0.6.0b863 
     283./configure --prefix=/usr/local/cherokee/0.6.0b863 
     284make 
     285sudo make install</pre><p>Here is the configuration for cherokee: </p><pre class="shell">server!port = 80 
     286server!timeout = 60 
     287server!keepalive = 1 
     288server!keepalive_max_requests = 500 
     289server!pid_file = /var/run/cherokee.pid 
     290server!server_tokens = full 
     291server!encoder!gzip!allow = html,html,txt 
     292server!panic_action = /usr/local/cherokee/0.6.0b863/bin/cherokee-panic 
     293server!mime_files = /usr/local/cherokee/0.6.0b863/etc/cherokee/mime.types 
     294 
     295vserver!default!document_root = /usr/local/cherokee/0.6.0b863/var/www 
     296vserver!default!directory_index = index.html 
     297 
     298vserver!default!directory!/!handler = common 
     299vserver!default!directory!/!handler!iocache = 1 
     300vserver!default!directory!/!priority = 1</pre><p>To run the web server I used: </p><pre class="shell">cd /usr/local/cherokee/0.6.0b863 
     301sudo sbin/cherokee -C etc/cherokee/cherokee.conf</pre><h2>Apache Setup Details</h2><p>The following is what I executed to build Apache: </p><pre class="shell">wget http://apache.oregonstate.edu/httpd/httpd-2.0.59.tar.gz 
     302tar zxvf httpd-2.0.59.tar.gz 
     303cd httpd-2.0.59 
     304./configure --prefix=/usr/local/apache/2.0.59 
     305make 
     306sudo make install</pre><p>I used the supplied <tt>highperformance.conf</tt> configuration file. I started the server with: </p><pre class="shell">cd /usr/local/apache/2.0.59 
     307sudo bin/httpd -k start -f conf/highperformance.conf</pre><p>The server ran using prefork. </p><h2>Lighttpd Setup Details</h2><p>The following is what I executed to build lighttpd: </p><pre class="shell">wget http://www.lighttpd.net/download/lighttpd-1.4.16.tar.gz 
     308tar zxvf lighttpd-1.4.16.tar.gz 
     309cd lighttpd-1.4.16 
     310./configure --prefix=/usr/local/lighttpd/1.4.16 
     311make 
     312sudo make install</pre><p>The configuration I used looked like this: </p><pre class="shell">server.modules = ( 
     313    &quot;mod_access&quot;, 
     314    &quot;mod_accesslog&quot; 
     315 
     316) 
     317 
     318server.document-root = &quot;/var/www&quot; 
     319 
     320mimetype.assign = ( 
     321    &quot;.html&quot; =&gt; &quot;text/html&quot;, 
     322    &quot;.txt&quot; =&gt; &quot;text/plain&quot; 
     323 
     324)</pre><p>I started the server with: </p><pre class="shell">cd /usr/local/lighttpd/1.4.16 
     325sudo sbin/lighttpd -f sbin/lighttpd.conf</pre><h2>Benchmark</h2><p>I will perform several different benchmarks on each webserver. This is to help gauge what type of performance each server can handle in the different conditions. Each test will have SSL turned on and turned off. </p><h3>small static file test</h3><ul><li>filesize: 99 bytes </li><li>command: <tt>ab -c 2 -t 2 -k http://localhost/index0.html</tt> </li></ul><h3>large static file test</h3><ul><li>filesize: 1.5MB </li><li>command: <tt>ab -c 2 -t 2 -k http://localhost/static.txt</tt> </li></ul><h2>Results</h2><p>I have included cherokee with both iocaching on and off. The out of the box setting is that iocache is turned on. </p><h3>small static file test w/ keepalive</h3><ul><li>cherokee 0.6.0b863 w/ iocache - 7816 reqs./sec. </li><li>cherokee 0.6.0b863 w/o iocache - 5761 reqs./sec. </li><li>lighttpd 1.4.16 - 4884 reqs./sec. </li><li>apache 2.0.59 - 2924 reqs./sec. </li></ul><p><img src="http://www.cherokee-project.com/images/benchmark1.png" /> </p><h3>small static file test w/o keepalive</h3><ul><li>cherokee 0.6.0b863 w/ iocache - 2182 reqs./sec. </li><li>cherokee 0.6.0b863 w/o iocache - 1874 reqs./sec. </li><li>lighttpd 1.4.16 - 2255 reqs./sec. </li><li>apache 2.0.59 - 1250 reqs./sec. </li></ul><h3>large static file test w/ keepalive</h3><ul><li>cherokee 0.6.0b863 w/ iocache - 108 reqs./sec. </li><li>cherokee 0.6.0b863 w/o iocache - 107 reqs./sec. </li><li>lighttpd 1.4.16 - 106 reqs./sec. </li><li>apache 2.0.59 - 94 reqs./sec. </li></ul><p><img src="http://www.cherokee-project.com/images/benchmark2.png" /> </p><h3>large static file test w/o keepalive</h3><ul><li>cherokee 0.6.0b863 w/ iocache - 88 reqs./sec. </li><li>cherokee 0.6.0b863 w/o iocache - 88 reqs./sec. </li><li>lighttpd 1.4.16 - 92 reqs./sec. </li><li>apache 2.0.59 - 118 reqs./sec. </li></ul><h2>Conclusion</h2><p>TODO: write something here </p><p>If you feel I need to supply any more detail about any part of this process, please let me know. I want this test to be as accurate as possible. I also would like to continue using this process to test future releases of each software and ultimately help improve them. I am a Cherokee developer, but if I notice anything that would help other webservers I am more than happy to let that community know. </p> 
     326 
     327 
     328 
     329 
     330 
     331 
     332<!-- Benchmark marker --> 
     333<hr> 
     334 
    158335 
    159336 
    …… 
    280457</pre> 
    281458 
     459<!-- Benchmark marker --> 
     460 
    282461<h3>Other 0.5.4 benchmarks:</h3> 
    283462<ul> 
    …… 
    285464<li><a href="http://www.rkblog.rk.edu.pl/w/p/lighttpd-and-cherokee-benchmark/">Lighttpd and Cherokee Benchmark</a></li> 
    286465</ul> 
     466 
     467<!-- Benchmark marker --> 
    287468 
    288469<h2>Older benchmarks</h2>