CS526Home Video PhotoAlbum Grade Assignments Homeworks Q&A Research Projects News Exams

Web Server Performance Evaluation

• For purchasing or assessing reasons, we evaluate various web server configurations using web benchmarking programs.
• The benchmarking system typically consists of a testing system possibly with many machines to generate the requests (static or dynamic web requests), send over network to the target web server to be evaluated.
• See an example of test configuration suggested by SPECweb2009'
• BeSim (Backend Simulator): a lightweight and efficient Backend Simulator (BeSim) to stand in for the Application and Database server portion of the solution architecture
• PTDaemon: power/temperature daemon
• SUT: System Under Test.
• Web server performance metrics: throughput, response time, failure rate, and new power consumption (Green Computing) and sus (simultaneous user sessions) .
  • http://www.spec.org/web2009/docs/faq.html#what_is_metric
• Fair Comparison: depending what one evaluates and how benchmark is conducted (SPECweb2009 has specific rules)
  • Compare Apache with IIS: should use the same hardware, same OS, same network, same Storage, what else?... (based on your assumptions), on a virtual platform?
  • Compare LAMP (Linux?/Apache2.2.22/MySQL?/PHP?) with Microsoft Solutions (2008R2/IIS7.5/SQL2008/ASP.NET4.0): same platform.
  • Same workload mix.
  • Same? clients with same launch distribution
  • Compare web server cluster serving global requests (future session topics).
• The load imposed on the web server is called workload. The combination of static web pages of different sizes, or the distribution of static vs. dynamic web requests, SSL vs nonSSL, launched by the tesing system is called workload mix.
• The intention to a mixed workload is to mimic the real world traffic on the potential web server candidate.
• Repeating the traffic based on the logs of existing web server will be ideal but may take too long a time permitted. Therefore the artificial workloads are generated for benchmarks.
• Most benchmarking programs allow the user to specify the workload mix.
• Organizations such as Standard Performance Evaluation Corporation, establish, maintain and endorse a standardized set of relevant benchmarks that can be applied to the newest generation of high-performance computers. SPEC develops benchmark suites and also reviews and publishes submitted results from our member organizations and other benchmark licensees. Not free.

SPECweb2009

• The last webbenchmark suite distributed by
• http://www.spec.org/web2009/docs/SPECweb2009_Setup.pdf
• http://www.spec.org/web2009/results/

Apache Benchmark (ab)

• ab is a tool for benchmarking your Apache Hypertext Transfer Protocol (HTTP) server. It is designed to give you an impression of how your current Apache installation performs. This especially shows you how many requests per second your Apache installation is capable of serving. http://httpd.apache.org/docs/2.2/programs/ab.html
• For example, ab -c 20 -n 5000 http://viva.uccs.edu:8345/ > vivaRn.txt will create 20 client thread each try to launch 5000/20=250 requests? to the web server listening on port 8345 of viva.uccs.edu.
• The report looks like this

  This is ApacheBench, Version 2.3 <$Revision: 655654 $>
  Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
  Licensed to The Apache Software Foundation, http://www.apache.org/
  
  Benchmarking walrus.uccs.edu (be patient)
  
  
  Server Software:        Apache/2.2.21
  Server Hostname:        walrus.uccs.edu
  Server Port:            80
  
  Document Path:          /
  Document Length:        7334 bytes
  
  Concurrency Level:      20
  Time taken for tests:   0.602 seconds
  Complete requests:      5000
  Failed requests:        0
  Write errors:           0
  Total transferred:      38045212 bytes
  HTML transferred:       36684668 bytes
  Requests per second:    8300.60 [#/sec] (mean)
  Time per request:       2.409 [ms] (mean)
  Time per request:       0.120 [ms] (mean, across all concurrent requests)
  Transfer rate:          61679.32 [Kbytes/sec] received
  
  Connection Times (ms)
                min  mean[+/-sd] median   max
  Connect:        0    1   0.2      1       1
  Processing:     0    1   1.9      1      35
  Waiting:        0    1   1.8      1      34
  Total:          1    2   1.9      2      35
  
  Percentage of the requests served within a certain time (ms)
    50%      2
    66%      2
    75%      2
    80%      2
    90%      2
    95%      2
    98%      2
    99%      2
   100%     35 (longest request)
    

• Other reports related to hw2 include in http://gandalf.uccs.edu/~cs526/cs526/wsbenchmark/
• To avoid the impact of system transient behavior, we run the benchmark several time.

Old WebStone 2.5

The Benchmark for Web Servers

WebStone Forum

We have created a Web-based forum to replace the WebStone mailing list. We did this to reduce the spamming that the list attracted and to provide a more useful way for the WebStone community to interact. Let us know what you think. ThisWebStone forum is available now! The old email list will go away on March 1, 2002.

WebStone 2.5

Both executable and source code for WebStone 2.5 are available for free and will stay that way. The highlights of the new release are:

• It's much more reliable.
• Tests are reproducible. The previous version of WebStone used a time of day random seed. Now it can optionally use a fixed random seed.
• WebStone 2.5 comes with a new license designed to encourage you to contribute enhancements while protecting you and your company from liability.
• There is an optional new dynamic workload that simulates ad rotation.
• WebStone 2.5 results can be compared to WebStone 2.01 results (unless you use the new dynamic workload).

WebBench 5.0 overview

• WebBench is a licensed PC Magazine benchmark program that measures the performance of Web servers. VeriTest developed WebBench.
• WebBench uses PC/Windows clients to send requests to the server for static files that you place on the server when you install WebBench or for a combination of static files and dynamic executables that run in order to produce the data the server returns to the client
• These clients simulate Web browsers. When the server replies to a client request, the client records information such as how long the server took and how much data it returned and then sends a new request. When the test ends, WebBench calculates two overall server scores
  • requests per second and
  • throughput in bytes per second
  as well as individual client scores.
• You can view all these results from the WebBench controller.
• You can use WebBench:
  • To measure the performance of different Web server software packages by running WebBench against them on the same server hardware.
  • To gauge how well different hardware systems perform as Web servers by running a given Web server package and WebBench on those systems.
• WebBench's easy-to-install controller and client programs, which include ready-to-run test suites, mean you can quickly get started with your benchmark testing and produce valid test results.
• WebBench provides templates for e-commerce and dynamic test suites. The e-commerce test suite template lets you test the secure transaction features of Web servers (as long as you have SSL configured on your Web server). The dynamic test suite template is similar to the e-commerce test suite template but it does not use SSL.
• There are other Web Server Benchmarking program such as specweb99. But it is $500 for education price.
• SGI developed a WebStone program which has similar architecture as WebBench. The controller utilizes unix rsh to coordinates several Unix clients to measure the performance of the web servers.
• We will use WebBench to measure the performance of different web servers (Apache, tomcat, IIS) and web server cluster.