• Learn how to configure apache as
  • web server.
  • cache server
• Learn how to use ab to evaluate the web server performance.

Assignment Date: 1/30/2012
Due Day: 2/8/2012
Related documents:

• Apache httpd version 2.2 document: http://httpd.apache.org/docs/2.2/
• Content cache: http://httpd.apache.org/docs/2.2/caching.html

Description:

Logistics:

• I have created your account on gandalf, viva, walrus, wins.csnet, and sam.csnet for this exercise and next on to form a cluster. I also copy the apache directory and setup.pl from ~chow/public_html/cs526 to your cs526 directory. If you garbled the content, feel free to download it again.
  
  

  Machine Name	Machine Type	Procssor spec	Main Memory	Network Speed	OS	Web sites to run
  viva	optiplex 620	duo core 3.2GHz	4GB	1Gbps	Fedora Core 16	ws1
  walrus	optiplex 960	quad core Q9550 2.83GHz	8GB	1Gbps	Fedora Core 16	ws2/caching
  wins.csnet	VM; host is willow.uccs.edu optiplex 990 Win7	vCPU with VBOX on Core i7 3.4 GHz	1GB	1Gbps	Fedora core 16	ws3
  gandalf	optiplex 960	quad core Q9550 2.83GHz	8GB	1Gbps	Fedora Core 16	testing
  willow	optiplex 990	i7 Q9550 2.83GHz	8GB	1Gbps	Win7	IIS7
  sam.csnet	vm on Dell server in EAS Cluster	Nehalem Gen. Xeon Core i7 core 2.66GHz	1GB	1Gbps	Fedora Core 16	ws4/reverseproxy

• Under apache directory there are five root directories (ws1, ws2, ws3, proxy, reverseproxy) for the five server configurations.
• To reduce your work:-) and avoid typing mistakes, I create a set up script setup.pl for change the apache server configuration to reflect your login and use 8 or 9 followed by the last 3 digits of your SID as port number. Just run "./setup.pl <yourlogin> <last3DigitsSID>" in cs526 directory to configure those sites.
  • For James Logan with jlogan as login and 345 as last three digits of SID, run "./setup.pl jlogan 345".
• Note that these five machines do not share NFS for a common home directory.
  • Since we are measuring the web server performance and do not want NFS acccess across network as a bottlenck.
  • You will have to run the setup.pl on each machine to configure the apache site.
  • I do rsync between walrus (master) and gandalf (slave) now. What will be the problem?
• If you gobble your apache directory, you can copy a fresh copy from that of chow with "cp -r ~chow/public_html/cs526/apache ."
• Use the cmd command to run the server. It will be idle there waiting to processing the requests. You use "./cmd &" to put it in background and use fg to bring back.
• Make sure your kill the process after your exercise.
  • The process id of web server will be saved by httpd in logs/httpd.pid.
  • You can also use ps aux | grep <login> to find out the process id.
  • Or you can use "lsof -i :8<last3DigitsSID>" to find out the process id.
  • Use "kill -TERM <pid>" to kill the process.
  • You can also use control-c or kill -9 <pid> to kill the httpd parent process. What are the differences?
    See the Q&A explanation at the buttom of the web page.
    • Use "kill -TERM <pid>" or "kill -15 <pid>" to inform httpd parent process to immediately attempt to kill off all of its chidren then exit.
    • There is a more gentle termination called "Graceful Stop" which allows children to exit after their current request is done (exit if idle). "kill -WINCH <pid>". It may take a while before you can run cmd again. It may report the "socket adres already in use."
    • For graceful restart (after apache update or configuration setting changes), use "kill -USR1 <pid>"
    • See more details in "apache stopping/restarting" and UNIX signal we page.
• After your exercise, please run the ipcs -s to see the semaphore arrays created. Too many of them not release will hang the system.
  Make sure you use " ipcs -s | grep `whoami` | perl -e 'while (<STDIN>) {@a=split(/\s+/); print `ipcrm sem $a[1]`}' to remove the semaphores created by you.
• We will reboot the systems 3am to clean out dangling zambies. But let me know if the system is very slow. Could be caused by a lot users running.

• Change the program of the cmd shell script in each of the directory to /home/cs526/public_html/httpd/httpd-2.2.21/bin/httpd (from 2.2.17 or 2.2.21)
• Make sure you check the parameters modified by the setup.pl. No perfect code here. From time to time, I still find bugs on my scripts:-)

Part 1. Configure apache to run as a web server

• In this exercise, we will learn how to configure apache httpd to run as a web server.
• In the following description, we assume jlogan is your account; the last three digits of your SID is 345.
• Read man page with "man httpd" for more info. The related http-2.2 version documents are available at http://httpd.apache.org/docs/2.2
  

  SYNOPSIS
  httpd [ -X ] [ -R libexecdir ] [ -d serverroot ] [ -f con-fig ] [ -C directive ] [ -c directive ] [ -D parameter ]
  

• If "/home/cs526/public_html/httpd/httpd-2.2.21/bin/httpd -d /home/cs526/public_html/cs526/apache/ws1 -X" is run, the httpd program will treat /home/cs526/public_html/cs526/apache/ws1 as serverroot directory and read the directives in the conf/httpd.conf file for configuring the apache server. -X is for debugging purpose, it indicates to the httpd daemon that do not fork spare child process or create threads for improving performance. Just run a single standalone process.
• The Apache provides a rich set of directives for configuring/customizing the web server. A subset of directives are realized by an Apache module. See http://httpd.apache.org/docs/2.2/mod/directives.html for the set of runtime directives.
• An example of httpd.conf is
  
  LoadModule log_config_module /etc/httpd/modules/mod_log_config.so
  LoadModule access_module /etc/httpd/modules/mod_access.so

  User jlogan
  ServerName walrus.uccs.edu
  Listen 8345

  DocumentRoot /home/jlogan/public_html/cs526/apache/ws1/htdocs

  ErrorLog logs/error_log
  TransferLog logs/access_log
  PidFile logs/httpd.pid

• Note that in your ws1/conf/htttpd.conf, there are no LoadModule directives. The reason is that the

  /home/cs526/public_html/httpd/httpd-2.2.21/bin/httpd

  we use is compiled with access and log_conifig module compiled in (built-in). See the extensive examples in /etc/httpd/conf/httpd.conf starting line 149.
• The first directive LoadModule tells the httpd program to dynamically load in /etc/httpd/modules/mod_log_config.so as dynamic shared object (very much like dll). The log_module code deals with creating error log entries in the log files.
• Access_module deals with creating access log entries in the log files.
• User directives tells httpd to run itself as jlogan account.
• ServerName tells httpd the domain to use when displaying error msg.
• Listen directive tells httpd to listen to port 8345 for incoming http requests. We need to enter http://walrus.uccs.edu:8345/ to access this webserver.
• DocumentRoot specifies the actual path to the default web document directory.
• ErrorLog specifies the name of the file for saving error log records.
• TransferLog specifies the name of the file for saving the access records.
• PidFile specifies the names fo the files for saving the process id of the httpd server. It can be used by system admin to restart or stop the httpd server. Note that the CSNET Unix machines are currently running httpd 2.2.21 (/usr/sbin/httpd) and the modules are located in /etc/httpd/module.
  
  You can try to run "telnet walrus.uccs.edu 80" and type "GET / HTTP/1.0 <enter><enter>" to find out the server version:
  

  [root@gandalf htdocs]# telnet walrus.uccs.edu 80
  Trying 128.198.60.197...
  Connected to walrus.uccs.edu.
  Escape character is '^]'.
  GET / HTTP/1.0
  
  HTTP/1.1 200 OK
  Date: Thu, 26 Jan 2012 00:49:07 GMT
  Server: Apache/2.2.21 (Fedora)
  Last-Modified: Sat, 31 Dec 2011 22:56:12 GMT
  ETag: "1a0185-1ca6-4b56b418dfc77"
  Accept-Ranges: bytes
  Content-Length: 7334
  Connection: close
  Content-Type: text/html; charset=UTF-8
  
    

Part1. Exercise1: Configure a simple apache server (ws1)

• Login to viva (use SSH)
  
• cd public_html/cs526
• run "./setup.pl jlogan 345" to configure ws1 site. Replace jlogan with your login and 345 with the last three digits of your SID.
• cd apache/ws1 and make sure
  • in the cmd, conf/httpd.conf, htdocs directory files, all instances of LOGIN are replaced with your own login name and
• in the conf/httpd.conf file, the port number with 8<last 3 digits of your SID> in the Listen directive are replaced with the last 3 digits of your SID.
• Make sure you have mime.types in your conf directory (new apache2 requirement).
• Run the apache web server with "./cmd "
  
• Go to any browser and enter "http://viva.uccs.edu:8<last 3 digits of your SID>", e.g., http://viva.uccs.edu:8345/, and see if the web page shows up.
• Check the logs in ws1/logs/access_log and see the access records is created. You can also type the url http://viva.uccs.edu/~jlogan/cs526/apache/ws1/logs/access_log (replace jlogan with your login name).
• 
  

Part1. Exercise2: Setup web server 2 on walrus

• Createa another SSH terminal window and login to walrus.
• cd public_html/cs526
• run "./setup.pl jlogan 345" to configure apache site. Replace jlogan with your login and 345 with the last three digits of your SID.
• cd apache/ws2 and make sure
  • in the cmd, conf/httpd.conf, htdocs directory files, all instances of LOGIN are replaced with your own login name and
• in the conf/httpd.conf file, the port number with 8<last 3 digits of your SID> in the Listen directive are replaced with the last 3 digits of your SID.
• Make sure you have mime.types in your conf directory (new apache2 requirement).
• Run the apache web server with "./cmd "
  
• Go to any browser and enter "http://walrus.uccs.edu:8<last 3 digits of your SID>", e.g., http://walrus.uccs.edu:8345/, and see if the web page shows up.
• Check the logs in ws2/logs/access_log and see the access records is created. You can also type the url http://walrus.uccs.edu/~jlogan/cs526/apache/ws2/logs/access_log
• 
  

Part1. Exercise3: Setup web server 3 on wins.csnet.

• Createa another SSH terminal window and login to wins.csnet
• cd public_html/cs526
• run "./setup.pl jlogan 345" to configure apache site. Replace jlogan with your login and 345 with the last three digits of your SID.
• cd apache/ws3 and make sure
  • in the cmd, conf/httpd.conf, htdocs directory files, all instances of LOGIN are replaced with your own login name and
• in the conf/httpd.conf file, the port number with 8<last 3 digits of your SID> in the Listen directive are replaced with the last 3 digits of your SID.
• Make sure you have mime.types in your conf directory (new apache2 requirement).
• Run the apache web server with "./cmd "
  
• Go to any browser and enter "http://wins.csnet.uccs.edu:8<last 3 digits of your SID>", e.g., http://wins.csnet.uccs.edu:8345/, and see if the web page shows up.
• Check the logs in ws3/logs/access_log and see the access records is created. You can also type the url http://wins.csnet.uccs.edu/~jlogan/cs526/apache/ws3/logs/access_log

• login to walrus and cd to the proxy directory.
• check to make sure the cmd and conf/httpd.conf are modified correctly.
  
• Run the proxy cache server with "./cmd "
• Setup your browser to use the proxy server as follows:
  • On Internet Explorer 9,
    • Select Tool | Internet options...
    • Select Connections tab and click the LAN Settings button.
  • Check "Use a proxy server for your LAN" and enter "walrus.uccs.edu" as address and "9<last3digts of your SID>" for the port number. This is the port number your cache server is listening.
  • Click OK.
    
  • On firefox browser,
    • Select Tools | options
    • Click Advanced icon and Network tab, click Settings
    • Click "Manual proxy configuration"
    • Enter walrus.uccs.edu as HTTP Proxy text entery; and 9<the last 3 digits of SID> as port number
    • Click OK
  • On Google Chrome browser,
    • Select "tool icon"| Options menuitem,
      
      
    • Options window appears
      
      
  • 
  • Click "Changel proxy settings..."
    
  • Note that if you are trying at home with uccsvpn, then you should the "Settings" under the "Dial-up and Virtual Private Network settings". Not the one under "LAN settings". This misconfiguration is very tricky. It tookls me almost 1 1/2 hours to figure this out!
  • Check use a proxy server for your LAN. Enter walrus.uccs.edu for HTTP Proxy:
  • Enter 9<last 3digits of SID> for Port:
    
  • Note that the port number starts with 9. Click OK.
• Enter the url to your personal web page for CS526 in the address box of your IE browser.
• Locate your personal web page in the cache directory.
• Include the url of the cached docuement in your hw2 web page. It should have the following prefix http://walrus.uccs.edu/~<login>/cs526/apache/proxy/cache/ and indicate the first line and the HTTP header section of the cached file. Verify that the path to the cached document indeed contained the subdirectory name followed the specification of CacheDirLength directive.
• Try downloading a medium file such as http://mirrors.isc.org/pub/apache/httpd/httpd-2.2.22.tar.bz2 twice. The second download should be quicker due to caching.
• After the exercise, do not forget to uncheck the "Use a proxy server for your LAN" Internet option. Otherwise your browser (IE in this case) will not work. If you use firefox, select "No proxy" radio button option on the Connection Settings dialog window. Remove the large file(s) from cache afterwards.
  
• Question 1. Compare the access log file of cache server with that of ws2. What is difference between the url field of access record? It is longer. Why browsers send http requests with different formats for cache server vs. normal web server?
• Question 2. Cache server after receiving the http request, will check the local cache to see if there is a document with the same url. If there is a cached document, they will send a http request with If-Modified-Since header to the original web server. Where they get the value of the If-Modified-Since header?
• Question 3. From time to time, a client browser will also send the http request with the If-Modified Since header. When will that be?
• Question 4. Akami Edge Side Include tries to improve the above cache server processing. Explain how they achieve that.
• Question 5. Assume that we use default CacheDirLevel of 3 and default CacheDirLength of 1, then with the original url as file path name, all cached documents will be saved in the same directory. With the hash value of url as path name, the cached documents will spread out hopefully evenly among the subdirectories.
  • a. If the original url is used as file path name, how many filename matching operations are needed on average, given 4096 doucments?
  • b. If the original url is used as file path name, what are names of those three sudirectories?

Exercise 4. Evaluate Web Server Performance using ab.

Create a homework web page hw2.html in your cs526 personal web page directory