SPECweb2009 Release 1.20 User's Guide

Version 1.20, Last modified 2010-02-05


1��������� Requirements

1.1������ Software Requirements

1.1.1����������� The SPECweb2009 kit

1.1.2����������� Additional Software Requirements:

1.2������ Hardware Requirements

2��������� Setting Up The Test Environment

2.1������ Running the SPECweb2009 installer

2.2������ Web Server Setup

2.3������ Besim Setup

2.4������ Client Setup

2.5������ Network Setup

2.6������ Measurement Device (Power and Temperature) Setup

2.7������ Controller Setup

2.8������ Power Analyzer Setup

2.9������ Temperature Sensor Setup

3��������� Running SPECweb2009

3.1������ Test.config Parameters

3.2������ Workload Specific Config Parameters

3.3������ Starting the benchmark

3.4������ Trouble Shooting your first run

3.4.1����������� Web Server Checks

3.4.2����������� BeSim Checks

3.4.3����������� Prime Client Checks

3.5������ Collecting Statistics (optional)

3.6������ Optimizing Performance

4��������� Appendices

4.1������ Appendix A - README file from SPECweb2009 client kit

4.2������ Appendix B - Wafgen README file

4.2.1����������� Invoking Wafgen:

4.2.2����������� Wafgen Overview

4.2.3����������� Storage Requirements:

4.2.4����������� Tips for Building Large Filesets:

4.3������ Appendix C - BeSim Make_Readme file

4.4������ Appendix D - Common Errors and Debugging Tips

4.5������ Appendix E - Support FAQ

4.6������ Appendix F - Installing BeSim as a FastCGI for the Apache HTTP Server

4.6.1����������� FastCGI source compilation

4.6.2����������� Apache FastCGI Web server module: mod_fastcgi

4.6.3����������� Testing the BeSim FastCGI


1������ Requirements

��� These instructions assume that you are familiar with the following:

2 ����� Setting Up The Test Environment

  1. Set localization on the server system to US.

  2. Install and configure your favorite Web server with SSL and PHP, ASP.NET, or JSP support configured in. If using JSP, a separate Java application server may be needed. Check your Web server software documentation for assistance.

  3. Run the SPECweb2009 installer (see above) and set your current working directory to the wafgen directory.
    # pwd /web2009-install/wafgen

Depending on the type of scripts you want to use choose one of the following options:

2.3���� Besim Setup

  1. Configure the BeSim system with Web server software installed on port 81 or another port (avoid port 80 to avoid confusion with the primary Web server).

  2. Run the SPECweb2009 installer, do a "BeSim" or "Full" installation, and change your current working directory to the <specweb_install_directory>/Besim directory.

  3. Review the BeSim Make_Readme
    NOTE: For Windows/IIS, a precompiled ISAPI is provided (BeSim.dll); copy this to a directory with ISAPI execute permissions, and skip to step 6.

  4. You may choose from FastCGI* for Zeus or Apache or ISAPI for Zeus or IIS or NSAPI for SUN One and edit the appropriate
    MakeIncl.<type>.<OS> file to reflect your compile and linker options and select one or more Defines.� Then you can build the BeSim code for your selected API. (If using FastCGI , please copy the FastCGI executable from the Redistributed Open Source code directory into the BeSim directory.)

  5. Install this code to your BeSim Web server.

  6. Start your BeSim Web server.

  7. Use the test_besim_bank, test_besim_ecom, and test_besim_support scripts (included in the "besim" subdirectory of the installation) to test whether you're getting valid BeSim responses.� Both Perl scripts and Linux bash scripts are provided.� The Perl scripts require a Perl interpreter as well as the following Perl modules: URI, HTML-Tagset, HTML-Parser, and libwww-perl.� The bash scripts require cURL to be in the path.

    Invoke these scripts with the URL to your compiled BeSim API.

��� ��� ��� ��� *Note: For more FastCGI instructions (particularly for the Apache HTTP Server), see Appendix F.

If you notice that one BeSim system has a bottleneck (CPU, disk, network, etc.), there is an option to use multiple physical BeSim systems.� Follow the same steps above to install BeSim on another system, and specify multiple BESIM_SERVERs (delimited by spaces) in Test.config.� See the example Test.config files for more information.
  1. Install a 1.6 JVM (or later) on all client systems.

  2. Run the SPECweb2009 installer per Section 2.1 on each client and set your current directory to the SPECweb2009 directory created by the installation.

  3. Select one of the client systems to serve as the "Prime Client" (can also be a load generating client).

  4. On the Prime Client, decide on an appropriate set of configuration files for your particular system.� For a Red Hat Linux server running Apache and PHP, you should use the *.Unix-PHP.config files.� Copy or rename these files to SPECweb_Banking.config, SPECweb_Ecommerce.config, SPECweb_Support.config, SPECweb_Power.config and Test.config, i.e.

# cp SPECweb_Banking.Unix-PHP.config SPECweb_Banking.config
# cp SPECweb_Ecommerce.Unix-PHP.config SPECweb_Ecommerce.config
# cp SPECweb_Support.Unix-PHP.config SPECweb_Support.config
# cp SPECweb_Power.Unix-PHP.config SPECweb_Power.config
# cp Test.Unix-PHP.config Test.config

For JSP, follow the same steps but use *Unix- JSP.config files instead . Similarly, for ASP.NET, follow the same steps using * ASPX.config files instead .

  1. Review the SPECweb_<workload>.<OS/Web server>-<Script Engine>.config, Test.<OS/Web server>-<Script Engine>.config, and Testbed.config to understand the parameters in these files, with attention to those in the beginning (configurable) section of the workload and Test configuration files which may need to be updated to match your test bed.

  2. Edit configuration files to match your Web server, BeSim configuration, Client, and network configuration.� Change only those parameters listed above the section marked "# Fixed workload properties" or in Test.config above the line marked " # SPECweb2009 fixed test properties":

Some tips:

2.5���� Network Setup

Configure all the test bed systems so that they can communicate with one another; make sure to update your /etc/hosts file (or equivalent). The Prime Client must be able to communicate with each Client, Besim and each Controller in the test environment. The Web server and the BeSim system must be able to communicate with each other.

2.6���� Measurement Device (Power and Temperature) Setup

Set up the analyzer/analyzers according to the instructions in the Power and Temperature�Hardware Setup Guide. Note that these measurement devices need to be set up for both the SUT System as well as the Storage as per the requirements outlined in the SPECweb2009 Run and Reporting Rules.

2.7���� Controller Setup

  1. PTDaemon version 1.4.0 is distributed with the kit. The latest version of PTDaemon can be downloaded from the Accepted Measurement Devices page.

  2. PTDaemon should install into <PTD_Home> when installing from the kit. If installing a newer version, copy PTDaemon into <PTD_Home>.

  3. Install an analyzer or a temperature sensor to the controller's interface (such as USB, Serial port etc).
  4. Start the PTDaemon using the PTDaemon script. Use ptd �h to get a brief help. �An example of the command is

<PTD_home>/ptd �p 1920 -l mylogfile 8 /dev/term/a

In the example above, the power daemon is communicating with the Yokogawa WT210, a type 8 analyzer, via /dev/term/a interface and communicates with the primary client via port 1920. All messages of the power analyzer are logged in mylogfile (logging power analyzer messages is not a requirement, so the �l option may be omitted).

For further details on hardware installation, please see the Power and Temperature Hardware Setup Guide.

2.8���� Power Analyzer Setup

Please refer to�Section 2 of the SPECweb2009 Run and Reporting Rules for the requirement of power measurement and consult the manufacturer's documentation for instructions that are specific to the exact analyzer used.�� Once you have correctly associated the device number with the corresponding integer values that apply to those devices,�follow�the steps listed below:

  1. Connect the communication interface of the power analyzer to the appropriate port of the controller server with a manufacturer-approved cable.

  2. Open runpower.bat for Windows or runpower.sh for Unix with a text editor, and locate the set DEVICE= for the runpower script respectively. Enter the correct device number for the power analyzer being used.

    set
    DEVICE=8 (for Windows & Unix)

    Now enter the DEVICE_PORT that the PTD uses to communicate with Controller system.


    set
    DEVICE_PORT=COM1 (for Windows)

    set
    DEVICE_PORT=/dev/tty2 (for Unix)

    To verify what port the power meter is using (for Windows) you may left click my computer then click �Manage�. Click �Device Manager� and then click �Ports� this should give you a list of all the ports attached to your computer at that time. Verify the port the power analyzer is connected to.

  1. Set power analyzer communication settings as required by the Power and Temperature Hardware Setup Guide.

  2. Connect the SUT's AC power cord to the power receptacle provided by the power analyzer.

2.9���� Temperature Sensor Setup

Please refer to�Section 2 of the SPECweb2009 Run and Reporting Rules for the requirement of temperature measurement and consult the manufacturer's documentation for instructions that are specific to the exact sensor used.�� Then follow�the steps listed below:

  1. Place the temperature sensor in accordance with the rules outlined in this section.

  2. Connect the temperature sensor's communication interface to the appropriate port on the controller server with a manufacturer-approved cable.

  3. Open runtemp.bat for Windows or runtemp.sh for Unix with a text editor, and locate the set DEVICE= for the runtemp script respectively. Enter the correct device number for the temperature sensor being used.

    set
    DEVICE=1001 (for Windows & Unix)

    Also locate the set DEVICE_PORT= for the runtemp script respectively. Enter the correct port number used to connect the temperature sensor.


    set
    DEVICE_PORT=USB (for Windows)

    set
    DEVICE_PORT=/dev/tty1 (for Unix)
To verify what port the temperature sensor is using (for Windows) you may left-click my computer, click Manage. Next click Device Manager and then click Ports this should give you a list of all the ports attached to your computer at that time. Find the port that the temperature sensor is connected to.

3 ����� Running SPECweb2009

  1. Start the specwebclient process on each client machine. Execute the following command in the directory where the client was installed, or run the start_client script provided in the client directory for more details.
    �������
    java-Xms512m -Xmx512m -jar specwebclient.jar

  2. Start the Power and temperature daemons. Please refer to the runpower and runtemp script files in the PTDaemon directory for more details.

  3. To start the test harness manager process on the prime client begin the test, execute the following command in the prime client directory, or modify the provided start_prime_client scripts.

    Windows:

    �������
    java-Xms512m -Xmx512m -cp .;lib\jcommon-1.0.15.jar;lib\jfreechart-1.0.12.jar;bin\specweb.jar specweb

    Linux:

    �������
    java-Xms512m -Xmx512m -cp .:lib/jcommon-1.0.15.jar:lib/jfreechart-1.0.12.jar:bin/specweb.jar specweb

After test has completed check the redirected output and the contents of the results. Please check whether there are power measurements in the output of the primary client as well.

Note: you may wish to set other Java parameters besides the heap parameters to optimize the specwebclient processes as you increase the test load.� Check the documentation for your JVM.

Did that test work?

Yes? Great, so did these instructions "work" or were there missing details or do you have additional questions that you are now writing up to send to mailto:[email protected]?

No? Well, there should be enough information in the debug output to start debugging. Please send any corrections or requests for clarification to these instructions to the email address above.� Continue reading and reviewing the Web server,�BeSim, and Client Checks below, along with the related README files for the various components in the attached appendixes, particularly Appendix D for common errors and Appendix E for the Support FAQ.

Now, that the basic settings are functional, you can try tests that use more Sessions and adding more clients.

3.4���� Trouble Shooting your first run

3.4.1��������������� Web Server Checks

  1. Have you configured your Web server to support the HTTP or HTTPS based on the workload you are running?� Banking is all HTTPS (SSL), Ecommerce requires both HTTP and HTTPS, and Support uses only HTTP.� Note: that you can temporarily override the use of HTTPS in Banking and Ecommerce by setting USE_SSL = 0 in the .config file.� Results will be noncompliant but this may be useful when debugging.

  2. Have you configured you Web server to support PHP, JSP or ASP.NET according to the directions provided with your Web server?� If using JSP and a separate Java application server such as Tomcat also review its documentation and configure appropriately.� Try running the PHP, JSP or ASP.NET "Hello World" equivalent to make sure your Web server's script engine is working.

  3. Have you configured the wafgen.rc files for the workload and run Wafgen to create the fixed and scaling file sets? Did you set the number of SIMULTANEOUS_SESSIONS to a value equal or greater than the value you plan to use in your testing?

  4. Can the Web server system ping the BeSim system?� Connect to the BeSim Web server (telnet [BESIM_IP] [BESIM_port])?

3.4.2��������������� BeSim Checks

3.4.3��������������� Prime Client Checks

3.5���� Collecting Statistics (optional)

The prime client has the ability to poll clients for data at specified intervals during a run and display it as CSV or in a GUI.� Data is collected and displayed only during the warm-up and benchmark run phases.� The following configuration parameters in Test.config control the collection and display of this information:

3.6���� Optimizing Performance

Once the SPECweb2009 benchmark is running the workload(s) without errors, you will want to optimize your testbed for optimal performance.� Here are some tips:


4 ����� Appendices