Configuring Pivotal Web Server Instances

Note: This product has been discontinued. Technical Guidance ends July 15th 2018.

The default configuration of a newly created Pivotal Web Server instance is fairly simple. Although the configuration is likely adequate for your needs, sometimes you might need to further configure the instance to enable one of its many useful features, such as load-balancing between two or more tc Runtime instances. This chapter provides some information to get you started.

For complete documentation on how to configure Pivotal Web Server instances, see Apache HTTP Server Version 2.2 Documentation. Because Pivotal Web Server is based on Apache HTTP server, the general configuration documentation on the Apache Web site applies to Pivotal Web Server as well.

Using Sample Configuration Files to Enable Features and Modify Configuration

All Pivotal Web Server instances include sample configuration files that you can use to enable extra features in the server instance or to modify its default configuration. These files are located in the INSTANCE-DIR/conf/extra directory, where INSTANCE-DIR refers to the instance directory, such as /opt/pivotal/webserver/myserver.

For example, the httpd-info.conf sample configuration file shows how you can get information about the requests being processed by the Pivotal Web Server instance as well as information about the configuration of the instance. The httpd-ssl.conf file shows how to provide SSL support. It contains the configuration directives to instruct the instance how to serve pages over an HTTPS connection.

For your convenience, the main Pivotal Web Server configuration file for a particular instance (INSTANCE-DIR/conf/httpd.conf) already includes commented-out lines for including each sample configuration file. For example, the line to include the httpd-info.conf configuration file is as follows:

#Include conf/extra/httpd-info.conf

To include the configuration file, simply uncomment the Include directive:

Include conf/extra/httpd-info.conf

You do not have to use Include in this way; you can simply copy and paste the information in a sample configuration file into the main configuration file.

The sample configuration files are full of comments on how exactly to enable the feature they configure. Be sure to read these comments before you proceed further.

What to do next

Restart the Pivotal Web Server instance for the configuration changes to take effect. For example, on Unix:

Configure Load Balancing Between Two or More tc Runtime Instances

You can configure a Pivotal Web Server instance to perform simple load balancing between two or more tc Server instances.

In the procedure that follows, you configure a Pivotal Web Server instance to run in front of the tc Runtime instances; this Pivotal Web Server instance receives all requests from users, and then passes them back to the tc Runtime instances using a specified load-balancing algorithm. Responses from the tc Runtime instances are then routed back through this same Pivotal Web Server instance. For this reason, the Pivotal Web Server instance acts like a proxy (both reverse and forward) so that the users never know the URLs of the backend tc Runtime instance that are actually doing all the work. Additionally, the Pivotal Web Server instance ensures that the load on each tc Runtime instance is balanced. You can specify that each tc Runtime instance take on an equal work load, or you can specify that one instance work twice as hard as the others.

In the procedure, the following scenario pertains. These assumptions are not requirements; your environment might be very different. The assumptions are listed only to make the procedure easier to understand.

Two tc Runtime instances are running at the following two hosts and port numbers:

http://192.168.0.203:8081

http://192.168.0.203:8082

The two tc Runtime instances are running on the same computer, are part of the same installation and their respective CATALINA_BASE variables are as follows:
- /var/opt/pivotal/pivotal-tc-server-standard/instanceOne
- /var/opt/pivotal/pivotal-tc-server-standard/instanceTwo

Each tc Runtime instance is configured exactly the same (other than the value of the various ports).

You have deployed the same application to both tc Runtime instances and the URL context is the same in both instances: /my-app.

You want all users of the application to first go through the front-end Pivotal Web Server instance, and any evidence of the backend tc Runtime instances upon which the application is actually deployed should be hidden from the user.

Pivotal Web Server is installed on a different computer than Pivotal tc Server. The name of the particular Pivotal Web Server instance is lb-server and its home directory is /opt/pivotal/webserver/lb-server.

You want to configure sticky sessions, which means that the Pivotal Web Server instance always routes the requests for a particular session to the same tc Runtime instance that serviced the first request for that session.

You want to use the HTTP protocol for all communication between the Pivotal Web Server and the tc Runtime instances.

The load balancing described in this procedure is very simple, although you have many options available to further customize it. At appropriate locations in the procedure, links to the Apache HTTP Server documentation are provided for additional configuration options not covered by this specific scenario. Adapt the procedure for your particular environment.

As part of the procedure, you update the configuration files of both the Pivotal Web Server instance and the two tc Runtime instances.

Prerequisites

Install Pivotal tc Server on the same or different computer as Pivotal Web Server, and create two more instances. Make note of the host and port numbers of the two instances. See the Pivotal tc Server documentation for details.

Deploy the same application to the two tc Runtime instances.

Procedure

To configure load balancing for the scenario described in the introduction to this section, follow these steps:

On the computer on which Pivotal Web Server is installed, stop the instance, if it is currently running. Following the example and assumptions:

prompt# cd /opt/pivotal/webserver/lb-server
prompt# bin/httpdctl stop

Open the httpd.conf configuration file of the Pivotal Web Server instance and ensure that the three required LoadModule directives (proxy_balancer_module, mod_proxy, and mod_proxy_http, are present and enabled (in other words, are not commented out):

The Pivotal Web Server configuration file is located in the conf directory of your Pivotal Web Server instance (/opt/pivotal/webserver/lb-server/conf in our example).

In the same httpd.conf file, add the proxy configuration.

Use the <Proxy> element to specify the list of tc Runtime instances and the method of load balancing you want to use. Then use the ProxyPass and ProxyPassReverse directives to specify the URLs that will use this proxy and load-balancing (both for requests and responses.) For example:

The balancer parameter of the <Proxy> element specifies a unique identifier for this load balancer configuration.

Each tc Runtime instance that is serviced by this load balancer must have its own BalancerMember; the first parameter of this directive specifies the full IP address (including port number) of the tc Runtime instance.

The route parameter contains session ID information. You later use the value of this parameter in the tc Runtime configuration file to configure sticky sessions; for now, just ensure that the values are unique for each BalancerMember.

The loadfactor parameter specifies how much load a particular member carries. If you want each member to carry the same load, set the numbers equal to each other (as in the example above). If, however, you want one member to work three times harder than the other, set the load factors to 3 and 1.

Use the lbmethod parameter of the ProxySet directive to specify the load balancing algorithm. The possible values are as follows:

byrequests: performs weighted request counting. This is the default value.

bytraffic: performs weighted traffic byte count balancing.

bybusyness: performs pending request balancing.

Use the ProxyPass and ProxyPassReverse to specify the context URLs of the application that will be routed to the tc Runtime instances that you have configured in the load balancing scheme. ProxyPass specifies that when the Pivotal Web Server instance receives a request at the /my-app URL, it routes the request to the load balancer that will in turn route it to the tc Runtime instance. ProxyPassReverse does the reverse: when the tc Runtime instance sends a response to a user who is using /my-app, the response appears to come from the Pivotal Web Server instance, and not the tc Runtime instance. Thus the details of the tc Runtime instance are hidden from the user.

Optional. If you want to enable the balancer manager Web application to watch the load balancing activity and control the behavior, add the following to the httpd.conf configuration file of your Pivotal Web Server instance:

<Location /balancer-manager>
SetHandler balancer-manager
Order Deny,Allow
Deny from all
# BE VERY RESTRICTIVE with YOUR ALLOW STATEMENT
Allow from 127.0.0.1
</Location>

Optional. If you want to enable sticky sessions, follow these steps:

In the httpd.conf file of the Pivotal Web Server instance, update the ProxySet directive of the <Proxy> element you configured in a preceding step by adding the stickysession=JSESSIONID|jsessionid parameter. This parameter configures the cookie/path that will be used for stickiness. For example (update shown in bold):

Go to the computer on which Pivotal tc Server is running and update the server.xml configuration file of both tc Runtime instances by adding the jvmRoute=value attribute to the Catalina <Engine> element. Set the value of this attribute equal to the value you specified (in a preceding step) for the route parameter of the BalancerMember directive in the Pivotal Web Server httpd.conf file that describes the tc Runtime instance.

Following our example, the updated <Engine> entry for the instanceOne tc Runtime instance (that uses port 8081) would be as follows (new attribute in bold):

If you configure sticky sessions, Pivotal recommends that you also configure session replication for the tc Runtime instances. For details, see the section titled Enabling Clustering for High Availability in the Pivotal tc Server documentation.

You have now configured load balancing for the two tc Runtime instance using the front-end Pivotal Web Server.

What to do next

For full reference documentation on the directives described in step 3, along with additional parameters you can use, see Apache Module mod_proxy on the Apache Software Foundation Web site.

Ensure that you can access your application through the Pivotal Web Server instance, which in turn routes the request to one of the tc Runtime instances. Do this by invoking your application in a browser, but specify the Pivotal Web Server instance rather than the tc Runtime instance. For example, if the URL to access the Pivotal Web Server is http://www.example.com, invoke the /my-app application using the following URL in your browser:

http://www.example.com/my-app

If you see your application, then you have correctly set up the Pivotal Web Server instance to route requests to the /my-app application to one of the two tc Runtime instances. The Pivotal Web Server instance will also balance the load between the two instances.

If you enabled the balancer manager Web application, use it to watch and control load-balancing activity. Access the balancer manager application by navigating to the following URL in your browser:

http://localhost:port/balancer-manager

where port is the port number of the Pivotal Web Server instance (80 by default.) For security, the balancer manager configuration allows access only to users who navigate to the application using a browser installed on the same computer on which the Pivotal Web Server instance is actually running.

Configure SSL Between Pivotal Web Server and Pivotal tc Server

For additional security, it is often desirable to configure SSL between a Pivotal Web Server instance and one or more tc Runtime instances, although it’s not required. (tc Runtime is the runtime component of Pivotal tc Server.)

SSL certificates are frequently used to confirm the identity of a server before consuming its services and to secure communications with the server. Typically, if you use a Pivotal Web server instance to load balance requests to one or more tc Runtime instances, the SSL encryption and certificate authentication is terminated at the Web Server instance. Communication between the Web Server and tc Runtime instances is then trusted and in clear text.

However, there are organizational security policies and B2B scenarios that might mandate secure communication between the Pivotal Web Server and tc Runtime instances. Furthermore, it might be important to restrict access to the tc Runtime instances to known instances of Pivotal Web Server.

This section provides details for configuring SSL communication and client certificate authentication between Pivotal Web Server and tc Server. The high-level steps are as follows, with detailed information about each step in its own sub-section:

Configure tc Runtime Instances to Use SSL

Pivotal recommends that you configure a tc Runtime instance to use SSL by specifying the bio-ssl template when you create or modify an instance; this template adds the correct configuration to the conf/server.xml file and automatically generates a keystore based on your inputs. You specify the bio-ssl template when you create a new tc Runtime instance using the tcruntime-instance command. Additionally, as of version 2.8 of Pivotal tc Server, you can also apply the template to an existing instance.

The following example shows how to create a new tc Runtime instance that uses the bio-ssl template:

In the preceding example, the tc Runtime instance will be located in the /var/opt/pivotal/pivotal-tc-server-standard directory and will use default values when creating the keystore. If you want to customize the keystore, use the --interactive option and the command will prompt you for specific information:

Note: The apply-template option of tcruntime-instance is available as of version 2.8 of Pivotal tc Server.

To invoke an application deployed to the tc Runtime instance using HTTPS, specify the HTTPS port. The default HTTPS port is 8443, although you might have configured a different port for your particular instance. For example:

https://host:8443/my-app

See Create and Modify a tc Runtime Instance in the Getting Started with Pivotal tc Server guide in this Documentation Center for details.

If you chose not to use the bio-ssl template, you can create your own keystore using the keytool command, as shown in the following example:

In the preceding example, CATALINA_BASE refers to the instance directory, such as /var/opt/pivotal/pivotal-tc-server-standard/instanceOne.

Update the appropriate <Connector /> element in the instance’s conf/server.xml file by adding the keyAlias, keystoreFile, and keystorePass attributes, setting the values to those you specified when you created the keystore using keytool as shown above. For example:

Configure the Pivotal Web Server Instance to Use SSL

The easiest way to do configure SSL for a Web Server instance is to use the newserver interactive command to create a new instance and specify that you want to enable SSL for the instance. The command performs configuration tasks and creates a private key. See Create Pivotal Web Server Instances.

Generate a JKS formatted certificate authority file. The following example shows how to use the keytool command to create the file cacerts.jks from the CA file ca.crt you created in the preceding step:

Update the Web Server configuration to communicate with the tc Runtime instances securely by editing the conf/http.conf file in the Web Server instance directory (such as /opt/pivotal/webserver/lb-server) and changing the tc Runtime URLs so they use HTTPS and specify the HTTPS port. Following the example from the load balancing section, if you specified that the HTTPS ports for instanceOne and instanceTwo were 8443 and 8553, respectively, the updated file would look like this:

This section describes how to configure the Web Server instance to authenticate itself using the client certificate you created in a previous step and configured for each tc Runtime instance.

On the computer on which you installed Pivotal Web Server, create a client certificate and key. Use the same certificate authority file (called ca.crt in the example) that you created in a previous step.

In the preceding example, the newly generated client key is called client.key and the client certificate file is called client.crt. Replace your-openssl.cnf-file with the full path name of the openssl.cnf file on your computer, such as /etc/pki/tls/openssl.cnf.

Concatenate the generated client key and client certificate files into a single file. In the following example, the new file is called client.crtkey:

prompt$ cat client.crt client.key > client.crtkey

Copy the generated client.crtkey file to the ssl directory of the Web Server instance directory. For example:

prompt$ cp client.crtkey /opt/pivotal/webserver/lb-server/ssl

Configure the mod_ssl module of the Web Server instance to use SSL for the proxy engine and to use the generated client certificate and key file by adding the following directives to the file that contains the SSL configuration, such as conf/extra/httpd-ssl.conf:

SSLProxyMachineCertificateFile "ssl/client.crtkey"
SSLProxyEngine on

Restart the Pivotal Web Server instance for the configuration changes to take effect. For example:

Test that everything is working correctly by accessing your application through the Pivotal Web Server host or IP address and HTTPS port. For example, if the Web Server IP address is 192.0.2.33 and configured an HTTPS port of 8663:

https://192.0.2.33:8663/my-app

Because you have configured your Web Server instance with the client certificate required by the tc Runtime instances, you will see your application, and yet all communication from your browser to the tc Runtime instance is using SSL.

Configure BMX for Monitoring Pivotal Web Server Instances

As of version 5.1, all new Pivotal Web Server instances are configured with BMX by default.

BMX is an Apache HTTPD framework that provides internal runtime information (performance metrics, status, configuration, and current capacity) to monitoring applications such as Pivotal Hyperic. In turn, these types of applications monitor the health of Pivotal Web Server instances by running BMX queries to gather metrics and configuration information.

New Pivotal Web Server instances have the following default BMX configuration:

Three main BMX modules (mod_bmx, mod_bmx_status, and mod_bmx_vhost) are all enabled. Together, these modules provide overall runtime statistics of the Web Server instance, as well as the virtual hosts running within the instance.

Access is allowed only to processes running on http://localhost (IP address 127.0.0.1), or in other words, only to monitoring applications running on the same computer as the Web Server instance.

Access requires no authentication.

BMX access is enabled for all virtual hosts defined for the Web Server instance.

The default BMX configuration for Pivotal Web Server instances make them immediately available for monitoring by monitoring applications.

The BMX-related modules are loaded into the Web Server instance using appropriate LoadModule directives in the conf/httpd.conf configuration file. Additional BMX configuration is in the conf/extra/httpd-info.conf file, which the main conf/httpd.conf file includes using the Include conf/extra/httpd-info.conf directive.

Procedure

To disable BMX access to your Pivotal Web Server instance, comment out the appropriate LoadModule directives in the conf/httpd.conf configuration file for your instance as shown:

To allow BMX access to processes running on hosts other than the localhost, edit the <Location /bmx> directive in the conf/extra/httpd-info.conf file and add the IP address or fully qualified domain name to the Allow from directive. For example, to allow example.com access in addition to localhost: