Tomcat cluster configuration steps

There are several significant problems related to running your web application on a single server. When your web site is successful and begins to get a high volume of requests, eventually one server computer just won’t be able to keep up with the processing load. Another common problem of using a single server computer for your web site is that it creates a single point of failure. If that server fails, your site is immediately out of commission. Regardless of whether it’s for better scalability or for fault tolerance, you will want your web applications to run on more than one server computer. So here we need clustering of server to handle the load and failovers.

Before we dig into the details about how to set up a Tomcat cluster, we want to be clear on the definitions of some terms so that it will be easy for understanding the concept

Fault tolerance

The degree to which the server software adapts to failures of various kinds (including both hardware and software failures) so that the system may still serve client requests transparently, usually without the client being aware of these failures.

Failover

When one server (software or hardware) suffers a fault and cannot continue to serve requests, clients are dynamically switched over to another server that can take over where the failed server left off.

High availability

A service that is always up, always available, always serving requests, and can serve an unusually large volume of requests concurrently is saidto be highly available. A highly available service must be fault tolerant, or else it will eventually fail to be available due to hardware or software failures.

Distributed

The term “distributed” simply means that some computing process may occur across multiple server computers working together to achieve a goal or to formulate an answer or multiple answers, ideally in parallel. For example, many web server instances each running on a separate server computer behinda TCP load balancer constitutes a distributed web server.

Replicated

Replication means that any state information is copiedverbatim to two or more server software instances in the cluster to facilitate fault tolerance and distributed operation. Usually, stateful services that are distributed must replicate client session state across the server software instances in the cluster.

Load balancing

When a request is made to a distributed service and the server instance that received the request is too busy to serve it in a reasonable amount of time, another server instance may not be as busy. A load-balanced service is able to forward the request to a less busy server instance within the cluster to be served. Load balancing can distribute the request-processing load to take advantage of all available computing resources.

Cluster

A cluster is made up of two or more server software instances running on one or more server computers that work together to transparently serve client requests so that the clients perceive the group as a single highly available service. The goal

of the group is to provide a highly available service to network clients, while utilizing all available computing resources as efficiently as reasonably possible.

In general, clustering exists to facilitate high availability and/or fault tolerance. Load balancing andstate replication are just two important elements of clustering.

To configure and run a Tomcat cluster, you need to set up more than just Tomcat. For example, you needto provide a facility so that requests coming into Tomcat are spread across multiple instances. This involves software that runs in addition to your Tomcat installations. To identify the points in the system where clustering features may be implemented to distribute the requests, let’s take a look at the steps of the average HTTP client request. The below figure shows the steps of one HTTP client’s request through the system.

We show using mod_proxy for the connector module from Apache httpd to Tomcat because depending on how your web application is written, you may not need to use Apache httpd or mod_proxy to set up anduse a Tomcat cluster. We show these components so that you can see how using them affects the HTTP request communication sequence and which types of clustering features you may want to use. If you use Apache httpd, httpd is your web server. If you use Tomcat standalone, Tomcat is your web server.

1. Local DNS request. The user’s web browser attempts to resolve the web site’s IP address from its name via a DNS lookup network request to the user’s local DNS server (usually her ISP’s DNS server or her own company’s DNS server). Most

web browsers ask for this IP address only once per run of the browser. Subsequent HTTP requests from the same browser are likely to skip this step as well as the next step.

2. Authoritative DNS request. Usually, the user’s local DNS server will not already have the web site’s IP address in its cache (from a prior request), so it must in turn ask the web site’s authoritative DNS server for the IP address of the web site that

the user wishes to view. The authoritative DNS server will reply to the local DNS server with the IP address that it should use for the web server. The local DNS server will attempt to cache this answer so that it won’t need to make the same

request to the authoritative DNS server again anytime soon. Subsequent requests from other browsers in the same network as the first browser are likely to skip this step because the local DNS server will already have the answer in its cache.

3. Local DNS response. The local DNS server replies, giving the browser the IP address of the web server.

4. HTTP request. The browser makes an HTTP request to the IP address given by DNS. This request may utilize HTTP keep-alive connections for network efficiency, and therefore this single TCP socket connection may be the only socket connection made from the browser to the web server for the entire duration of the browser’s HTTP session. If the browser does not implement or use HTTP keep-alive, each request for a document, image, or other content file will create a separate TCP socket connection into the web server.

5. Tomcat sends one or more requests to backend server(s). Tomcat may depend on other servers to create the dynamic content response that it forwards back to the browser. It may connect to a database by way of JDBC, or it may use JNDI to look up other objects, such as Enterprise JavaBeans, and call one or more methods on the objects before being able to assemble the dynamic content that makes up the response.

Upon completion of the necessary steps above, the direction of flow reverses and replies to each step are made in the reverse order that the request steps were made, working back through the already open network connections.

To support a cluster of Apache httpd andTomcat instances, you can implement clustering features in multiple spots along this request sequence. Below figure shows the same request sequence, only this time the web site is served on a cluster of Apache httpd and Tomcat instances.

Here are some of the clustering technologies that you could set up and run:

DNS request distribution

Instead of configuring your DNS server to give out one IP address to one Apache httpd server instance, you can configure it to give out three IP addresses that each go to a separate Apache httpd or Tomcat instance.

TCP Network Address Translation (NAT) request distribution

Regardless of how many IP addresses DNS gives to the client’s browser, the web server’s first contact IP address(es) can be answered by a TCP NAT request distributor that acts as a gateway to two or more web servers behindit. You can use

the NAT request distributor for both load balancing and failover.

mod_proxy_balancer load balancing and failover

If you run two or more Tomcat instances behindone or more Apache httpd instances, you can use mod_proxy_balancer for loadbalancing andfailover to distribute requests across your Tomcat cluster. You can also use it to keep requests

from being distributed to any failed Tomcat instances.

JDBC request distribution and failover

You could use a clustered database and a JDBC driver that load balances connections among the machines in the database cluster or a replicated database with a JDBC driver that knows when to failover to the secondary database server.

Here we will apply the mod_proxy_balancer load balancing and failover

If you decide to use Apache httpd as your web server, andyou’re using mod_proxy to sendrequests to Tomcat (either via HTTP or AJP), you can take advantage of mod_proxy_balancer‘s loadbalancing andfault tolerance features. These Apache httpd modules are part of the Apache httpd web server project, andyou’ll be happy to know that the mod_proxy_* modules are almost always built and shipped with Apache httpd. This means that you can configure and use them without downloading, building, and installing anything extra, which makes it quite a bit easier to set up and use a load balancer.

Here are some of the things that each Apache httpd with mod_proxy_balancer in your cluster can do:

Distribute requests to one or more Tomcat instances

You can configure many Tomcat instances in your Apache httpd‘s configuration, giving each Tomcat instance an lb_factor value that functions as a weighted request distribution metric.

Detect Tomcat instance failure

mod_proxy_balancer will detect when a Tomcat instance’s connector service is no longer responding and will stop sending requests to it. Any remaining Tomcat instances will take the additional load for the failed instance until it is brought back online.

Detect when a Tomcat instance comes back up after failing

After mod_proxy_balancer has stoppedd istributing requests to a Tomcat instance due to the instance’s connector service failure, mod_proxy_balancer periodically checks to see if the server is available again andwill automatically converge it into

the pool of active Tomcat instances when it becomes available again.

Manually mark a Tomcat instance available or unavailable

During the runtime of Apache httpd you may use the balancer-manager web page (a feature built into Apache httpd, implementedby mod_proxy_balancer and mod_status) to mark Tomcat instances as being either available or unavailable.

This allows adding or removing clustered Tomcat instances without any web site down time as it does not require restarting Apache httpd for the change to take effect. Again, the goodnews here is that these modules are almost always built

into Apache httpd, andall you needto do to use these features is configure your Apache httpd to turn it on.

The following steps outline how to set up one Apache httpd (on a server computer called apache1) to do HTTP load balancing across two Tomcat instances that reside on two separate server computers called tc1 and tc2.

First, add the following configuration to your Apache httpd‘s configuration files (we added it in the form of a new config file named /etc/httpd/conf.d/proxy-balancer.conf, but you may need to place it in a different file for your installation of Apache httpd):

<IfModule !proxy_module>

LoadModule proxy_module modules/mod_proxy.so

</IfModule>

#<IfModule !proxy_ajp_module>

# LoadModule proxy_ajp_module modules/mod_proxy_ajp.so

#</IfModule>

<IfModule !proxy_http_module>

LoadModule proxy_http_module modules/mod_proxy_http.so

</IfModule>

<IfModule !proxy_balancer_module>

LoadModule proxy_balancer_module modules/mod_proxy_balancer.so

</IfModule>

<IfModule !status_module>

LoadModule status_module modules/mod_status.so

</IfModule>

<IfModule proxy_balancer_module>

ProxyRequests off

<Proxy balancer://tccluster>

BalancerMember http://tc1:8080 loadfactor=1 max=150 smax=145

BalancerMember http://tc2:8080 loadfactor=1 max=150 smax=145

Order Deny,Allow

Allow from all

</Proxy>

<Location /balancer-manager>

SetHandler balancer-manager

Order Deny,Allow

Allow from all

</Location>

<Location /my-webapp>

ProxyPass balancer://tccluster/my-webapp stickysession=jsessionid

ProxyPassReverse balancer://tccluster/my-webapp

Order Deny,Allow

Allow from all

</Location>

<Location /examples>

ProxyPass balancer://tccluster/examples stickysession=jsessionid

ProxyPassReverse balancer://tccluster/examples

Order Deny,Allow

Allow from all

</Location>

</IfModule>

This configuration will loadbalance two Tomcat instances running on two separate hosts (named tc1 and tc2). The loadwill be distributedevenly between both Tomcat instances, but once Tomcat creates a session for the client andsend s the client a

JSESSIONID cookie, mod_proxy_balancer will distribute that client’s requests to the same Tomcat instance each time. The above configuration proxies the /my-webapp and /examples base URIs through to the cluster of Tomcat instances, so that requests for those webapps are handled by the Tomcat cluster. The configuration also turns on the /balancer-manager page so that the cluster instances may be managedvia a web browser.You can set the loadfactors to any integer values you want. The higher the number you use, the more preferredthe Tomcat instance is; the lower the loadfactor, the fewer requests the Tomcat instance will be given. If a Tomcat instance is not responding, mod_proxy_balancer marks that instance as unavailable andfails over to the next instance in the list.

Next, configure and run the Tomcat instances on the Tomcat server computers. Set up your Java environment on tc1 and tc2:

$ JAVA_HOME=/usr/java/jdk1.6.0_02


$ export JAVA_HOME


$ PATH=$JAVA_HOME/bin:$PATH


$ export PATH


$ java -version


java version "1.6.0_02"

Java(TM) SE Runtime Environment (build 1.6.0_02-b06)

Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_02-b06, mixed mode)

Make sure that CATALINA_HOME is set on tc1 and tc2:

$ CATALINA_HOME=/usr/local/apache-tomcat-6.0.14


$ export CATALINA_HOME


$ cd $CATALINA_HOME

Then, on each of the Tomcat instance machines, configure the CATALINA_HOME/conf/server.xml file so that the Engine’s jmvRoute is set to the same string you set the Tomcat instance’s tomcatId to in the workers2.properties file:

<Engine name=”Catalina” defaultHost=”localhost” jvmRoute=”tomcat1″>

Set the secondTomcat instance’s jvmRoute to “tomcat2”, etc. Each Tomcat instance’s jvmRoute value must be unique.

Also, in the same file, make sure that the Connector you’re using is configured properly for being used through mod_proxy.

To test that the request distribution is indeed working, we’ll add some test content.

In each Tomcat instance’s webapps/ROOT/ directory, do the following:

$ cd $CATALINA_HOME/webapps/examples


$ echo 'Tomcat1' > instance.txt

Do the same in the secondTomcat’s webapps/ROOT/ directory, labeling it as Tomcat2:

$ cd $CATALINA_HOME/webapps/examples


$ echo 'Tomcat2' > instance.txt

Then, start up each of the two Tomcat instances:

$ cd $CATALINA_HOME


$ bin/catalina.sh start

Once it’s all running, access the Apache httpd instance on the apache1 machine, and request the instance.txt page by loading the URL http://apache1/examples/instance.txt
in your browser. The first request will likely be slow because Tomcat initializes everything on the first request. The page will display either Tomcat1 or Tomcat2, depending on which Tomcat instance mod_proxy_balancer sent you to. Reloads of the same URL shouldsendyou back to the same instance each time, proving that mod_proxy_balancer is performing session affinity load balancing. Try accessing mod_proxy_balancer‘s /balancer-manager page by loading the URL http://apache1/balancer-manager in your browser. It shows information about mod_proxy_balancer‘s cluster of configured loadbalanced and proxied backend server instances.

Once above clustering is done we shall do the session replication which will enable tomcat to continue the app session incase of failure so the user wont need to login back to app incase of failover.

Configuring All-to-All Replication

This is the most common configuration for Tomcat session replication—two or more Tomcat nodes, and each node sends replication messages to each of the other nodes in the cluster. With this configuration, you would still have fault tolerance if

all of your servers crashedexcept for a single server because all sessions would failover to the one remaining node.

Here is what you will needbefore you begin configuring Tomcat to do all-to-all replication:

• Your webapp must run on a Java JDK version 1.5.0 or higher. (You must already do this if you are running it on Tomcat version 6.0 or higher.)

• Your webapp must only add objects to users’ sessions that properly implement the java.io.Serializeable interface. If any objects added to the session are not serializeable, session replication attempts will not work.

• One or more server machines capable of running two or more Tomcat instances. For instance, you can run two Tomcat instances on a single machine to test Tomcat session replication and, optionally, context attribute replication. Or, you

can set up two or more server machines, each machine running a single instance of Tomcat. If your instances are on separate machines, just make sure that the two machines are communicating with each other properly before configuring

andtesting Tomcat clustering. For example, if you are going to use the multicast group membership autodiscovery, make sure that multicast communication is configured and working between all of the server machines first.

• Each of your Tomcat instances must set a unique jvmRoute value on their <Engine> element in server.xml. This value gets appended to the end of the session cookie, which enables the loadbalancer to know which Tomcat node to send subsequent requests to.

• Session affinity loadbalancing; you must first set up your loadbalancer in front of your Tomcat nodes, and it must be performing session affinity (sticky session) loadbalancing. One completely free loadbalancer that you coulduse for this is Apache httpd and mod_proxy_balancer

• You must synchronize time across all of the server machines that participate in the same cluster group. Some features of Tomcat’s clustering messaging code are time dependent, and a difference in clock time even as small as a second or two

couldmake it malfunction. We highly suggest using Network Time Protocol (NTP) to set your servers’ clocks so that they are properly synchronized.

• If you are going to use the multicast cluster node autodiscovery, you must make sure that multicast works between the computers running each of the Tomcat nodes. If you cannot use multicast or do not wish to use multicast, you must

statically configure the cluster group members. See the “Configuring Static Membership” section.

Now that all of the prerequisites are out of the way, we can configure clustering. First, configure your webapp to be a distributed webapp. Each webapp that you wish to run as a distributed webapp must have the <distributable/> element in its WEBINF/web.xml file. This is the servlet specification compliant way to tell the servlet container that the webapp is designed to be able to run in a distributed servlet container with session replication enabled.

Each <Context> for a distributable webapp must have the distributable=”true” attribute setting. This tells Tomcat that not only is the webapp distributable but that you are directing Tomcat to run the webapp as a distributed webapp (as opposed to the default which is nondistributed, nonreplicated). You must have both the distributable=”true” attribute set on the webapp’s Context andthe <distributable/> element in the webapp’s web.xml for session clustering to work.

Next, we’ll configure the Tomcat nodes to cluster together as a group. In the first Tomcat’s server.xml file, we’ll add a <Cluster> element andsome subelements under Tomcat’s <Engine> element, like this:

<Engine name="Catalina" defaultHost="www.example.com" jvmRoute="tc1">

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8">

<Manager className="org.apache.catalina.ha.session.DeltaManager"

expireSessionsOnShutdown="false"

notifyListenersOnReplication="true"/>

<Channel className="org.apache.catalina.tribes.group.GroupChannel">

<Membership className="org.apache.catalina.tribes.membership.McastService"

address="228.0.0.4"

port="45564"

frequency="500"

dropTime="3000"/>

<Sender className="org.apache.catalina.tribes.transport.

ReplicationTransmitter">

<Transport className="org.apache.catalina.tribes.transport.nio.

PooledParallelSender"/>

</Sender>

<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"

address="auto"

port="4000"

autoBind="100"

selectorTimeout="5000"

maxThreads="6"/>

<Interceptor className="org.apache.catalina.tribes.group.interceptors.

TcpFailureDetector"/>

<Interceptor className="org.apache.catalina.tribes.group.interceptors.

MessageDispatch15Interceptor"/>

</Channel>

<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"

filter=""/>

<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

<ClusterListener className="org.apache.catalina.ha.session.

JvmRouteSessionIDBinderListener"/>

<ClusterListener className="org.apache.catalina.ha.session.

ClusterSessionListener"/>

</Cluster>

Under your Tomcat’s <Engine> line in server.xml file, add these configuration lines in each Tomcat node that you want to be part of the cluster group. Make sure, though, that the <Engine> element’s jvmRoute attribute is set to a different value in each of your Tomcat nodes.

With the Cluster configuration shown above, you shouldbe able to run each Tomcat node and each node should automatically discover each other node in the cluster via multicast. Then, once the nodes are aware of each other, they can begin replicating session data to each other as long as the same version of the same webapp is deployed on all nodes that are participating in the webapp’s session replication. Leave all of the numbers of the Cluster configuration the same on all nodes, including the Membership address and port attribute values. The code is smart enough to figure out how to use the network to communicate without interfering with the networking of another Tomcat node, even when more than one node is running on the same physical computer.

When you start your Tomcat nodes, in the catalina.out log, you shouldsee something like this:

=====================================================================

INFO: Starting Servlet Engine: Apache Tomcat/6.0.14

Sep 27, 2008 7:07:39 PM org.apache.catalina.ha.tcp.SimpleTcpCluster start

INFO: Cluster is about to start

Sep 27, 2008 7:07:39 PM org.apache.catalina.tribes.transport.ReceiverBase bind

INFO: Receiver Server Socket bound to:www.example.com:4000

Sep 27, 2008 7:07:39 PM org.apache.catalina.tribes.membership.McastServiceImpl

setupSocket

INFO: Setting cluster mcast soTimeout to 500

Sep 27, 2008 7:07:39 PM org.apache.catalina.tribes.membership.McastServiceImpl

waitForMembers

INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:4

Sep 27, 2008 7:07:40 PM org.apache.catalina.tribes.membership.McastServiceImpl

waitForMembers

INFO: Done sleeping, membership established, start level:4

Sep 27, 2008 7:07:40 PM org.apache.catalina.tribes.membership.McastServiceImpl

waitForMembers

INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:8

Sep 27, 2008 7:07:41 PM org.apache.catalina.tribes.membership.McastServiceImpl

waitForMembers

INFO: Done sleeping, membership established, start level:8

Sep 27, 2008 7:07:42 PM org.apache.catalina.ha.session.DeltaManager start

INFO: Register manager /examples to cluster element Engine with name Catalina

Sep 27, 2008 7:07:42 PM org.apache.catalina.ha.session.DeltaManager start

INFO: Starting clustering manager at /examples

Sep 27, 2008 7:07:42 PM org.apache.catalina.ha.session.DeltaManager

getAllClusterSessions

INFO: Manager [www.example.com#/examples]: skipping state transfer. No members active

in cluster group.

Sep 27, 2008 7:07:42 PM org.apache.catalina.ha.session.JvmRouteBinderValve start

INFO: JvmRouteBinderValve started

Sep 27, 2008 7:07:42 PM org.apache.coyote.http11.Http11Protocol start

INFO: Starting Coyote HTTP/1.1 on http-8080

Sep 27, 2008 7:07:42 PM org.apache.jk.common.ChannelSocket init

INFO: JK: ajp13 listening on /0.0.0.0:8009

Sep 27, 2008 7:07:42 PM org.apache.jk.server.JkMain start

INFO: Jk running ID=0 time=0/26 config=null

Sep 27, 2008 7:07:42 PM org.apache.catalina.startup.Catalina start

INFO: Server startup in 4128 ms

Sep 27, 2008 7:09:45 PM org.apache.catalina.tribes.io.BufferPool getBufferPool

INFO: Created a buffer pool with max size:104857600 bytes of type:org.apache.

catalina.tribes.io.BufferPool15Impl

Sep 27, 2008 7:09:46 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded

INFO: Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp:/

/tc2.example.com:4001,tc2.example.com,4001, alive=1208,id={56 -55 94 23 26 -33 72 -66

-101 -47 109 5 -122 89 51 68 }, payload={}, command={}, domain={}, ]

================================================================================

This shows that the Tomcat node tc1 startedup successfully andthen automatically discovered another node named tc2 and added tc2 as a member of the cluster group. At that point, the two nodes are ready to replicate session data via TCP. They have exchangedTCP host andport information andhave connectedto each other via TCP for replication communications. If you do not see the SimpleTcpCluster memberAdded message in your catalina.out logfile, you shouldrecheck your server.xml files to make sure that you have the configuration set correctly andalso retest multicast communications between the computers where the Tomcat nodes are running.

This ends the clustering with session replication and failover set up of tomcat. For additional information for debugging and configuration you can refer below links.

If you want a more step to step approach for tomcat clustering you can go to this link for

Steps To Configure Clustering in Apache Tomcat

Additional Resources

High Availability Software

http://backhand.org/wackamole

http://www.linuxvirtualserver.org

http://www.linux-ha.org

Message Oriented Middleware

http://www.spread.org

http://www.javagroups.com

Database Clustering

http://sequoia.continuent.org

http://www.oracle.com/database/rac_home.html

http://dev.mysql.com/doc/refman/5.0/en/replication.html

Commercial HA Hardware

http://www.citrix.com/English/ps2/products/product.asp?contentID=21679

http://www.foundrynet.com/solutions/sol-app-switch

IP Multicast

http://www.tldp.org/HOWTO/Multicast-HOWTO.html

In case of any ©Copyright or missing credits issue please check CopyRights page for faster resolutions.