Using Apache load balancer with Glassfish

Although Glassfish comes out of the box with provisions to add load balancing and failover but there are many reasons to add a third party web server to achieve that goal. For me the most compelling reason is the flexibility that this configuration provides. You can disconnect any glassfish domain and take it offline and then bring it back up again and connect it to the web server without anyone ever noticing something happened in the background.

Why use Apache web server? Now this is another question that can easily spark a debate but I am not going to go into the details, I am using Apache simply because this is easy to use and have been around since ages and has already proven as a sturdy and bulletproof server.

Just to illustrate the working, I shall use a very simple configuration, one Glassfish server with three domains running the same application and one Apache web server with mod_jk load balancing the requests evenly across all three domains.

 

So there we go, every request coming from the user is ending up on the load balancer and from there the load balancer splits these requests evenly across the three glassfish domains. If any of the domains fails then the load balancer will automatically split the load between the remaining two.

Prerequisites:

• Install the Glassfish V2
• Create the three domains with any http ports other than 80
• Install Apache Web Server on port 80

You also need to download these files from Apache website

• mod_jk.so
• commons-logging.jar
• commons-modeler.jar
• tomcat-ajp.jar

Once the files are downloaded then place all three jar files in the Glassfish lib folder.

Place the mod_jk.so in the Apache Webserver’s modules folder.

Now create a workers.peroperties file and place it in the Apache Webserver’s conf folder.

Now you need to enable JK in the Glassfish Server for each domain. This is where you specify the port for mod_jk to use for load balancing. To enable that you have two options at your disposal

• Update the domain.xml file
• Use the asadmin Command Line Interface

You can you any oen of these approaches, whichever works for you. Let's see both of these options..

I am using port 9001 for my first domain and for the other three domains I am using 9002 and 9003 respectively.

• Updating the domain.xml file

Open the domain.xml file and find the <java-config> tag and add the following line after the last <jvm-options> </jvm-options> tag within the java-config block just before the </java-config> tag.

-Dcom.sun.enterprise.web.connector.enableJK=9001

• Using asadmin commands

Open the asadmin prompt and enter the following commands to create and enable the mod-jk listener

PS C:\Westcoast\glassfish4\bin> .\asadmin --port 4848
Use "exit" to exit and "help" for online help.
asadmin> create-http-listener --listenerport 9001 --listeneraddress 0.0.0.0 --defaultvs server mod-jk-listener
Command create-http-listener executed successfully.
asadmin> set server-config.network-config.network-listeners.network-listener.mod-jk-listener.jk-enabled=true
server-config.network-config.network-listeners.network-listener.mod-jk-listener.jk-enabled=true
Command set executed successfully.
asadmin>

Whichever way you choose, you need to repete the same steps to create mod-jk ports for all of your domains that you want to load balance. Depending on the Glassfish version, one of the above option nay not work.

NOTE: You need to use the asaadmin commands for Glassfish 4.x.x

Now open the workers.properties file and add the following lines:

# The advanced loadbalancer LB worker
# All request are processed by all workers

worker.list=loadbalancer

#First Worker for MyGlassfihsServer
worker.worker1.port=9001
worker.worker1.host=MyGlassfihsServerName
worker.worker1.type=ajp13
worker.worker1.lbfactor=33
# Define preferred failover node for worker11
worker.worker1.redirect=worker2

#Second Worker for MyGlassfihsServer
worker.worker2.port=9002
worker.worker2.host=MyGlassfihsServerName
worker.worker2.type=ajp13
worker.worker2.lbfactor=33
# Define preferred failover node for worker11
worker.worker2.redirect=worker3

#Third Worker for MyGlassfihsServer
worker.worker3.port=9003
worker.worker3.host=MyGlassfihsServerName
worker.worker3.type=ajp13
worker.worker3.lbfactor=34
# Define preferred failover node for worker11
worker.worker3.redirect=worker1

# Define the LB worker
worker.loadbalancer.type=lb
worker.sticky_session=false
worker.loadbalancer.balance_workers=worker1,worker2,worker3

I believe all the properties in this file are self-explanatory. You just need to create configurations for the workers and tell the mod_jk to balance all of the workers with the given load factor. The last line is where you tell the load balancer which workers it needs to balance, if you remove any worker from this line, apache will not forward any requests to that server.

Now open the httpd.conf file. You can see in the file quite a few lines where Apache is loading different modules. After the last LoadModule command, add your own lines to load the mod_jk.so module.

LoadModule jk_module modules/mod_jk.so
JkWorkersFile conf/worker.properties
# Where to put jk logs
JkLogFile "|bin/rotatelogs logs/mod_jk.log 86400"

# Set the jk log level [debug/error/info]
JkLogLevel error
# Select the log format
JkLogStampFormat "[%d-%b-%Y %H:%M:%S] "

# JkOptions indicate to send SSL KEY SIZE,
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
JkRequestLogFormat "%w %V %T"

# Send all jsp requests to GlassFish
JkMount / *.jsf loadbalancer

Notice the last line, i.e. “JkMount / *.jsf loadbalancer” This is where you are telling the Apache web server to load balance the requests. It will ONLY load balance the requests that have .jsf in the URL.

Now restart the glassfish domains to load the new jvm-options added to the java-config tag and then restart the apache web server.

You now have three Glassfish domains running behind an Apache Webserver and if you simply call the URL for your application, you will see from the Glassfish and Apache log files how this is splitting the load between these three domains.

For me the best thing is that if I need to take any of these domains down for whatever reason, all I need to do is to update this line in the workers.properties file

worker.loadbalancer.balance_workers=worker1,worker2,worker3

Remove the worker for that domain from the list of balanced workers and restart apache, which takes less than a second. Once I am ready to bring the domain live, I simply add that worker back to this line and restart apache.

If you find any difficulties then simply double check if you have all the files in the right places and also check for the syntax mistakes in the worker.properties file and the httpd.conf file in Apache. For Glassfish there is no further configuration required other than adding the enableJK JVM option.