Monday, January 26, 2015

ActiveMQ Configuration Explained: Master-Slave With Network Of Brokers

Introduction

In my previous posts, I have covered tutorial about ActiveMQ with the following aspects:

I have explained how master-slave paradigm with shared file system works. In this post, I will explain more advanced topic, namely, Network Of Brokers. I will demonstrate the procedures to setup a network of ActiveMQ brokers with Master-Slave pairs.

Network Topology

As shown in the above figure, I configured two clusters. The WEST cluster contains two ActiveMQ nodes, namely, north and south. These two node forms a master-slave pair. Depending on applications, the nodes number could be 2 or 3. These nodes sharing the same NFS where the kahaDB is located. The same is for the EAST cluster.

Configurations

Directory Layout

The ActiveMQ is installed at:

/opt/app/activemq/apache-activemq-5.10.0

The tomcat is installed at:

/opt/app/activemq/cluster/master-slave/tomcat/apache-tomcat-8.0.17

north and south nodes

/opt/app/amq/NetworkOfBrokers/cluster/north
/opt/app/amq/NetworkOfBrokers/cluster/south

activemq.xml

To setup network of brokers as shown in the introduction, we just need to update the activemq.xml file from west cluster. The file contents is as the following: 
 
  1 
  2
  3 
  6
  7     
  8         
  9             file:${activemq.conf}/credentials.properties
 10         
 11     
 12
 13     
 19     
 20
 21     
 25
 26         
 27             
 28               
 29                 
 30                   
 31                      
 32                   
 33                   
 34                     
 35                   
 36                 
 37                 
 38                   
 39                     
 40                   
 41                 
 42               
 43             
 44         
 45
 46         
 47             
 48         
 49
 50         
 51             
 62
 63                      
 64                      
 65
 66                      
 67                         
 68                      
 69             
 70             
 81
 82                      
 83                      
 84
 85                      
 86                         
 87                      
 88             
 89         
 90
 91         
 92             
 93         
 94
 95         
 96             
 97                 
 98                     
 99                 
100                 
101                     
102                 
103                 
104                     
105                 
106             
107         
108
109         
110             
111             
112             
113             
114             
115         
116
117         
118             
119         
120
121     
122

From the above configuration, you can see that I have configured two network connections, namely, Queue_Connector and Topic_Connector. Copy this activemq.xml to both north and south nodes, and leave the east cluster as it is as show my my previous blog of pure master-slave clustering.

at No comments:

Sunday, January 25, 2015

Monitoring ActiveMQ Using Tomcat Web Container

So far I have covered tutorial about ActiveMQ with the following aspects:

In this post, I will cover the topic on how to setup Apache Tomcat to monitoring ActiveMQ node in a Master-Slave cluster. In the next post, I will expain the more advanced high availability features for ActiveMQ, namely, Network Of Brokers

Installation

Current release of apache tomcat is version 8.0.17. We can download it from http://tomcat.apache.org/download-80.cgi. I am using RedHat Linux, so I downloaded tar.gz version.

After download Tomcat, we need to download two more archives:

Here are the download files I have:

[amq@SANDBOXFUSEV01 current]$ ll /software/TOMCAT/
total 42069
-rwxr-xr-x. 1 16777217 wheel  6371237 Jan 20 09:16 activemq-all-5.10.0.jar
-rwxr-xr-x. 1 16777217 wheel 12753574 Jan 20 09:16 activemq-web-console-5.10.0.war
-rwxr-xr-x. 1 16777217 wheel  9264418 Jan 20 09:15 apache-tomcat-8.0.17.tar.gz
[amq@SANDBOXFUSEV01 current]$

Once it is download executed the following commands:

cd /opt/app/amq/tomcat
tar -vxzf /software/TOMCAT/apache-tomcat-8.0.17.tar.gz

Here is the directory structures:

[amq@SANDBOXFUSEV01 tomcat]$ pwd
/opt/app/amq/tomcat
[amq@SANDBOXFUSEV01 tomcat]$ ll
total 4
drwxrwxr-x. 9 amq amq 4096 Jan 20 11:09 apache-tomcat-8.0.17
lrwxrwxrwx. 1 amq amq   20 Jan 20 12:38 current -> apache-tomcat-8.0.17
[amq@SANDBOXFUSEV01 tomcat]$

Now we need to copy the activemq-all-5.10.0.jar to tomcat's lib directory and copy activemq-web-console-5.10.0.war to webapps with the following commands: 

cp /software/TOMCAT/activemq-all-5.10.0.jar /opt/app/amq/tomcat/current/lib/.
cp /software/TOMCAT/activemq-web-console-5.10.0.war  /opt/app/amq/tomcat/current/webapps/.

Configuration Using TCP Connector

With all the software in place, we need to modify catalina.sh file by adding the following lines: [I attach few extra lines, so that you know where are the lines in the catalina.sh file]

# Uncomment the following line to make the umask available when using the
# org.apache.catalina.security.SecurityListener
#JAVA_OPTS="$JAVA_OPTS -Dorg.apache.catalina.security.SecurityListener.UMASK=`umask`"

JMX_OPTS=" -Dwebconsole.jmx.url=service:jmx:rmi:///jndi/rmi://sandboxfusev01:11099/jmxrmi"
JMX_OPTS="$JMX_OPTS,service:jmx:rmi:///jndi/rmi://sandboxfusev01:12099/jmxrmi"

JAVA_OPTS="-Dwebconsole.type=properties "
JAVA_OPTS="$JAVA_OPTS -Dwebconsole.jms.url=\"failover:(tcp://sandboxfusev01:61616,tcp://sandboxfusev01:61626)\" "
JAVA_OPTS="$JAVA_OPTS $JMX_OPTS "
JAVA_OPTS="$JAVA_OPTS -Dwebconsole.jmx.user=admin "
JAVA_OPTS="$JAVA_OPTS -Dwebconsole.jmx.password=activemq"


# ----- Execute The Requested Command -----------------------------------------

After configuring the Tomcat for monitoring ActiveMQ, you may disable the embeded web console from ActiveMA. By doing this, you can simple comment out the line:

Configuration Using NIO+SLL Connector

If ActiveMQ's transport connector uses nio+ssl, the connection url will have the following form:

"failover:(nio+ssl://activemq-host-name:61617,nio+ssl://activemq-host-name:61627)" \

For more details about how to configure ActiveMQ with nio+ssl, see my blog at:

In order to connect ActiveMQ brokers, we need update the catalina.sh file with the following: 

JMX_OPTS=" -Dwebconsole.jmx.url=service:jmx:rmi:///jndi/rmi://devrhamqv01:11099/jmxrmi"
JMX_OPTS="$JMX_OPTS,service:jmx:rmi:///jndi/rmi://devrhamqv01:12099/jmxrmi"

JAVA_OPTS="-Dwebconsole.type=properties "
#JAVA_OPTS="$JAVA_OPTS -Dwebconsole.jms.url=\"failover:(tcp://host-name:61616,tcp://hostname:61626)\" "
JAVA_OPTS="$JAVA_OPTS -Dwebconsole.jms.url=\"failover:(nio+ssl://host-name01:61617,nio+ssl://host-name:61627)\" "
JAVA_OPTS="$JAVA_OPTS -Dwebconsole.jms.user=admin"
JAVA_OPTS="$JAVA_OPTS -Dwebconsole.jms.password=admin"

JAVA_OPTS="$JAVA_OPTS -Djavax.net.ssl.keyStore=/opt/app/activemq/cluster/master-slave/north/conf/shared.ks "
JAVA_OPTS="$JAVA_OPTS -Djavax.net.ssl.keyStorePassword=amqadmin@ "
JAVA_OPTS="$JAVA_OPTS -Djavax.net.ssl.trustStore=/opt/app/activemq/cluster/master-slave/north/conf/shared.ks "

JAVA_OPTS="$JAVA_OPTS $JMX_OPTS "
JAVA_OPTS="$JAVA_OPTS -Dwebconsole.jmx.user=admin "
JAVA_OPTS="$JAVA_OPTS -Dwebconsole.jmx.password=activemq"

Monitoring ActiveMQ In Action

Now lets start tomcat by running startup.sh. Make sure the ActiveMQ-WEB is load successfully. You can check the log file at /opt/app/amq/tomcat/current/logs/catalina.out. And the following lines should be shown: 

25-Jan-2015 04:18:48.567 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive /opt/app/amq/tomcat/apache-tomcat-8.0.17/webapps/activemq-web-console-5.10.0.war has finished in 2,863 ms

Open the web broser with the following URL: 

http://sandboxfusev01:8080/activemq-web-console-5.10.0/

As shown in the above picture, tomcat container has successfully connected to the north ActiveMQ node on the host SANDBOXFUSEV01

we verify north node is running be the following:

[amq@SANDBOXFUSEV01 bin]$ ./north status
INFO: Using default configuration
(you can configure options in one of these file: /etc/default/activemq /home/amq/.activemqrc)

INFO: Invoke the following command to create a configuration file
/opt/app/amq/apache-activemq-5.10.0/bin/activemq setup [ /etc/default/activemq | /home/amq/.activemqrc ]

INFO: Using java '/usr/bin/java'
ActiveMQ is running (pid '17720')
[amq@SANDBOXFUSEV01 bin]$

We know that is correct. Now let's stop the north node and see what happen. After refresh the web browser I see the following:

The new master ActiveMQ node is north from SANDBOXFUSEV02. I did not find a particular order about which slave will become next master. It seems whichever slave grab the lock at a particular moment after the master dies, will be come the new master.

Enable TLS/SSL

For details about configuring TLS/SSL, please refer: https://tomcat.apache.org/tomcat-8.0-doc/ssl-howto.html. Here are the procedure I have executed:

  1. generate private key using keytool
  2. put the keystore $HOME/$HOSTNAME.ks
  3. update $TOMCAT_HOME/conf/server.xml
  4. open the port 8443 from firewall using system-config-firewall

And here is the connector configuration:

Summary

We have successfuly configured the tomcat to monitor ActiveMQ instance. The good thing about using Tomcat we container to monitoring activemq instance is that it automatically connects to the master node. The installation and configuration are fairly straightforward.
at 2 comments:

Monitor ActiveMQ Nodes Using JMX

Introduction

In my previous post, http://ggl-consulting.blogspot.com/2015/01/configure-activemq-for-high-available.html, I have demonstrate in detail about how to configure ActiveMQ for high-availability using Master-Slave Paradigm. In this post, I will demonstrate the process to setup jVisualVM to monitoring all ActiveMQ instances in the cluster.

jVisualVM Setup

jVisualVM is a tool from JDK. This is a very powerful visual tool to monitor java process, create thread and memory dumps, display dashboard, etc. It is much powerful than jconsole which also comes with JDK.

To start jVisualVM, simple execute the following command [assume JDK is installed and jVisualVM is in your path]: 

jvisualvm

As shown in the above figure, if you run for the first time the jvisualvm, you will need to install MBean plugin by Tool-->Available Plugins. In my case, I have already installed the plugin, so that the available plugins only show 15 items.

Once the MB plugin is installed, we can add jmx connections. In my case, I will need to add the following 4 connections:

service:jmx:rmi:///jndi/rmi://sandboxfusev01:11099/jmxrmi
service:jmx:rmi:///jndi/rmi://sandboxfusev01:12099/jmxrmi
service:jmx:rmi:///jndi/rmi://sandboxfusev02:11099/jmxrm
service:jmx:rmi:///jndi/rmi://sandboxfusev02:12099/jmxrm

After that I can see connections as the following

If I browe all 4 connections, I can see only the north ActiveMQ node from sandboxfusev01 is active. The other three node will show information like this:

This is indicating that 3 ActiveMQ nodes are in standby modes or they are slaves.

at No comments:

Thursday, January 22, 2015

Configure ActiveMQ For High Available Using Master-Slave Paradigm With Shared File System

Introduction

Enterprise messaging with ActiveMQ is very popular nowadays. It is very matured platform and it is free! Currently, the 5.10.0 release is available at http://activemq.apache.org/activemq-5100-release.html.
In this blog, I will explain the detailed configuration about how to configure ActiveMQ instances to achieve high availability using master-slave paradigm with shared file system. The system diagram is shown below
As shown in the above diagram, I use two RedHat Linux boxes to configure 4 ActiveMQ broker nodes. All of these nodes share one file system [a directory where all nodes point to]. In general, we do not configure 2 activemq instance on one linux server in the production environment. Two or three ActiveMQ broker nodes to form a Master-Slave clustering is good enough to achieve high availability. You may create multiple clusters. I am going to explain that in later post to use Network Of Brokers with multiple clusters.
Enough topology, now let's create the nodes.

Master-Slave Configurations

We install the apache-activemq-5.10.0 at /opt/app/amq, then create 2 nodes name north and south with the following command: 
cd /opt/app/amq/apache-activemq-5.10.0/bin
[amq@SANDBOXFUSEV01 bin]$ ./activemq-admin create ../../north
[amq@SANDBOXFUSEV01 bin]$ ./activemq-admin create ../../south
Now we have the following file structures

[amq@SANDBOXFUSEV01 amq]$ pwd
/opt/app/amq

[amq@SANDBOXFUSEV01 amq]$ ls -lart
total 24
drwxr-xr-x.  3 root root     4096 Jan 14 10:27 ..
lrwxrwxrwx.  1 amq  amq        23 Jan 14 10:29 activemq -> apache-activemq-5.10.0/
drwxr-xr-x. 11 amq  amq      4096 Jan 14 10:42 apache-activemq-5.10.0
drwxrwxr-x.  3 amq  amq      4096 Jan 20 12:38 tomcat
drwxrwxr-x.  4 amq  amq      4096 Jan 24 14:04 north
drwxr-xr-x.  6 amq  amqadmin 4096 Jan 24 14:06 .
drwxrwxr-x.  4 amq  amq      4096 Jan 24 14:06 south

Configure North

Now we need to modify the activemq.xml file with the following changes: ]
 17     
 
 47         
 48             
 49         
 
 58         
 59             
 60         
 
 88         
 89             
 90             
 91             
 92             
 93             
 94             
 95         
            ....
The line with useJmx="true" is for enable JMX and the line
is to tell ActiveMQ to use port 11099 as JMX connection port. The rmiServerPort must be defined, otherwise, the remote access will not be possible. The reason for this is that most enterprise environment block the unspecified port.

Configure South

The South instance will be similar as the North. We just need to change the port number. For the completeness, here are the changes:
 
 17     
 
 47         
 48             
 49         
 
 58         
 59             
 60         
 
 88         
 90             
 91             
 92             
 93             
 94             
 95         
            ....

Do the same for the second node. The start the north node on the first server by executing the following command:

Master-Slave In Action

Now, we have done all the configuration changes. We can start all the process and to see how this paradigm of high availability works. 
cd /opt/app/amq/north/bin
./north start

I have created two aliases which are very useful:

alias amqjava='ps -eaf | egrep java | egrep -v grep | egrep amq'
alias amqport='netstat -anp | egrep tcp | egrep java | egrep -v grep'

Where amqjava is to check java process owned by amq user, and amqport is to check port information about amq java process. Run amqport command immediately after you start the amq java process. Here the port information you will see:

tcp        0      0 :::41882                    :::*                        LISTEN      17720/java
tcp        0      0 :::1883                     :::*                        LISTEN      17720/java
tcp        0      0 :::11099                    :::*                        LISTEN      17720/java
tcp        0      0 :::44444                    :::*                        LISTEN      17720/java
tcp        0      0 :::8161                     :::*                        LISTEN      17720/java
tcp        0      0 :::5672                     :::*                        LISTEN      17720/java
tcp        0      0 :::61613                    :::*                        LISTEN      17720/java
tcp        0      0 :::61614                    :::*                        LISTEN      17720/java
tcp        0      0 :::61616                    :::*                        LISTEN      17720/java
tcp        0      0 ::ffff:10.66.13.119:37444   ::ffff:10.66.13.119:44444   ESTABLISHED 17720/java
tcp        0      0 ::ffff:127.0.0.1:36187      ::ffff:127.0.0.1:11099      ESTABLISHED 17720/java
tcp        0      0 ::ffff:10.66.13.119:44444   ::ffff:10.66.13.119:37444   ESTABLISHED 17720/java
tcp        0      0 ::ffff:127.0.0.1:11099      ::ffff:127.0.0.1:36187      ESTABLISHED 17720/java

Another way to check these port is to use lsof command as shwon below:

[amq@SANDBOXFUSEV01 bin]$ lsof -P -iTCP
COMMAND   PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
java    17720  amq    9u  IPv6 1300784      0t0  TCP *:41882 (LISTEN)
java    17720  amq  111u  IPv6 1300785      0t0  TCP *:11099 (LISTEN)
java    17720  amq  113u  IPv6 1300786      0t0  TCP *:44444 (LISTEN)
java    17720  amq  125u  IPv6 1300795      0t0  TCP *:61616 (LISTEN)
java    17720  amq  126u  IPv6 1300796      0t0  TCP *:5672 (LISTEN)
java    17720  amq  127u  IPv6 1300797      0t0  TCP *:61613 (LISTEN)
java    17720  amq  128u  IPv6 1300798      0t0  TCP *:1883 (LISTEN)
java    17720  amq  129u  IPv6 1300800      0t0  TCP *:61614 (LISTEN)
java    17720  amq  136u  IPv6 1300811      0t0  TCP *:8161 (LISTEN)

At this time, the ActiveMQ north instance is running. I can verify this by open brower with URL of sandboxfusev01:8161. This is activemq embedded web console. The default password is admin/admin

Now lets start the second instance in the folder of /opt/app/amq/south

cd /opt/app/amq/south/bin
./south start

I can see the following stdout:

[amq@SANDBOXFUSEV01 bin]$ ./south start
INFO: Using default configuration
(you can configure options in one of these file: /etc/default/activemq /home/amq/.activemqrc)

INFO: Invoke the following command to create a configuration file
/opt/app/amq/apache-activemq-5.10.0/bin/activemq setup [ /etc/default/activemq | /home/amq/.activemqrc ]

INFO: Using java '/usr/bin/java'
INFO: Starting - inspect logfiles specified in logging.properties and log4j.properties to get details
INFO: pidfile created : '/opt/app/amq/south/data/activemq-SANDBOXFUSEV01.pid' (pid '17879')

When I chack amqjava, I get the following

[amq@SANDBOXFUSEV01 bin]$ amqjava
amq      17720     1  0 18:52 pts/0    00:00:08 /usr/bin/java -Xms1G -Xmx1G -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config=/opt/app/amq/north/conf/login.config -Dcom.sun.management.jmxremote -Djava.awt.headless=true -Djava.io.tmpdir=/opt/app/amq/north/tmp -Dactivemq.classpath=/opt/app/amq/north/conf; -Dactivemq.home=/opt/app/amq/apache-activemq-5.10.0 -Dactivemq.base=/opt/app/amq/north -Dactivemq.conf=/opt/app/amq/north/conf -Dactivemq.data=/opt/app/amq/north/data -jar /opt/app/amq/apache-activemq-5.10.0/bin/activemq.jar start
amq      17879     1 24 19:24 pts/0    00:00:02 /usr/bin/java -Xms1G -Xmx1G -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config=/opt/app/amq/south/conf/login.config -Dcom.sun.management.jmxremote -Djava.awt.headless=true -Djava.io.tmpdir=/opt/app/amq/south/tmp -Dactivemq.classpath=/opt/app/amq/south/conf; -Dactivemq.home=/opt/app/amq/apache-activemq-5.10.0 -Dactivemq.base=/opt/app/amq/south -Dactivemq.conf=/opt/app/amq/south/conf -Dactivemq.data=/opt/app/amq/south/data -jar /opt/app/amq/apache-activemq-5.10.0/bin/activemq.jar start
[amq@SANDBOXFUSEV01 bin]$

This indicating there are two activemq java process are running with process ID of 17720 and 17879. Now lets check what port has been opened by these two java processes

[amq@SANDBOXFUSEV01 bin]$ amqport
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 :::41882                    :::*                        LISTEN      17720/java
tcp        0      0 :::1883                     :::*                        LISTEN      17720/java
tcp        0      0 :::11099                    :::*                        LISTEN      17720/java
tcp        0      0 :::44444                    :::*                        LISTEN      17720/java
tcp        0      0 :::44445                    :::*                        LISTEN      17879/java
tcp        0      0 :::8161                     :::*                        LISTEN      17720/java
tcp        0      0 :::12099                    :::*                        LISTEN      17879/java
tcp        0      0 :::5672                     :::*                        LISTEN      17720/java
tcp        0      0 :::36010                    :::*                        LISTEN      17879/java
tcp        0      0 :::61613                    :::*                        LISTEN      17720/java
tcp        0      0 :::61614                    :::*                        LISTEN      17720/java
tcp        0      0 :::61616                    :::*                        LISTEN      17720/java
[amq@SANDBOXFUSEV01 bin]$

Interestingly, I see only 44445 and 12099 ports opened by the second java process. You may expect 61626 and other ports are open. This is because the south instance is in standby mode. Now if I tail the log at /opt/app/amq/south/data/activemq.log, if the following output:

[amq@SANDBOXFUSEV01 bin]$ tail -10f ../data/activemq.log
2015-01-24 19:33:21,686 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-01-24 19:33:31,688 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-01-24 19:33:41,689 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-01-24 19:33:51,691 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-01-24 19:34:01,696 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-01-24 19:34:11,699 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-01-24 19:34:21,728 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-01-24 19:34:31,730 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-01-24 19:34:41,731 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-01-24 19:34:51,733 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-01-24 19:35:01,736 | INFO  | Database /amqdata/master-slave/data/kahadb/lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/amqdata/master-slave/data/kahadb/lock' could not be locked. | org.apache.activemq.store.SharedFileLocker | main

As you can see, every 10 seconds, the south instance is throwing an IOException sayiing the lock at '/amqdata/master-slave/data/kahadb/lock' is not accessable. This is exactly what we want. This means the south instance is a slave node now. For the purpose of showing you extra information, I can use lsof utility to check who is locking the file of '/amqdata/master-slave/data/kahadb/lock' by the following command:

[amq@SANDBOXFUSEV01 kahadb]$ cd /amqdata/master-slave/data/kahadb
[amq@SANDBOXFUSEV01 kahadb]$ ls
db-2.log  db.data  db.redo  lock
[amq@SANDBOXFUSEV01 kahadb]$ lsof | egrep lock | egrep java
java      17720       amq  118uw     REG               0,21        0 4321950309 /amqdata/master-slave/data/kahadb/lock
[amq@SANDBOXFUSEV01 kahadb]$ amqjava
amq      17720     1  0 18:52 pts/0    00:00:09 /usr/bin/java -Xms1G -Xmx1G -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config=/opt/app/amq/north/conf/login.config -Dcom.sun.management.jmxremote -Djava.awt.headless=true -Djava.io.tmpdir=/opt/app/amq/north/tmp -Dactivemq.classpath=/opt/app/amq/north/conf; -Dactivemq.home=/opt/app/amq/apache-activemq-5.10.0 -Dactivemq.base=/opt/app/amq/north -Dactivemq.conf=/opt/app/amq/north/conf -Dactivemq.data=/opt/app/amq/north/data -jar /opt/app/amq/apache-activemq-5.10.0/bin/activemq.jar start
amq      17879     1  0 19:24 pts/0    00:00:03 /usr/bin/java -Xms1G -Xmx1G -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config=/opt/app/amq/south/conf/login.config -Dcom.sun.management.jmxremote -Djava.awt.headless=true -Djava.io.tmpdir=/opt/app/amq/south/tmp -Dactivemq.classpath=/opt/app/amq/south/conf; -Dactivemq.home=/opt/app/amq/apache-activemq-5.10.0 -Dactivemq.base=/opt/app/amq/south -Dactivemq.conf=/opt/app/amq/south/conf -Dactivemq.data=/opt/app/amq/south/data -jar /opt/app/amq/apache-activemq-5.10.0/bin/activemq.jar start
[amq@SANDBOXFUSEV01 kahadb]$

From above command and result, I can see that the java process with ID of 17720 is holding the lock at /amqdata/master-slave/data/kahadb/lock.

In the same maner, we can start the two ActiveMA instance in the second server. We have proved the only one instance of activemq node in the cluster is holding the lock and function

Summary

So far we have covered the basic configuration of ActiveMQ for high availability using the Master-Slave paradigm with shared file system. In this paradigm, only master will grab the lock all the ActiveMA instances in the same cluster will be in a standby mode. We call them slaves. I also explained how to setup JMX and how to check ports which are ready for connections.

Before we further explain how Master-Slave paradigm works, I would like demonstrate how to use JVisualVM and Tomcat container to monitor ActiveMA instances (nodes). I will cover these two topics in the next two blogs.

at 1 comment:

Tuesday, January 13, 2015

ActiveMQ 5.10.0 Installation And Configuration

Introduction

In my previous ActiveMQ introduction, I have explained the installation process, how to check what ports are used by ActiveMQ, and how to enable JMX. In this post, I am going to further explain the ports used by ActiveMQ. However the main purpose of this post to explain how to run the examples coming with ActiveMQ. This technique is important for the future discuss on how to set up high availability features with ActiveMQ, so that we can verify the configuration.

In the coming few posts, I will explain the enterprise messaging configuration for high availability with:

  1. Master-Slave HA with Share File System
  2. Use Tomcat Web Container to Host ActiveMQ Web Console
  3. Bridging the Network Broker
  4. Network Brokers
  5. Best Practices

In the future posts, I will explain java programming techniques for writing producer and consumers for java developers, as well as advanced programing for enterprise integration using Apache Camel and ActiveMQ. Thus, we will study these examples in detail.

HTT

Now first lets discuss a bit the transport connectors come with the default configuration in activemq.xml

Default Ports Used By ActiveMQ

Take a look at activemq.xml, you will find this xml segment:
All these protocols can be found at activemq.apache.org website. ActiveMQ support many other transport connectors, such as VM, SSL, NIO, NIO+SSL, HTTP, HTTPS, etc. The details can be found at http://activemq.apache.org/configuring-version-5-transports.html

Run ActiveMQ Examples

ActiveMQ comes with many examples. For easy testing, we can run the examples at examples/openwire/swissarmy. Under the dir, there is a readme file. Typically, we will use the following lines for the producer and consumers:

    ant consumer -Durl=tcp://localhost:61616 -Dtopic=true -Ddurable=true -Dmax=5000
    ant producer -Durl=tcp://localhost:61616 -Dtopic=true -DtimeToLive=30000 -Dmax=5000

Make sure ant is installed and the ANT_HOME and PATH are set properly. In the following post about ActiveMQ High Available Configuration with Master-Slave, I will cover more about how to run these examples.

at No comments:

Saturday, December 20, 2014

Control Multiple MacBook Pro and iMacs with One Keyboard and Mouse

Summary

  • teleport is absolutely free
  • It is very easy to install and configure. Thank you Julien Robert!
  • It allows to control multiple machine using one pair of keyboard and mourse
  • You can drag and drop files from one machine to another. Beautiful!

Introduction

For those people like me who have at least one MacBook pro and iMac, thunderbolt display, it is desirable to use one keyboard and one mouse to control both machines. And to drag and drop files from one mac to another. Teleport is the solution! Teleport is developed at Abyssoft by Julien Robert. The current version is 1.1.3.

Download Teleport

The zip format file can be downloaded from http://www.abyssoft.com/software/. you can unzip the file teleport.zip by the following command on terminal:
unzip teleport.zip
After you unzip the file, a folder of teleport will be created with the following contents:


As you can see, there are 2 files underneath it. One is a pdf file with brief description on how install and configure the teleport and the other is a folder contain all the contents of the software.

Remember, you have download and configure the teleport on all the mac machines you want to control

The current version does not work out of box you have to perform the following procedures

  • In Finder navigate to teleport folder
  • Right or control click for context menu and "Show Contents"
  • Navigate to "Resources" folder and copy "teleportd.app" to your desktop (don't move it just copy it)
  • double click teleport.prePane
  • check the "activate" box
  • Go to Security & Privacy to Accessibility area
  • Hit the plus at the bottom of the list and navigate to the desktop and add teleportd.app

After the configuration, you should see the following in your System Preferences panel:

Once you have perform this procedures on all of you machines, you should be able to control your machine with only one keyboard and one mouse. It seems the last activated machine will be the master. Now you can drag and drop files from one machine to another. It is really beautiful!

If you have 2 sets of keyboard and mouse, and you want to control each other, you can set the layout the same for both machine as shown in the following figure.

at No comments:

Tuesday, December 2, 2014

NEXUS - A Kick Start - Install Configure And Deploy Maven Artifact

Introduction

Nexus is the best java artifact repository management software so far as I know. Enterprise software development shop should use repository management system to proxy the outside world repository and also to manage the life cycle of their internal software.
This article will walk you through the fundamental process to install, start Nexus, and deploy an maven artifact to the Nexus repository. For the complete reference, you may take a look at this reference book. Personally, I don't like this kind of reference. Nowadays, who has the time to read through the whole books?

Download and Install Nexus OSS

  • Go to this page to download the Nexus OSS
  • Download nexus-2.11.0-02-bundle.tar.gz (this is the current version)
  • cd ; mkdir nexus ; cd nexus
  • tar xvzf ~/Downloads/nexus-2.11.0-02-bundle.tar.gz
Now I can the the following contents:
[/Users/Gary2013/nexus/nexus-2.11.0-02]$ ls
LICENSE.txt NOTICE.txt bin  conf  lib  logs  nexus  tmp
At this point the installation is done. we can start the Nexua OSS by: 
cd bin
./nexus start

Configure Nexus OSS

By default, Nexus use http port 8081. You can open a browser and go to http://localhost:8081/nexus Once the default Nexus page is rendered, you can login in using the default admin: 
username: admin
password: admin123
Nexus has preconfigured a deployment user with: 
username: deployment
password: deployment123
First thing I did is to add a new uer [from left panel -> security] as shown in the following figure:

Configure Maven

In order to deploy java artifacts to Nexus Repository, we will need to update maven configuration file ~/.m2/setttings.xmls 

        
                deployment
                deployment
                deployment123
        
        
                gary
                gary
                gary1234

Project Configuration To Use Local Nexus

After Nexus starts, we need to configure our project to be deployable to the Nexus repository. Here is all we need to do in our pom.xml file


 4.0.0
 com.ggl.fuse
 fuse.message
 0.0.2-SNAPSHOT

 
  
   deployment
   http://localhost:8081/nexus/content/repositories/releases
  
  
   deployment
   http://localhost:8081/nexus/content/repositories/snapshots
  
 
 

 
  
   
    org.sonatype.plugins
    nexus-staging-maven-plugin
    1.6.3
    
     
      default-deploy
      deploy
      
       deploy
      
     
    

    
     deployment
     http://localhost:8081/nexus/
     
     true

If you look at the Nexus repository, you will find releases and snapshots as shown in the following figure:

The question is how does maven know into which repository the artifacts should be put? The answer is that in our version tag. If we put the version like:

0.0.2-SNAPSHOT

The maven will put the artifact into snapshots repository. If the version has no SNAPSHOT, the artifacts will be put into release repository

at No comments:
Subscribe to: Posts (Atom)

Anypoint Studio Error: The project is missing Munit lIbrary to run tests

Anypoint Studio 7.9 has a bug. Even if we following the article: https://help.mulesoft.com/s/article/The-project-is-missing-MUnit-libraries-...