Deploy Mule 4 Application To Anypoint Runtime Fabric Using Maven Plugin

Introduction

In CI/CD process, it is very common to use Mule Maven plugin to build and deploy application to the Cloudhub, on-premise private cloud like AWS, AZure, Google Cloud, etc. Since Mule 4, a lot of changes related to the deployment has changed, in particular, related to the Mule Runtime Fabric (RTF). Actually, RTF is a completely new infrastructure for Mule application deployment. I will cover more on that topic later. In this article, I am going to cover the following topics related to the deployment to Anypoint Runtime Fabric (RTF):

1. Prepare pom.xml setup to deploy mule project to Anypoint RTF
2. Encrypt password
3. Troubleshooting

If everything works, at the end, we should be able to achieve the following goals:

• deploy mule projects (assets) to Anypoint Exchange
• deploy mule projects to Anypoint Runtime Fabric

In order for mule maven plugin to work, we need change the following two files:

• pom.xml
• ./m2/settings.xml

The complete project for this article can be find at my github repository

Settings.xml

In order to deploy mule applications using mule maven plugin, we need to set the Anypoint credentials, which are the ones we use to login to anypoint portal (http://anypoint.mulesoft.com). The best way to do this is to add server information in the .m2/settings.xml as the following:

        
                
                      ExchangeRepository
                      gary_liu_client
                      {5XLgXNDBGBIHa99xNAyJ6gL+ZxyUiyIJNHWu0H7Ctew=}
                
        

The password is encrypted. To encrypt password we need to add another file namely: ~/.m2/settings-security.xml:

  {Vq5Aso1ZkO4HdJrUscJTZEii4BcFy+khGiGxDNVNgc4=}

The encrypt master password in the above is created by the following commands:

$ mvn --encrypt-master-password MasterPassword
{+4nnH6EW9HcHAHYBGnloFCAZZHSC4W3Xp9Zls0LvBqk=}

Once we encrypted the master password, we can encrypt the password to the anypoint portal as the following:

mvn --encrypt-password AnypointPortalPassword

For more details about the maven password encryption, you may refer to the following page: https://maven.apache.org/guides/mini/guide-encryption.html

pom.xml

 4.0.0

 fea874ca-11d9-4779-b1ce-90d49f738259
 mule-maven-plugin
 1.0.2
 mule-application

 mule-maven-plugin

 
  UTF-8
  UTF-8
  3.2.3
  https://anypoint.mulesoft.com
  MC
  QA

  4.1.4
  gary-deployment
  100m
  500Mi
 

 
  
   
    org.mule.tools.maven
    mule-maven-plugin
    ${mule.maven.plugin.version}
    true
    
     
      ${anypoint.uri}
      ${anypoint.provider}
      ${deployment.environment}
      ${deployment.target}
      ${app.runtime}
      ExchangeRepository
      ${app.name}
      
       ${deployment.replica}
       ${app.cores}
       ${app.memory}
      
     
     mule-application
    
   

   
    org.codehaus.mojo
    properties-maven-plugin
    1.0.0
    
     
      initialize
      
       read-project-properties
      
      
       
        ${maven.properties}
       
      
     
    
   
  
 

 
  
   ExchangeRepository
   Corporate Repository
   https://maven.anypoint.mulesoft.com/api/v1/organizations/${groupId}/maven
   default
  
 

 
           ......
 
        ......

The details are explained in the next section. But one item I must point out here: <groupId>fea874ca-11d9-4779-b1ce-90d49f738259</groupId>. The groupId is your Anypoint platform organization ID.

Explanations

The configuration in both pom.xml and settings.xml are pretty straightforward. Few items worths to explain. First, what is the latest version of mule maven plugin? This can be found at: https://docs.mulesoft.com/release-notes/mule-maven-plugin/mule-maven-plugin-release-notes. Currently, the latest verson is 3.2.3.

Secondly, I use another plugin in addition to the mule maven plugin, namely, properties-maven-plugin. This plugin allow us to pass a property file to the pom.xml.

   
    org.codehaus.mojo
    properties-maven-plugin
    1.0.0
    
     
      initialize
      
       read-project-properties
      
      
       
        ${maven.properties}
       
      
     
    
   

Note that in the configuration, we put ${maven.properties} variable. This allow as to pass the property file as:

mvn clean deploy -DmuleDeploy -Dmaven.properties=src/main/resources/mule.rtf.deploy.properties

Thirdly, note that in the mule maven plugin, I use <server>ExchangeRepository</server> as shown below. The ExchangeRespository is the server defined in the ~/.m2/settings.xml

    
     
      https://anypoint.mulesoft.com
      MC
      QA
      qa-azure-rtf
      4.1.4
      ExchangeRepository
      ${app.name}
      
       1
       100m
       500Mi
      
     
     mule-application
    

The details for the parameters can be referred at: https://docs.mulesoft.com/mule-runtime/4.1/runtime-fabric-deployment-mmp-reference.

Publish Assets To Exchange

In order to deploy mule application to RTF using Mule Maven Plugin, we must publish the application (asset to Anypoint Exchange). To me this is extra step unnecessary. To publish an artifact (asset) to the Anypoint Exchange, we run the following command:

mvn clean package deploy -Dmaven.properties=src/main/resources/mule.rtf.deploy.properties

After it completes the publishing, you can verify the result by the Anypoint Port using the following web address:
https://anypoint.mulesoft.com/exchange/{groupId}/{artifactId}
Here is an my example:
https://anypoint.mulesoft.com/exchange/fea874ca-11d9-4779-b1ce-90d49f738259/mule-maven-plugin/

Deploy Application To RTF

Deployment to RTF could be slow and sometimes could take very long time if it failed. I think the plugin should be improved on this by using asynchronous deployment instead of waiting and constantly checking with the runtime manager. Any way the deployment to RTF can be done using the following command:

mvn clean package deploy -DmuleDeploy -Dmaven.properties=src/main/resources/mule.rtf.deploy.properties

Take Aways

The commands used in the process:

mvn --encrypt-master-password GaryLiu1234
mvn --encrypt-password GaryLiu1234
mvn clean package deploy-Dmaven.properties=src/main/resources/mule.rtf.deploy.properties
mvn clean package deploy -DmuleDeploy -Dmaven.properties=src/main/resources/mule.rtf.deploy.properties

Web address to check assets in the Anypoint Exchange in the Anypoint Portal:
https://anypoint.mulesoft.com/exchange/{groupId}/{artifactId} Key Considerations using the Mule Maven Plugin in the CI/CD:

• Make sure don't wait for the deployment to finish as it can take very long time
• There are rest API you can check the deployment process. You should use the APIs to check the deploy
• At the moment, anypoint-cli is not working for RTF

Mule 4 Integration With Cassandra

Introduction

Mule 4 Cassandra Database Connector, current version of 3.10, has made significant improvement over the previous version. It is fairly straight forward to setup the integration to the Cassandra Database using the connector. This article is an introduction of the Cassandra Connector to Cassandra cluster.

Cassandra Cluster Configuration

I have created 2 a two node cluster as shown the in following diagram. The details on how to setup the Cassandra clustering will be covered in another post. Basically, I opened the native transport port 9042, which is the default vale on both nodes.

We need to some initial setup using cqlsh too by run the following command:

$ cqlsh -u cassandra -p casandra
cassandra@cqlsh> create keyspace if not exists dev_keyspace with replication = {'class' : 'SimpleStrategy', 'replication_factor' : 2};

The above command will create a keyspace, namely dev_keyspace. We can query the keyspaces by the following command:

cassandra@cqlsh> desc keyspaces;

You should see the following:

system_schema  system      system_distributed
system_auth    dev_keyspace  system_traces

The next step is to create emp table by the following command:

cassandra@cqlsh> create table emp (empid int primary key, emp_first varchar, emp_last varchar, emp_dept varchar);

And insert a row:

create table emp (empid int primary key, emp_first varchar, emp_last varchar, emp_dept varchar);
insert into emp (empid, emp_first, emp_last, emp_dept) values (1, 'Gary','liu','consulting');

That is all and we are ready to do the integration.

Integration Using Mule 4 Cassandra Connector

Add Cassandra Connector

First, we need to search the Exchange and add the Cassandra Connection as shown in the following snapshot:

Create CassandraDB Config

Create a mule configuration, namely, global-config.xml. Then create CasandraDB Config as the following:

Enter the General setting as the following. Note: Leave the Host empty as we use cluster configuration.

Now, click the "Advanced Settings" tab and enter the information as the following:

As you can see, we can enter the ip addresses separated by comma. In this way, we can achieve high availability from client side of view. Now, we test the connectivity. If the port of 9042 is exposed correctly, it should work fine. I will explain more in the next article on how to make sure we expose the native transport port correctly.

Create Integration Flows

Read Flow

The read flow is very straight forward. The cassandra-db:select operation uses payload as the whole query. We just need to set payload. In this case, it is "select * from emp;"

<flow name="select-objecsFlow" doc:id="8bdcc4fe-cf4e-49be-b2f9-2dfacbddaf4b" >

<http:listener doc:name="Listener" doc:id="ab57d8ab-2623-468b-b635-a0c6efd6c829" config-ref="HTTP_Listener_config" path="/cassandra/emp"/>

<set-payload value="select * from emp;" doc:name="Set Payload"  />

<cassandra-db:select doc:name="Select"  config-ref="CassandraDB_Config_cluster" />

<ee:transform doc:name="Transform Message"  >

<ee:message >

<ee:set-payload ><![CDATA[%dw 2.0

output application/json

---

payload]]></ee:set-payload>

</ee:message>

</ee:transform>

<logger level="INFO" doc:name="Logger" doc:id="e6097436-1909-4d8b-8c81-27be82c11714" message="#[payload]" />

</flow>

</mule>

Insert A Row

To insert a row, we need to use insert operation as the following:

<flow name="insert-objecsFlow" doc:id="8bdcc4fe-cf4e-49be-b2f9-2dfacbddaf4b" >

<http:listener doc:name="Listener" doc:id="ab57d8ab-2623-468b-b635-a0c6efd6c829" config-ref="HTTP_Listener_config" path="/cassandra/emp/create"/>

<ee:transform doc:name="Transform Message" doc:id="0d484493-fa23-4a56-9547-58f0bb54dbd1" >

<ee:message >

<ee:set-payload ><![CDATA[%dw 2.0

output application/java

---

payload]]></ee:set-payload>

</ee:message>

</ee:transform>

<cassandra-db:insert table="emp" doc:name="Insert" doc:id="71cb7584-981a-4475-a839-ea82ed3b9832" config-ref="CassandraDB_Config_vm1" keyspaceName="rogers_dev"/>

<ee:transform doc:name="Transform Message"  >

<ee:message >

<ee:set-payload ><![CDATA[%dw 2.0

output application/json

---

payload]]></ee:set-payload>

</ee:message>

</ee:transform>

<logger level="INFO" doc:name="Logger" doc:id="e6097436-1909-4d8b-8c81-27be82c11714" message="#[payload]" />

</flow>

Currently, we can only insert one row at a time. To insert multiple rows, we can use for loop or use batch processes for large volumes. 

About CassandraDB Connector

The detailed information can be found at the following Mulesoft Document:

https://docs.mulesoft.com/connectors/cassandra/cassandra-connector

Important Cassandra Information

When you using cqlsh to connect Cassandra cluster, you should notice the following:

cqlsh -u cassandra -p cassandra
Connected to DevelopmentCluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.11.3 | CQL spec 3.4.4 | Native protocol v4]

The v4 is the current native protocol version, which is required to configure the connector.

Mule 4 : Dataweave 2 In Action - Use Function Modules

Introduction

In Mule 4, the mel2 functionalities are replaced by the Dataweave 2 ones. For those who worked in Mule 3, you will find the mule 4 ways of using function modules are much more advanced than the old way (see my post). In this article, I am going to present Mule 4 ways to define and use global functions.

Define Functions In Dataweave Modules

Here is my project layout:

As you can see, I place the function modules in the dir: src/main/resources/modules. And the dataweave function module is defined in CommonFunc.dwl.

%dw 2.0
fun concatName(aPerson) = aPerson.firstName ++ ' ' ++ aPerson.lastName

fun stringToDateUTC(dateString) = ((dateString as String {format: "yyyy-MM-dd'T'HH:mm:ss.SSSZ"} >> "UTC"
 )) as DateTime {format: "yyyy-MM-dd'T'HH:mm:ss.SSS"} 

I defined two functions: concatName, stringToDateUTC. The purposes are self-explained.

Use Function Modules

I place the normal dataweave modules in the dir of src/main/resources/dataweave. The usage of the functions is demonstrated at the following dwl file:

%dw 2.0
import * from modules::CommonFunc
output application/json
---
{
 "Name" : concatName(payload),
 "CreatedDate" : stringToDateUTC(payload.createdDate)
}

Here are few points:

• import functions
1. import modules::CommonFuc
2. import * from modules::CommonFunc
3. import concatName stringToDateUTC from modules::CommonFunc
• use functions in dwl
1. CommonFunc::concatName(payload)
2. concatName(payload)

For those who know python, you may find the syntax is very similar between the two languages, in terms of modules, and references.

Testing Data

Here is the complete flow:

Here is testing data:

{
 "firstName" : "Gary",
 "lastName" : "Liu",
 "createdDate":"2018-07-17T16:18:03+00:00"
}

Mule 4: Dataweave 2 In Action - Using Filter

Use Cases

The requirement is to update overall status to PARTIAL if there is any failure record in the payload, i. e. item.status = "Failed"

Input:

{
    "status": "SUCCESS",
    "items": [
        {
            "accountId": "10101xyzabc",
            "status": "Success"
        },
        {
            "accountId": "10102aaabbb",
            "status": "Failed"
        }
    ]
}

Expected Output:

{
    "status": "PARTIAL",
    "items": [
        {
            "accountId": "10101xyzabc",
            "status": "Success"
        },
        {
            "accountId": "10102aaabbb",
            "status": "Failed"
        }
    ]
}

Datawave 2.0

%dw 2.0
output application/json
---
{
 status: if (sizeOf(payload.items map($) filter ($.status == "Failed")) > 0) "PARTIAL" else "SUCCESS",
 items: payload.items 
}

Complete Flow

The following is the complete flow. Please note that I have commented out the transformation payload to java. This is a big improvement of Mule 4 over mule 3.

Key Learnings

1. sizeOf function
2. filter function
3. if else, in Dataweave 1.0, this will be when else

if else is a very good syntax improvement over Dataweave 1.0

Enable JMX Authencation And SSL For Mule Runtime

Introduction

In my previous blog, I demonstrated how to change mule application logging level dynamically by using JMX MBeans. In that blog, I skipped the procedure on how to enable SSL for JMX of Mule runtimes. Apparently, in production environment, we will need to enable both authentication and SSL for the security purpose.

I will demonstrate the details about enabling SSL for on-premises Mule Runtimes. I will use local generated Cert for demonstration purpose. You may need to authorized the cert for your organization, but the basic procedures are the same.

Generate Keystore and Truststore

On mule runtime server, execute the following commands:

mkdir ${MULE_HOME}/ssl
cd ${MULE_HOME}/ssl
keytool -genkey -alias tc401 -keyalg RSA -keystore tc401_keystore.jks
keytool -export alias tc401 -file tc401_cert -keystore tc401_keystore.jks
keytool -import -alias tc401 -keystore tc401_truststore.jks -file tc401_cert

The above commands will create keystore and truststore, which will be used by Mule Runtimes. To instruct a Mule Runtime to use the keystore and truststore, we need to update wrapper.conf file.

Configure Mule Runtime with Authentication and SSL

Add the following lines to ${MULE_HOME}/conf/wrapper.conf

wrapper.java.additional.50=-Dcom.sun.management.jmxremote=true
wrapper.java.additional.51=-Dcom.sun.management.jmxremote.port=1099
wrapper.java.additional.53=-Dcom.sun.management.jmxremote.access.file=%MULE_HOME%/conf/jmxremote.access
wrapper.java.additional.54=-Dcom.sun.management.jmxremote.password.file=%MULE_HOME%/conf/jmxremote.password
wrapper.java.additional.56=-Dcom.sun.management.jmxremote.authenticate=true
wrapper.java.additional.57=-Dcom.sun.management.jmxremote.ssl=true
wrapper.java.additional.58=-Djavax.net.ssl.keyStore=%MULE_HOME%/ssl/tc401_keystore.jks
wrapper.java.additional.59=-Djavax.net.ssl.keyStorePassword=changeme
wrapper.java.additional.60=-Djavax.net.ssl.trustStore=%MULE_HOME%/ssl/tc401_keystore.jks
wrapper.java.additional.61=-Djavax.net.ssl.trustStorePassword=changeme

Note that I use jmxremote.access and jmxremote.password for the user permission and authentication. The details can be refered in my last blog.

Start jvisualvm

jvisualvm -J-Djavax.net.ssl.trustStore=./tc401_truststore.jks -J-Djavax.net.ssl.trustStorePassword=changeme

The following snapshots shows how the page of login with SSL enabled.

Dynamically Change Mule Application Logging Level At Runtime

Introduction

In many situations, we need to change the logging level from WARN to DEBUG, then change it back to WARN after a period of time. There are few ways to do so as the following. Many unexperienced developers will change log4j2.xml file. For instance, if we want to log all the requests and responses for all HTTP Listener, we can change the log4j2.xml by add the following lines:

<AsyncLogger name="org.mule.module.http.internal.HttpMessageLogger" level="INFO"/>

<AsyncLogger name="com.ning.http" level="INFO" />

This approach works, but it is often very changing if not impossible. To change code in production environment should be considered as a last resort. There are other ways such as  command line, or use web services.
< However all the above mentioned approach requires more effort the JMX approach, which is simplest way in my humble opinion. This article will demonstrate how we can achieve this.

Enable JMX For Mule Runtime

To enable jmx, we will need to update wrapper.conf, which is at ${MULE_HOME}/conf/wrapper.conf. Add the following lines:

wrapper.java.additional.50=-Dcom.sun.management.jmxremote
wrapper.java.additional.50=-Dcom.sun.management.jmxremote=true
wrapper.java.additional.51=-Dcom.sun.management.jmxremote.port=1099
wrapper.java.additional.53=-Dcom.sun.management.jmxremote.access.file=%MULE_HOME%/conf/jmxremote.access
wrapper.java.additional.54=-Dcom.sun.management.jmxremote.password.file=%MULE_HOME%/conf/jmxremote.password
wrapper.java.additional.56=-Dcom.sun.management.jmxremote.authenticate=true
wrapper.java.additional.55=-Dcom.sun.management.jmxremote.ssl=false

The above configuration requires to create two files: jmxremote.access and jmxremote.password under ${MULE_HOME}/conf. The following are examples:

$cat jmxremote.access
admin readwrite
gary  readonly

$cat jmxremote.password
admin admin
gary gary

To enable ssl requires generate certificates. I will cover the topic later

Configure jvisualvm And Change Logging Level

jvisualvm comes with Java SDK. For the details about the setup you may refer to my blog . The most important thing is to make sure you install MBeans pluging. The following snapshot shows the details:

You can traverse to the component at org.apache.logging.log4j2, under Loggers, you can change any log4j2 bean's log level as shown in the following snapshot:

In practice, you can do a lot of more with jvisualvm to inspect the mule runtime. I will cover more in my later blogs.

Install anypoint-cli on-premise in Off-Line mode

Introduction

anypoint-cli is a very powerful tool, which can perform a lot of operations of mule application management, environment setup, etc. Mulesoft provides a good document online: https://docs.mulesoft.com/runtime-manager/anypoint-platform-cli. However, in many on-premise environment there is no direct access to outside world. We need to install anypoint-cli in the off-line mode. Mulesoft has a documentation on this: https://support.mulesoft.com/s/article/How-to-perform-offline-installation-of-Anypoint-CLI This post is to address the details of the deployment, which includes install node, configure anypoint-cli environment.

Make Sure Your Proxy Setup Is Correct

On-premise setup mule often requires proxy setup. Here is the example of the proxy configuration:

export https_proxy=http://functional-account-name:password@YOUR-DOMAIN.com:YOUR-PORT
export no_proxy=xyz.com,localhost,,192.168.0.0/16,127.0.0.1,.xyz.com,.abc.com

Install Node

The following link is very helpful: https://tecadmin.net/install-latest-nodejs-and-npm-on-centos/ Make sure you have root access.

Install anypoint-cli

Follow the mulesoft document. https://support.mulesoft.com/s/article/How-to-perform-offline-installation-of-Anypoint-CLI. Make sure use root.

  npm install -g npm-bundle
  npm-bundle --verbose anypoint-cli

The last command will download anypoint-cli tgz file: anypoint-cli-2.3.2.tgz Once the lastest anypoint-cli package is downloaded, you can copy it to target linux node and do the following (as sudo user, don't run as root directly):

  npm install -g /tmp/anypoint-cli-2.3.2.tgz

Configuration & Test Run

Put the following in you .anypoint_env.dev file:

ANYPOINT_ENV=dev
ANYPOINT_HOST=anypoint.mulesoft.com
ANYPOINT_USERNAME=anypoint-user-name
ANYPOINT_PASSWORD=password
ANYPOINT_ORG=org-name

You can create many file like this. When you work on one, say, dev environemtn, you can source the file of .anypoint_env.dev. Once you have completed the installation you can run the following command as normal user:

anypoint-cli runtime-mgr standalone-application list
ID      Name                                        Target ID Target Name           Status  Updated

2400923 gcc-mule-services-cache-management          507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400924 gcc-mule-services-cfgmgmt                   507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400925 gcc-mule-services-cfgmgmt-ptdata-consumer   507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400926 gcc-mule-services-cfgmgmt-ptupdate-consumer 507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400927 gcc-mule-services-datacapture               507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400928 gcc-mule-services-datastore-consumer        507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400929 gcc-mule-services-fileexchange-process      507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400930 gcc-mule-services-filemanager               507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400931 gcc-mule-services-fileupload                507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400932 gcc-mule-services-fileupload-process        507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400933 gcc-mule-services-processlog-consumer       507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400934 gcc-mule-services-sfg                       507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400935 gcc-mule-services-transformation-consumer   507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400936 gcc-mule-services-transporter               507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400937 gcc-mule-services-validation-consumer       507876    gcc-mule-dit1-cluster STARTED 5 hours ago