Mule 4: Dataweave 2 In Action - Using Filter

Use Cases

The requirement is to update overall status to PARTIAL if there is any failure record in the payload, i. e. item.status = "Failed"

Input:

?

1 2 3 4 5 6 7 8 9 10 11 12 13

{     "status": "SUCCESS",     "items": [         {             "accountId": "10101xyzabc",             "status": "Success"         },         {             "accountId": "10102aaabbb",             "status": "Failed"         }     ] }

Expected Output:

?

1 2 3 4 5 6 7 8 9 10 11 12 13

{     "status": "PARTIAL",     "items": [         {             "accountId": "10101xyzabc",             "status": "Success"         },         {             "accountId": "10102aaabbb",             "status": "Failed"         }     ] }

Datawave 2.0

?

1 2 3 4 5 6 7

%dw 2.0 output application/json --- {  status: if (sizeOf(payload.items map($) filter ($.status == "Failed")) > 0) "PARTIAL" else "SUCCESS",  items: payload.items }

Complete Flow

The following is the complete flow. Please note that I have commented out the transformation payload to java. This is a big improvement of Mule 4 over mule 3.

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

<flow name="use-filterFlow" doc:id="ddae1e7c-0e9c-424c-842a-5ac4ae018b21">   <http:listener doc:name="Listener" doc:id="f0a9f441-4585-4e81-969a-a2d10ad480d1" config-ref="HTTP_Listener_config" path="/mule4/poc/filter">    <ee:repeatable-file-store-stream>   </ee:repeatable-file-store-stream></http:listener> <!--     <ee:transform doc:name="Transform Message" doc:id="f471f35a-3050-43d8-99fe-dfd1fbb07292" >    <ee:message >     <ee:set-payload resource="dataweave/toJava.dwl" />    </ee:message>   </ee:transform>  -->    <ee:transform doc:name="Transform Message" doc:id="c9ef1500-033f-48f6-92f0-a760c75188d3">    <ee:message>     <ee:set-payload resource="dataweave/useFilter.dwl">    </ee:set-payload></ee:message>    <ee:variables>    </ee:variables>   </ee:transform>   <logger level="INFO" doc:name="Logger" doc:id="9958ba07-b13b-4bea-82ea-a28696437345" message="#[payload]">  </logger></flow>

Key Learnings

1. sizeOf function
2. filter function
3. if else, in Dataweave 1.0, this will be when else

if else is a very good syntax improvement over Dataweave 1.0

Enable JMX Authencation And SSL For Mule Runtime

Introduction

In my previous blog, I demonstrated how to change mule application logging level dynamically by using JMX MBeans. In that blog, I skipped the procedure on how to enable SSL for JMX of Mule runtimes. Apparently, in production environment, we will need to enable both authentication and SSL for the security purpose.

I will demonstrate the details about enabling SSL for on-premises Mule Runtimes. I will use local generated Cert for demonstration purpose. You may need to authorized the cert for your organization, but the basic procedures are the same.

Generate Keystore and Truststore

On mule runtime server, execute the following commands:

mkdir ${MULE_HOME}/ssl
cd ${MULE_HOME}/ssl
keytool -genkey -alias tc401 -keyalg RSA -keystore tc401_keystore.jks
keytool -export alias tc401 -file tc401_cert -keystore tc401_keystore.jks
keytool -import -alias tc401 -keystore tc401_truststore.jks -file tc401_cert

The above commands will create keystore and truststore, which will be used by Mule Runtimes. To instruct a Mule Runtime to use the keystore and truststore, we need to update wrapper.conf file.

Configure Mule Runtime with Authentication and SSL

Add the following lines to ${MULE_HOME}/conf/wrapper.conf

wrapper.java.additional.50=-Dcom.sun.management.jmxremote=true
wrapper.java.additional.51=-Dcom.sun.management.jmxremote.port=1099
wrapper.java.additional.53=-Dcom.sun.management.jmxremote.access.file=%MULE_HOME%/conf/jmxremote.access
wrapper.java.additional.54=-Dcom.sun.management.jmxremote.password.file=%MULE_HOME%/conf/jmxremote.password
wrapper.java.additional.56=-Dcom.sun.management.jmxremote.authenticate=true
wrapper.java.additional.57=-Dcom.sun.management.jmxremote.ssl=true
wrapper.java.additional.58=-Djavax.net.ssl.keyStore=%MULE_HOME%/ssl/tc401_keystore.jks
wrapper.java.additional.59=-Djavax.net.ssl.keyStorePassword=changeme
wrapper.java.additional.60=-Djavax.net.ssl.trustStore=%MULE_HOME%/ssl/tc401_keystore.jks
wrapper.java.additional.61=-Djavax.net.ssl.trustStorePassword=changeme

Note that I use jmxremote.access and jmxremote.password for the user permission and authentication. The details can be refered in my last blog.

Start jvisualvm

jvisualvm -J-Djavax.net.ssl.trustStore=./tc401_truststore.jks -J-Djavax.net.ssl.trustStorePassword=changeme

The following snapshots shows how the page of login with SSL enabled.

Dynamically Change Mule Application Logging Level At Runtime

Introduction

In many situations, we need to change the logging level from WARN to DEBUG, then change it back to WARN after a period of time. There are few ways to do so as the following. Many unexperienced developers will change log4j2.xml file. For instance, if we want to log all the requests and responses for all HTTP Listener, we can change the log4j2.xml by add the following lines:

<AsyncLogger name="org.mule.module.http.internal.HttpMessageLogger" level="INFO"/>

<AsyncLogger name="com.ning.http" level="INFO" />

This approach works, but it is often very changing if not impossible. To change code in production environment should be considered as a last resort. There are other ways such as  command line, or use web services.
< However all the above mentioned approach requires more effort the JMX approach, which is simplest way in my humble opinion. This article will demonstrate how we can achieve this.

Enable JMX For Mule Runtime

To enable jmx, we will need to update wrapper.conf, which is at ${MULE_HOME}/conf/wrapper.conf. Add the following lines:

?

1 2 3 4 5 6 7

wrapper.java.additional.50=-Dcom.sun.management.jmxremote wrapper.java.additional.50=-Dcom.sun.management.jmxremote=true wrapper.java.additional.51=-Dcom.sun.management.jmxremote.port=1099 wrapper.java.additional.53=-Dcom.sun.management.jmxremote.access.file=%MULE_HOME%/conf/jmxremote.access wrapper.java.additional.54=-Dcom.sun.management.jmxremote.password.file=%MULE_HOME%/conf/jmxremote.password wrapper.java.additional.56=-Dcom.sun.management.jmxremote.authenticate=true wrapper.java.additional.55=-Dcom.sun.management.jmxremote.ssl=false

The above configuration requires to create two files: jmxremote.access and jmxremote.password under ${MULE_HOME}/conf. The following are examples:

?

1 2 3 4 5 6 7

$cat jmxremote.access admin readwrite gary  readonly   $cat jmxremote.password admin admin gary gary

To enable ssl requires generate certificates. I will cover the topic later

Configure jvisualvm And Change Logging Level

jvisualvm comes with Java SDK. For the details about the setup you may refer to my blog . The most important thing is to make sure you install MBeans pluging. The following snapshot shows the details:

You can traverse to the component at org.apache.logging.log4j2, under Loggers, you can change any log4j2 bean's log level as shown in the following snapshot:

In practice, you can do a lot of more with jvisualvm to inspect the mule runtime. I will cover more in my later blogs.

Install anypoint-cli on-premise in Off-Line mode

Introduction

anypoint-cli is a very powerful tool, which can perform a lot of operations of mule application management, environment setup, etc. Mulesoft provides a good document online: https://docs.mulesoft.com/runtime-manager/anypoint-platform-cli. However, in many on-premise environment there is no direct access to outside world. We need to install anypoint-cli in the off-line mode. Mulesoft has a documentation on this: https://support.mulesoft.com/s/article/How-to-perform-offline-installation-of-Anypoint-CLI This post is to address the details of the deployment, which includes install node, configure anypoint-cli environment.

Make Sure Your Proxy Setup Is Correct

On-premise setup mule often requires proxy setup. Here is the example of the proxy configuration:

export https_proxy=http://functional-account-name:password@YOUR-DOMAIN.com:YOUR-PORT
export no_proxy=xyz.com,localhost,,192.168.0.0/16,127.0.0.1,.xyz.com,.abc.com

Install Node

The following link is very helpful: https://tecadmin.net/install-latest-nodejs-and-npm-on-centos/ Make sure you have root access.

Install anypoint-cli

Follow the mulesoft document. https://support.mulesoft.com/s/article/How-to-perform-offline-installation-of-Anypoint-CLI. Make sure use root.

?

1 2	npm install -g npm-bundle npm-bundle --verbose anypoint-cli

The last command will download anypoint-cli tgz file: anypoint-cli-2.3.2.tgz Once the lastest anypoint-cli package is downloaded, you can copy it to target linux node and do the following (as sudo user, don't run as root directly):

?

1	npm install -g /tmp/anypoint-cli-2.3.2.tgz

Configuration & Test Run

Put the following in you .anypoint_env.dev file:

?

1 2 3 4 5

ANYPOINT_ENV=dev ANYPOINT_HOST=anypoint.mulesoft.com ANYPOINT_USERNAME=anypoint-user-name ANYPOINT_PASSWORD=password ANYPOINT_ORG=org-name

You can create many file like this. When you work on one, say, dev environemtn, you can source the file of .anypoint_env.dev. Once you have completed the installation you can run the following command as normal user:

anypoint-cli runtime-mgr standalone-application list
ID      Name                                        Target ID Target Name           Status  Updated

2400923 gcc-mule-services-cache-management          507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400924 gcc-mule-services-cfgmgmt                   507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400925 gcc-mule-services-cfgmgmt-ptdata-consumer   507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400926 gcc-mule-services-cfgmgmt-ptupdate-consumer 507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400927 gcc-mule-services-datacapture               507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400928 gcc-mule-services-datastore-consumer        507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400929 gcc-mule-services-fileexchange-process      507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400930 gcc-mule-services-filemanager               507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400931 gcc-mule-services-fileupload                507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400932 gcc-mule-services-fileupload-process        507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400933 gcc-mule-services-processlog-consumer       507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400934 gcc-mule-services-sfg                       507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400935 gcc-mule-services-transformation-consumer   507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400936 gcc-mule-services-transporter               507876    gcc-mule-dit1-cluster STARTED 5 hours ago
2400937 gcc-mule-services-validation-consumer       507876    gcc-mule-dit1-cluster STARTED 5 hours ago