tag:blogger.com,1999:blog-60833925503272023922024-03-27T16:53:11.310-07:00Gary Liu's Technical Knowledge BaseGary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.comBlogger118125tag:blogger.com,1999:blog-6083392550327202392.post-12373260104404584002021-06-28T19:39:00.002-07:002021-06-28T19:39:22.362-07:00Anypoint Studio Error: The project is missing Munit lIbrary to run testsAnypoint Studio 7.9 has a bug. Even if we following the article: https://help.mulesoft.com/s/article/The-project-is-missing-MUnit-libraries-to-run-tests-Mule-4, it still would not fix the proble.
The only solution at the moment is to add the followiing dependency:
<pre class="brush:xml">
<dependency>
<groupId>org.mule.weave</groupId>
<artifactId>assertions</artifactId>
<version>1.0.2</version>
<scope>test</scope>
</dependency>
</pre>Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com1tag:blogger.com,1999:blog-6083392550327202392.post-59244244815012668092021-02-16T15:17:00.001-08:002021-02-16T15:20:17.369-08:00Access Multiple Github Accounts From One Computer<p>
Step 1: <a href="https://docs.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent">Step 1: Generate SSH and Add it to ssh-agent</a>
<p>
Note: if you have two github accounts, you need to generate two SSH Key
<p>
Step 2: <a href="https://gist.github.com/oanhnn/80a89405ab9023894df7">Configure ~/.ssh/config</a>
<p>
Note: In order to access the different account, you must use the hostname specified inside the ~/.ssh/config file.
Here is my example:
<pre class="brush:js">
$ cat .ssh/config
Host *
ServerAliveInterval 240
# garyliu1119 account
Host github1119.com
HostName github.com
AddKeysToAgent yes
IdentityFile ~/.ssh/id_ed25519
IdentitiesOnly yes
# garyliu19 account
Host github19.com
HostName github.com
AddKeysToAgent yes
IdentityFile ~/.ssh/id_rsa
IdentitiesOnly yes
</pre>
When I clone the github repo, I use the following command:
<pre class="brush:js">
git clone -b develop git@github19.com:garyliu19/FN-External-API.git
</pre>
Note: the domain name is: github19.com, instead of github.comGary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com1tag:blogger.com,1999:blog-6083392550327202392.post-28647190241873168162020-10-06T13:04:00.004-07:002020-10-06T13:04:35.539-07:00Dataweave 2.0: Convert Calendar Date To Julian DateA Julian date is commonly used in main frame. It's format is YYDDD. DDD is the date since begining of the year.
In DetaWeave 2.0 the code should be as the following:
<pre class="brush:sh">
julianNow: now() as String {format: "YYD"},
julianAt: "07-10-2020" as Date {format: "dd-MM-yyyy"} as String {format: "YYD"}
</pre>Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com1tag:blogger.com,1999:blog-6083392550327202392.post-78568869707108704512020-09-22T13:49:00.001-07:002020-09-22T13:49:53.844-07:00Mulesoft Send Email With CSV Attachment<h1>Introduction</h1>
This blog explain the tricks to convert csv data to plain text for the purpuse of send csv as attachment. The Mulesoft document can be found at <a href="https://docs.mulesoft.com/email-connector/1.2/email-send">here</a>
<pre class="brush:xml>
<flow name="attachment">
<file:read path="/file.json"/>
<email:send config-ref="config">
<email:to-addresses>
<email:to-address value="example@domain.com"/>
</email:to-addresses>
<email:body>
<email:content><![CDATA["<h1>Hello this is an important message</h1>"]]></email:content>
</email:body>
<email:attachments>
#[{
'csv-attachment' : payload
}]
</email:attachments>
</email:send>
</flow>
</pre>
<h1>Code Details</h1>
As shown in the follow snapshot, we need to perform two transformations:
<ul>
<li>from csv to base64 binary</li>
<li>from base64 binary to plain text>
</ul>
The challenge is that we cannot direct convert to "application/csv" to "text/plain".
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqoxIi0J4kCp7hBuBq-4MrQoPakPw5zbyHkBJHVM9hdVdVvxEqoLVJV4NxcA_f6YornI9urUyh1TG11q3I_shuqB1TdkZCr7uSG3kYo8oK-YeRzDUQnQBgVghxnqpmmGUvyUnpuEmNlvXp/s619/Screen+Shot+2020-09-22+at+3.39.23+PM.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="400" data-original-height="158" data-original-width="619" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqoxIi0J4kCp7hBuBq-4MrQoPakPw5zbyHkBJHVM9hdVdVvxEqoLVJV4NxcA_f6YornI9urUyh1TG11q3I_shuqB1TdkZCr7uSG3kYo8oK-YeRzDUQnQBgVghxnqpmmGUvyUnpuEmNlvXp/s400/Screen+Shot+2020-09-22+at+3.39.23+PM.png"/></a></div>
The details are shown below:
<H2>
Step 1: Initial code JSON to CSV
</H2>
<pre class="brush:xml">%dw 2.0
output application/csv separator=","
---
[
{
name: "Gary Liu",
ssn: "1234"
},
{
name: "John Smith",
ssN: "4567"
}
]
</pre>
<H2>Step 2: CSV to Base64 Binary</H2>
<pre class="brush:js">
%dw 2.0
import * from dw::core::Binaries
output text/plain
---
toBase64(write (payload, 'application/csv'))
</pre>
<H2>Step 3: Base64 Binary to CSV Plain Text</H2>
<pre class="brush:js">
%dw 2.0
import fromBase64 from dw::core::Binaries
output text/plain
---
fromBase64(payload)
</pre>
<H1>Take Aways</H1>
The line of thinking is that Mule runtime engine internally keeps that data as array of records. This java object cannot direct convert to plain text. The code is really mimic the way we store the data to file and then read the data from file to plain text.
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com1tag:blogger.com,1999:blog-6083392550327202392.post-35521589901718752642020-08-10T14:45:00.004-07:002020-08-10T14:45:59.732-07:00Mule Develop Tricks: Get The Phone Number<h1 style="text-align: left;"><span style="font-family: arial;">The Requirement</span></h1><div><span style="font-family: arial;">The phone number from customer web site is free string. It can take the following forms:</span></div><div><ol style="text-align: left;"><li><span style="font-family: arial;"> +1 (219) 555-8888</span></li><li><span style="font-family: arial;"> +1 219-555-8888</span></li><li><span style="font-family: arial;"> (219) 555 8888</span></li><li><span style="font-family: arial;"> 219 555 8888</span></li><li><span style="font-family: arial;"> 219 555-8888</span></li><li><span style="font-family: arial;"> 219-555-8888</span></li><li><span style="font-family: arial;"> -1-219-555-8888</span></li></ol><div><span style="font-family: arial;">There could be other form. </span></div></div><div><span style="font-family: arial;"><br /></span></div><div><span style="font-family: arial;">We need to use dataweave 2.0 to extract 10 digits of the phone number to: 2195558888.</span></div><h1 style="text-align: left;"><span style="font-family: arial;">Solution</span></h1><div><span style="font-family: arial;"><p class="p1" style="color: #969696; font-family: Helvetica; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">%dw<span class="s1" style="color: black;"> </span><span class="s2" style="color: #7d7d7d;">2</span><span class="s1" style="color: black;">.</span><span class="s2" style="color: #7d7d7d;">0</span></p><p class="p2" style="color: #030303; font-family: Helvetica; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;"><b>output</b><span class="s1" style="color: black;"> </span><b>application/json</b></p><p class="p3" style="font-family: Helvetica; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">---</p><p class="p3" style="font-family: Helvetica; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">{</p><p class="p3" style="font-family: Helvetica; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;"><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="s3" style="color: #3e59b0;">mobile</span>: (<span class="s4" style="color: #cb8c15; text-decoration-line: underline;">payload</span><span class="s5" style="text-decoration-line: underline;">.mobilePhone </span><span class="s4" style="color: #cb8c15; text-decoration-line: underline;">replace</span><span class="s5" style="text-decoration-line: underline;"> /([^0-9.]+)/</span> <span class="s6" style="color: #cb8c15;">with</span>(<span class="s7" style="color: #019722;">""</span>))[-<span class="s2" style="color: #7d7d7d;">10</span> <span class="s6" style="color: #cb8c15;">to</span> -<span class="s2" style="color: #7d7d7d;">1</span>]</p><p class="p3" style="font-family: Helvetica; font-size: 12px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 0px;">}</p></span></div><div><span style="font-family: arial;"><br /></span></div><h1 style="text-align: left;"><span style="font-family: arial;">Take Aways</span></h1><div><ol style="text-align: left;"><li><span style="font-family: arial;">dataweave built in function</span> <span style="color: #6aa84f; font-family: courier;">replace / / with ()</span></li><li><span style="font-family: arial;">substring </span><span style="color: #6aa84f; font-family: courier;">[-10 to -1]</span><span style="font-family: arial;"> take last 10 digits</span></li></ol></div>Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com1tag:blogger.com,1999:blog-6083392550327202392.post-41019474805481496942020-02-15T18:52:00.000-08:002020-02-15T18:52:32.273-08:00Prototyping Mule OAuth2 Client Application<H1>Introduction</H1>
This article explains how to prototype Mule OAuth 2.0 client application with grant type of "Client Credentials", which is most popular grant type for Mule integration.
Most modern APIs enforce OAuth2.0 security policies.
OAuth2.0 has the following Grant Types (<a href="https://oauth.net/2/">The detailed explanation can be found here</a>):
<ul>
<li>Authorization Code
<li>PKCE
<li>Client Credentials
<li>Device Code
<li>Refresh Token
<li>Legacy: Implicit Flow
<li>Legacy: Password Grant
</ul>
This post will cover the following topics:
<UL>
<li>Using postman to retrieve access token
<li>Prototyping retrieve access token using cURL
<li>Mule flow to retrieve access token
<li>Sample flow using the access token with caching scope
</UL>
<H1>OAuth 2.0 With Client Credential Grant Type</H1>
In order to access the OAuth2.0 enabled APIs, we first have to retrieve the access token from the Identity Providers. Then we can access the API by passing the access token.
The parameters for Client Credential can be the following:
<ul>
<li>grant_type (required)
<li>scope (optional)
<li>client_id (required)
<li>client_secret (required)
</ul>
In most cases, cilent_id and client_secret are encrypted as Basic Authentication. The encryption can be done using base64 or openssl as the following:
<pre class="brush:bash">
echo -n "${CLIENT_ID}:${CLIENT_SECRET}" | base64
</pre>
<pre class="brush:bash">
echo -n "${CLIENT_ID}:${CLIENT_SECRET}" | openssl enc -base64
</pre>
Note: the "-n" option of echo is for not printing the trailing newline character. This is very important.
<H1>Using Postman to Retrieve Access Token</H1>
The postman is the best tool to do prototyping for the OAuth 2 client. The following snapshot shows the setup of the Postman: <br>
for body:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgO07fUUJBC4YHsj0_05UQz1-pTJa26wwpeNRYaTZrRYn7gO0aAE5tPk2XxDtoP0j3ewPEctsZZjx3Am4QYfY2NPdfM3xtlHjTyzuQQOOMEOMNdSOdQUg2Ig-PoEXe4OQyFs5CI22Yh9sXS/s1600/Screen+Shot+2020-02-15+at+8.07.29+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgO07fUUJBC4YHsj0_05UQz1-pTJa26wwpeNRYaTZrRYn7gO0aAE5tPk2XxDtoP0j3ewPEctsZZjx3Am4QYfY2NPdfM3xtlHjTyzuQQOOMEOMNdSOdQUg2Ig-PoEXe4OQyFs5CI22Yh9sXS/s400/Screen+Shot+2020-02-15+at+8.07.29+PM.png" width="400" height="170" data-original-width="1264" data-original-height="536" /></a></div>
for Headers:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkVJf6tvGsxHdi-NT1weIVR7DCHtQ1n0y4I4gktDMSyxGGgt-Q1SLGNPzFJBuyfRFD6_KY1eot_F498TukakSgBU9_1q5e404uE98pBfTQv5obBdQmH8v4wp4KfXUXoXdmnOwEf1888sdD/s1600/Screen+Shot+2020-02-15+at+8.13.05+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkVJf6tvGsxHdi-NT1weIVR7DCHtQ1n0y4I4gktDMSyxGGgt-Q1SLGNPzFJBuyfRFD6_KY1eot_F498TukakSgBU9_1q5e404uE98pBfTQv5obBdQmH8v4wp4KfXUXoXdmnOwEf1888sdD/s400/Screen+Shot+2020-02-15+at+8.13.05+PM.png" width="400" height="141" data-original-width="1265" data-original-height="447" /></a></div>
for Authorization:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8in0lz3uOnQBB52UBNW70SPWme4mArFvxV9sL5qBTOrieod8Du1ii7HDBofnFWQQ5UXtUfwTcgUSZdwmp-4RBa6ZvWgfdjY7O9DMfCmIvSH37W85C_conlfxv7eBXlD6IuwKXVhG5GLo8/s1600/Screen+Shot+2020-02-15+at+8.14.03+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8in0lz3uOnQBB52UBNW70SPWme4mArFvxV9sL5qBTOrieod8Du1ii7HDBofnFWQQ5UXtUfwTcgUSZdwmp-4RBa6ZvWgfdjY7O9DMfCmIvSH37W85C_conlfxv7eBXlD6IuwKXVhG5GLo8/s400/Screen+Shot+2020-02-15+at+8.14.03+PM.png" width="400" height="179" data-original-width="1254" data-original-height="561" /></a></div>
<H1>cURL Solution</H1>
Once we have the postman, the solution of cUrl is very straight forward.
<pre class="brush:bash">
$ cat oauth2-client.sh
#!/bin/bash
#
CLIENT_ID=MY-CLIENT-ID-GOES-HERE-WITHOUT-QUOTE
CLIENT_SECRET=YOUR-CLIENT-SECRET-GOES-HERE-WITHOUT-QUOTE
OAUTH_HEADER=$(echo -n "${CLIENT_ID}:${CLIENT_SECRET}" | base64)
curl -d "grant_type=client_credentials&scope=https://graph.microsoft.com/.default" \
-H "Content-Type: application/x-www-form-urlencoded" \
-H "Authorization: Basic ${OAUTH_HEADER}" \
-XPOST https://login.microsoftonline.com/keurig.onmicrosoft.com/oauth2/v2.0/token
</pre>
<H1>Mule Application Solution - Retrieve access_token</H1>
The mule application flow for retrieving access token is the following:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-XkmTSQn023ASILVP_-urManw8Q0d5FXjPRH1OoN5YfKzpCczddXN4TNzttgQ745hdzWPOq5lpCMrfCOkHUTv33ASdoNZKeXSVvaYbXxvtDrkFvXr-_Ib1Hb5s-z-NqdSjilMTwQ2wrdd/s1600/Screen+Shot+2020-02-15+at+8.20.52+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-XkmTSQn023ASILVP_-urManw8Q0d5FXjPRH1OoN5YfKzpCczddXN4TNzttgQ745hdzWPOq5lpCMrfCOkHUTv33ASdoNZKeXSVvaYbXxvtDrkFvXr-_Ib1Hb5s-z-NqdSjilMTwQ2wrdd/s400/Screen+Shot+2020-02-15+at+8.20.52+PM.png" width="400" height="149" data-original-width="434" data-original-height="162" /></a></div>
The Data-Weave transformation code is the following:
<pre class="brush:js">
%dw 2.0
output application/x-www-form-urlencoded
---
{
grant_type: "client_credentials",
scope: "https://graph.microsoft.com/.default"
}
</pre>
The request configuration is as the following:
<pre class="brush:xml">
<http:request method="POST" doc:name="Request" doc:id="65997138-84c0-48b3-8347-68abf386b3a1"
config-ref="HTTP_Request_configuration"
path="/keurig.onmicrosoft.com/oauth2/v2.0/token">
<http:headers ><![CDATA[#[output application/java
---
{
"Content-Type" : "application/x-www-form-urlencoded"
}]]]></http:headers>
</http:request>
</pre>
The HTTPS connector configuration referred in the request is the following:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxdYYiIVrgauemeJhpbd2M1aHDXYBVOlA83M3BLyzttdegysEdmOeEVj10_V8yup8LqiCZBUgckBNRJqm-YIUeXcMFu2HPxRQ1tgwRleUdkgYgAt3FnRwGv_rP1M5Yu9LPCPvBRgicXJJp/s1600/Screen+Shot+2020-02-15+at+8.27.49+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxdYYiIVrgauemeJhpbd2M1aHDXYBVOlA83M3BLyzttdegysEdmOeEVj10_V8yup8LqiCZBUgckBNRJqm-YIUeXcMFu2HPxRQ1tgwRleUdkgYgAt3FnRwGv_rP1M5Yu9LPCPvBRgicXJJp/s400/Screen+Shot+2020-02-15+at+8.27.49+PM.png" width="371" height="400" data-original-width="932" data-original-height="1005" /></a></div>
The xml configuration is the following:
<pre class="brush:xml">
<http:request-config name="HTTP_Request_configuration" doc:name="HTTP Request configuration" doc:id="fb6e2c7d-010f-4141-bafe-a4a50c4fc540" >
<http:request-connection protocol="${oauth2.protocol}" host="${oauth2.host}" port="${oauth2.port}" >
<reconnection >
<reconnect frequency="5000" />
</reconnection>
<http:authentication >
<http:basic-authentication username="${secure::oauth2.client.id}" password="${secure::oauth2.client.secret}" />
</http:authentication>
</http:request-connection>
</http:request-config>
</pre>
As you can see, we pass the client_id and client_secret as the username and password of the basic authentication. This is just base64 encoded string.
Here is an example of the response from the retrieval of access token.
<pre class="brush:js">
{
"token_type": "Bearer",
"expires_in": 3599,
"ext_expires_in": 3599,
"access_token": "eyJ0eXAi......"
}
</pre>
<H1>Mule Application Solution - Use access_token</H1>
The following diagram shows the usage of the access_token. The access_token is passed to the server as header.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj69BfJAhHPyqes7vsP4vuRb4-eHr3J73kX9uSfCWeQBiJ4ULdh5wI4g0EUVVEx3RE5xJ7mXzO3eBqmz8g0JLWXYiO5PRcmJ9SyVi8zTciT0fLNHS5GvZfT7hoigKJABhQiaHBuQh5gDG5o/s1600/Screen+Shot+2020-02-15+at+8.35.51+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj69BfJAhHPyqes7vsP4vuRb4-eHr3J73kX9uSfCWeQBiJ4ULdh5wI4g0EUVVEx3RE5xJ7mXzO3eBqmz8g0JLWXYiO5PRcmJ9SyVi8zTciT0fLNHS5GvZfT7hoigKJABhQiaHBuQh5gDG5o/s400/Screen+Shot+2020-02-15+at+8.35.51+PM.png" width="400" height="315" data-original-width="847" data-original-height="667" /></a></div>
As you can see, we need to use cache scope. This allow us to avoid calling the server if the token is not expired. In this case, the token will expire in one hour. Thus our object store TTL should be less then 60 minutes.
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com4tag:blogger.com,1999:blog-6083392550327202392.post-41729504647565782662020-02-02T16:39:00.000-08:002020-02-03T08:26:36.301-08:00Anypoint Runtime Fabric : Part 1 - Pre-Installation Network Checking<H1>Introduction</H1>
In the next few weeks, I will write more details about Mule runtime fabric installation, configuration, and troubleshooting. This article is about to verify the network connectivity before install Anypoint Runtime Fabric (RTF).
In order to install Anypoint RTF, the network firewall must open. The details about the port and host whitelisting can be find here: https://docs.mulesoft.com/runtime-fabric/1.4/install-port-reqs.
<H1>Tools for Network Connectivity Checking</H1>
The following tools should be installed and configure on the Linux system:
<ul>
<li>nslookup
<li>curl
<li>nc
<li>openssl
<li>chrony
</ul>
nslookup must configured properly. To verify nslookup, you can use the following command:
<pre class="brush:bash">
nslookup anypoint.mulesoft.com
</pre>
The above command should print out something like:
<pre class="brush:bash">
Name: anypoint-prod-936665732.us-east-1.elb.amazonaws.com
Address: 52.87.103.123
Name: anypoint-prod-936665732.us-east-1.elb.amazonaws.com
Address: 3.217.25.79
...
</pre>
This means it is working. If you find a problem, most likely the file /etc/resolv.conf has the incorrect content. It should look like the following:
<pre class="brush:js">
$ cat /etc/resolv.conf
# Generated by NetworkManager
search gmcr.com
nameserver 10.64.2.13
nameserver 10.124.240.104
</pre>
The next tool is the well-known cURL. run the following command:
<pre class="brush:bash">
$ curl -sk https://anypoint.mulesoft.com:443
</pre>
You should see something like the following:
<pre class="brush:xml">
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
</pre>
This is good. If it hung, then we have problem.
I have covered the usage of nc and openssl in <a href="https://ggl-consulting.blogspot.com/2020/02/anypoint-runtime-fabric-ssl-handshaking.html">my previous post</a>
<H1>NTP - Network Time Protocol</H1>
In order for Anypoint RTF to work, we must make sure NTP is working in the system. Linux use chrony to perform NTP. Run the following command:
<pre class="brush:bash">
chronyc tracking
</pre>
If the NTP is working correctly, it should print out the something like the following:
<pre class="brush:bash">
Reference ID : A3EDDA13 (toddadev.your.org)
Stratum : 2
Ref time (UTC) : Mon Feb 03 00:18:55 2020
System time : 0.000235624 seconds slow of NTP time
Last offset : -0.000094732 seconds
RMS offset : 0.000081353 seconds
Frequency : 16.987 ppm slow
Residual freq : -0.003 ppm
Skew : 0.062 ppm
Root delay : 0.025243049 seconds
Root dispersion : 0.000095447 seconds
Update interval : 514.2 seconds
Leap status : Normal
</pre>
Note the Leas status must be "Normal", otherwise, Anypoint RTF will be not working. The real for the Leap status is not normal is the network firewal is not open to the RedHat's time server. In this case, we need to use company's internal time server.
To do this, modify the file: /etc/chrony.conf
<pre class="brush:bash">
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
#server 0.rhel.pool.ntp.org iburst
#server 1.rhel.pool.ntp.org iburst
#server 2.rhel.pool.ntp.org iburst
#server 3.rhel.pool.ntp.org iburst
server server 10.70.0.200
...
</pre>
The line: server server 10.70.0.200 is company's private time server. Make sure comment out all the RedHat's entries. After the modification of the file of /etc/chrony.conf, make sure run the following command:
<pre class="brush:bash">
sudo systemctl restart chronyd
sudo systemctl enable chronyd
</pre>
You also may verify the sources of the Ntp by the following command:
<pre class="brush:bash">
chronyc sources
</pre>
This will restart the chrony daemon.
<H1>Network Connectivity Test Scripts</H1>
Mulesoft provided a scripts to<a href="https://github.com/mulesoft-labs/rtf-utilities/blob/master/scripts/rtf_test_backend_connectivities.sh">test the network connectivity at here</a>
If you have gone through the test in the previous sections, this script should run with the following results.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkwpobedd7sKxWTKJgafbiBxTw57cH9TsbXH3Sfwinl2A2RrHKEOCe6Uh3a-sni7fLxPz9Qc3Ol2me9mEtF422q0-8zXLK3pK5QMZBcxAHs_-7RnjVfdJEz7UXdFWoxi1CEsQQskJgWsRE/s1600/Screen+Shot+2020-02-02+at+6.37.46+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkwpobedd7sKxWTKJgafbiBxTw57cH9TsbXH3Sfwinl2A2RrHKEOCe6Uh3a-sni7fLxPz9Qc3Ol2me9mEtF422q0-8zXLK3pK5QMZBcxAHs_-7RnjVfdJEz7UXdFWoxi1CEsQQskJgWsRE/s400/Screen+Shot+2020-02-02+at+6.37.46+PM.png" width="400" height="211" data-original-width="587" data-original-height="310" /></a></div>
Once the above procedures are all done, we are read to perform the installation for Anypoint Runtime Fabric.
I will cover that in the following articles.
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com1tag:blogger.com,1999:blog-6083392550327202392.post-41371561833284762722020-02-02T11:35:00.001-08:002020-02-02T11:51:47.145-08:00Anypoint Runtime Fabric - SSL Handshaking Troubleshooting<H1>Introduction</H1>
Mulesoft's Anypoint Runtime Fabric is gaining momentum in the hybrid deployment model. Since version 1.4.1, the installation, configuration, and management have improved significantly. However, as it involves TLS/SSL, many things can go wrong. This article will provide some technical insights on how to diagnose those issues. In many cases, it may not be the configuration issues. Sometimes, the issues could be related to client code or network configuration. In later section, I have provided java code to test HTTPS REST API.
First let's review the architecture of Anypoint Runtime Fabric.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9VQlFzOei9-msz1zDsGyPWbzgaDOwhhke8_7tW5pwiBGjpjvr_tXPoQ92MVkiTASj97tpdi11BVVMaK45Su0EnTUi6y1dKZneBiqL_ckcCa-I0UkIgBGuIk4LOrjDFEMkeYfEXus0eXUq/s1600/pasted+image+0.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9VQlFzOei9-msz1zDsGyPWbzgaDOwhhke8_7tW5pwiBGjpjvr_tXPoQ92MVkiTASj97tpdi11BVVMaK45Su0EnTUi6y1dKZneBiqL_ckcCa-I0UkIgBGuIk4LOrjDFEMkeYfEXus0eXUq/s400/pasted+image+0.png" width="400" height="268" data-original-width="1209" data-original-height="810" /></a></div>
As shown from the above diagram, the TLS/SSL is applied to all the controllers. Let's say the IP addresses are: <br>
10.64.6.65<br>
10.64.6.66<br>
10.64.6.67<br>
On all of the above controllers, the port 443 should be open. And we should be able to connect to all these controllers by using networking tools like nc, openssl, etc.
<H1>Verify Controller's TLS/SSL</H1>
First, from the client point of view, we need to make sure the port 443 is open and reachable. To do so, we can execute the following command:
<pre class="brush:bash">
$ nc -zv 10.64.6.65 443
Connection to 10.64.6.65 port 443 [tcp/https] succeeded!
</pre>
As you can see, we can connect the port 443 successfully.
nc is a very light and powerful tool for quickly scan the port of the server.
Secondly, we can use openssl to verify the TLS/SSL:
<pre class="brush:bash">
$ openssl s_client -connect 10.64.6.65:443
</pre>
openssl is a heavy weight tool. It can do a lot things, such as, connection verification, ssl certification generation and conversion, etc. The above command will print out information about the server's TLS/SSL certificates, ssh handshaking, cipher, etc. Here are an example:
<pre class="brush:bash">
$ openssl s_client -connect 10.64.6.65:443
CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
verify return:1
depth=0 C = US, ST = Texas, L = Plano, O = "Keurig Dr. Pepper, Inc.", CN = *.gmcr.com
verify return:1
---
Certificate chain
0 s:/C=US/ST=Texas/L=Plano/O=Keurig Dr. Pepper, Inc./CN=*.gmcr.com
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
subject=/C=US/ST=Texas/L=Plano/O=Keurig Dr. Pepper, Inc./CN=*.gmcr.com
issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
---
No client certificate CA names sent
Server Temp Key: ECDH, X25519, 253 bits
---
SSL handshake has read 4087 bytes and written 289 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: 820EF7CA565E40E495662B0D1AAAB24E2AE431B660A86DF7659D2DEFB7B986B1
Session-ID-ctx:
Master-Key: 98F3D990B13A2847041E6275682C74F3D371A15C8D0358434957D6B87B865DA9E839AB95950FE6F24E9D5CDD0DE59F93
TLS session ticket lifetime hint: 7200 (seconds)
TLS session ticket:
...
Start Time: 1580667077
Timeout : 7200 (sec)
Verify return code: 0 (ok)
---
closed
</pre>
The most important information client should notice are the following:
<ul>
<li>CN = *.gmcr.com -- This dictate how client should invoke the service
<li>SHA2 High Assurance Server CA -- SHA2 is Secure Hashing Algorithm 2.
<li>Protocol : TLSv1.2 -- We are using TLSv1.2, now Anypoint RTF also support TLSv1.3
<li>Cipher : ECDHE-RSA-AES128-GCM-SHA256
<li>Verify return code: 0 (ok) -- This is very important.
</ul>
The last line tells us the TLS certificate is valid and can be verified. If the self-signed certificate is applied, the last line will be something like:
<pre class="brush:bash">
Verify return code: 18 (self signed certificate)
</pre>
<H1>Verify Server's Cipher Suites</H1>
It is not very uncommon that server and client could not find the common supported ciphers. To handle this, we need to scan server's cipher suite. I use two tools: nmap and cipherscan. There are many free software available, but I find they are very easy to use and very powerful. Wireshark is another very popular tool. If all of the simple tools are exhausted, we can use Wireshark.
Here is example using cipherscan:
<pre class="brush:bash">
$ ./cipherscan 10.64.6.65
Warning: target is not a FQDN. SNI was disabled. Use a FQDN or '-servername <fqdn>'
...............
Target: 10.64.6.65:443
prio ciphersuite protocols pfs curves
1 ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 ECDH,P-256,256bits prime256v1,secp384r1,secp521r1
2 ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 ECDH,P-256,256bits prime256v1,secp384r1,secp521r1
3 AES256-GCM-SHA384 TLSv1.2 None None
4 DHE-RSA-AES128-GCM-SHA256 TLSv1.2 DH,2048bits None
5 AES128-GCM-SHA256 TLSv1.2 None None
Certificate: trusted, 4096 bits, sha256WithRSAEncryption signature
TLS ticket lifetime hint: 7200
NPN protocols: None
OCSP stapling: not supported
Cipher ordering: server
Curves ordering: server - fallback: no
Server supports secure renegotiation
Server supported compression methods: NONE
TLS Tolerance: yes
Intolerance to:
SSL 3.254 : absent
TLS 1.0 : PRESENT
TLS 1.1 : PRESENT
TLS 1.2 : absent
TLS 1.3 : absent
TLS 1.4 : absent
</pre>
As you can see, it provide supported cipher and TLS protocols. You can down cipherscan from github: https://github.com/mozilla/cipherscan.
nmap is another very powerful and easy to use tool. nmap is a bit slow.
<pre class="brush:bash">
$ nmap -sV --script ssl-enum-ciphers -p 443 10.64.6.65
Starting Nmap 7.80 ( https://nmap.org ) at 2020-02-02 13:11 CST
Nmap scan report for hello-earth.kdrp.com (10.64.6.65)
Host is up (0.086s latency).
PORT STATE SERVICE VERSION
443/tcp open ssl/https
| fingerprint-strings:
| FourOhFourRequest, GetRequest, HTTPOptions:
| HTTP/1.1 404 NOT FOUND
| Content-Length: 0
| Connection: Close
| RTSPRequest, SIPOptions:
| HTTP/1.1 400 BAD REQUEST - bad version
| Content-Length: 0
|_ Connection: Close
| ssl-enum-ciphers:
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
| TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 4096) - A
| TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 2048) - A
| TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 4096) - A
| compressors:
| NULL
| cipher preference: server
| warnings:
| Key exchange (dh 2048) of lower strength than certificate key
| Key exchange (ecdh_x25519) of lower strength than certificate key
|_ least strength: A
1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at https://nmap.org/cgi-bin/submit.cgi?new-service :
SF-Port443-TCP:V=7.80%T=SSL%I=7%D=2/2%Time=5E371EE9%P=x86_64-apple-darwin1
SF:7.7.0%r(GetRequest,40,"HTTP/1\.1\x20404\x20NOT\x20FOUND\r\nContent-Leng
SF:th:\x200\r\nConnection:\x20Close\r\n\r\n")%r(HTTPOptions,40,"HTTP/1\.1\
SF:x20404\x20NOT\x20FOUND\r\nContent-Length:\x200\r\nConnection:\x20Close\
SF:r\n\r\n")%r(FourOhFourRequest,40,"HTTP/1\.1\x20404\x20NOT\x20FOUND\r\nC
SF:ontent-Length:\x200\r\nConnection:\x20Close\r\n\r\n")%r(RTSPRequest,50,
SF:"HTTP/1\.1\x20400\x20BAD\x20REQUEST\x20-\x20bad\x20version\r\nContent-L
SF:ength:\x200\r\nConnection:\x20Close\r\n\r\n")%r(SIPOptions,50,"HTTP/1\.
SF:1\x20400\x20BAD\x20REQUEST\x20-\x20bad\x20version\r\nContent-Length:\x2
SF:00\r\nConnection:\x20Close\r\n\r\n");
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 46.51 seconds
</pre>
<H1>Verify REST API</H1>
From network point view, everything seems working, but client may still have problem to invoke the REST API exposed on the Anypoint RTF. In these cases, we need to check the following.
Using postman, if the certificate applied to Anypoint RTF is self-signed, we need to make sure to turn off the ssl verification on the postman.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgT7ihcpdBDOboB5laV6QA62XXFtD-kGK31xiwZJk33qaGE2g83mzjfj9-QIon5ooHTliqDKMAR6r1V2wViCSBcrWn6L32oOGL63BwblhfN8qzvgmawqhMYVCaPIXhcdro52WluOgOfYFut/s1600/Screen+Shot+2020-02-02+at+1.17.12+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgT7ihcpdBDOboB5laV6QA62XXFtD-kGK31xiwZJk33qaGE2g83mzjfj9-QIon5ooHTliqDKMAR6r1V2wViCSBcrWn6L32oOGL63BwblhfN8qzvgmawqhMYVCaPIXhcdro52WluOgOfYFut/s400/Screen+Shot+2020-02-02+at+1.17.12+PM.png" width="400" height="292" data-original-width="831" data-original-height="607" /></a></div>
Still some client are using plain java to invoke REST API. In this case, we can use the following code:
<pre class="brush:java">
import org.apache.http.Header;
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
public class HttpsHelloWorld {
public static void main(String[] args) {
// TODO Auto-generated method stub
System.out.println("Hello, World");
HttpGet request = new HttpGet("https://kknxplvmsftct.gmcr.com/earth/hello");
request.addHeader("Host", "hello-earth-one.gmcr.com");
CloseableHttpClient httpClient = HttpClients.createDefault();
try {
CloseableHttpResponse response = httpClient.execute(request);
// Get HttpResponse Status
System.out.println(response.getStatusLine().toString());
HttpEntity entity = response.getEntity();
Header headers = entity.getContentType();
System.out.println(headers);
if (entity != null) {
// return it as a String
String result = EntityUtils.toString(entity);
System.out.println(result);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
</pre>
add the following dependencies to pom.xml
<pre class="brush:xml">
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.10</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>5.2.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-web</artifactId>
<version>5.2.3.RELEASE</version>
</dependency>
</pre>
Or if you prefer to use REST template, you can use the following example:
<pre class="brush:java">
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URI;
import java.net.URL;
import javax.net.ssl.HttpsURLConnection;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.RequestEntity;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.client.HttpClientErrorException;
import org.springframework.web.client.RestTemplate;
public class HttpsPostMes {
public static void main(String[] args) {
try {
httpsRequestCall3();
} catch(Exception e) {
e.printStackTrace();
}
}
public static void httpsRequestCall3() {
CloseableHttpClient httpClient
= HttpClients.custom()
.setSSLHostnameVerifier(new NoopHostnameVerifier())
.build();
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(httpClient);
String urlOverHttps = "https://kknxplvmsftct.gmcr.com/earth/hello";
HttpHeaders headers = new HttpHeaders();
headers.add("Host", "hello-earth-one.gmcr.com");
HttpEntity<String> entity = new HttpEntity<>("header", headers);
ResponseEntity<String> response = new RestTemplate(requestFactory).exchange(urlOverHttps, HttpMethod.GET, entity, String.class);
org.springframework.http.HttpStatus code = response.getStatusCode();
System.out.println(code);
System.out.print(response);
}
}
</pre>
<H1>Take Aways</H1>
In this article, I have covered the following:
<ol>
<li>Basic Anypoint runtime fabric's controllers
<li>Tools to verify servers ports, SSL protocols, and ciphers
<ul>
<li>nc
<li>openssl
<li>nmap
<li>ciphjerscan
</ul>
<li>java code to invoke REST API using https
</ol>
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com3tag:blogger.com,1999:blog-6083392550327202392.post-55123693912382265972019-10-26T08:42:00.001-07:002019-10-26T08:57:39.658-07:00Mule Integration - Write Elegant Code<H1>Introduction</H1>
<p>
Recently, I encountered the following piece of Mule Dataweave 2.0 function code.
<pre class="brush:js">
fun getEmployeePovince(worker) = getStateAbbrevation(worker) match {
case "AB" -> "AB"
case "BC" -> "BC"
case "MB" -> "MB"
case "NB" -> "NB"
case "NL" -> "NL"
case "NS" -> "NS"
case "NT" -> "NT"
case "NU" -> "NU"
case "ON" -> "ON"
case "PE" -> "PE"
case "QC" -> "QC"
case "SK" -> "SK"
case "YT" -> "YT"
else -> "ZZ"
}
</pre>
<p>
The purpose of the above function is to get home province's code of a Canadian employee. The Canadian province code is like, BC for British Columbia, ON for Ontario, etc. The input is an xml object which contains information about the employee. If the worker's province code is not among the list, default to "ZZ".
</p>
<p>The above code works and about to go to production. However, this kind of code is really not very cool to say the least. It is just an amateur's code!
<H1>Improvement One</H1>
First of all, the gist of this kind of problem is to find a match from a given list of Strings. Mule Dataweave 2.0 provides a function, namely find. <a href="https://docs.mulesoft.com/mule-runtime/4.2/dw-core-functions-find">The documentation can be found here.</a>. The find function works like the following:
<pre class="brush;js">
['aa', 'bb', 'cc'] find 'xy' //return [], empty array
['aa', 'bb', 'cc', 'aa'] find 'aa' //return [0, 3]
</pre>
Now the code using find function should be clear. If the return array from the find is empty (isEmpty(findFunction)), use the input, otherwise, use default, "ZZ".
<p>
The following is the datawave code:
<pre class="brush:js">
%dw 2.0
var CanadianProvinces = ["BC", "MB", "NB", "NL", "NS", "NT", "NU", "NT", "NU", "ON", "PE", "QC", "SK", "YT"]
var pv = payload.state
fun findAKey(aKey) = CanadianProvinces find aKey
output application/json
---
{
province: if(isEmpty(findAKey(pv))) "ZZ" else pv,
}
</pre>
The above code is much cleaner. We put the constants of the province code into an array, and use find function. However the above code is still not good for maintenance. Let's see if I want to use the same to logic to US state's code. We have to modify the dataweave code. We can further improve the code. That is to put the constant into property file.
<H1>Improvement Two</H1>
First, we put the two constants to the yaml property file as the following:
<pre class="brush;js">
provinces: ["BC", "MB", "NB", "NL", "NS", "NT", "NU", "NT", "NU", "ON", "PE", "QC", "SK", "YT"]
constants:
default: "ZZ"
</pre>
<pre class="brush;js">
%dw 2.0
var provinces = p('provinces')
var pv = payload.state
var dp = p('constants.default')
fun findAKey(aKey) = provinces find aKey
output application/json
---
{
provice: if(isEmpty(findAKey(pv))) dp else pv,
}
</pre>
<H1>Take Aways</H1>
<ol>
<li> Two dataweave 2.0 functions: switch and find
<li> Array constants in yaml property file
<li> When we encounter some strange code, we should think about the improvement. There is always ways to write elegant code.
</ol>
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com3tag:blogger.com,1999:blog-6083392550327202392.post-59821627728735572562019-09-03T13:11:00.001-07:002019-09-03T16:22:05.791-07:00Install JSON Plugin In Anypoint Studio<H1>Introduction</H1>
JSON Plugin is very useful tool for editing and verifying JSON schemas or json examples data in our API deployment. Unfortunately, it does not come with Anypoint Studio. To me, AnypointStudio should more or less behave like Eclipse which allow us to drag and paste any available plugin. This short article describe the procedure to install the plugin.
<H1>Installation of JSON Plugin</H1>
<p>First download the plugin zip file from <a href="https://sourceforge.net/projects/eclipsejsonedit/">this website</a> to ~/Download. The file name should be like:
<pre class="brush:bash">
rw-r--r--@ 1 gl17 staff 112K Sep 3 14:38 jsonedit-repository-0.9.7.zip
</pre>
<p>Second, in AnypointStudio, Help --> Install New Software --> Add
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzPprOLwzXHn_0jcjyZefqE9mBy92HgKvexGyqtHhaZ0yvJPWefO3LuSixVVYlotOMP1qbmK1Kzg1DWk7dDT3OsUYxCg4zf_S4Co6CERKpPUlm643Q0T8aocR_DZdNCinBPODafpZQFtXO/s1600/Screen+Shot+2019-09-03+at+3.08.37+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzPprOLwzXHn_0jcjyZefqE9mBy92HgKvexGyqtHhaZ0yvJPWefO3LuSixVVYlotOMP1qbmK1Kzg1DWk7dDT3OsUYxCg4zf_S4Co6CERKpPUlm643Q0T8aocR_DZdNCinBPODafpZQFtXO/s400/Screen+Shot+2019-09-03+at+3.08.37+PM.png" width="400" height="147" data-original-width="944" data-original-height="348" /></a></div>
<p>
Note, select Achive and from local file.
<p>After installation, restart the AnypointStudion. Now if you open any json file, the Studio will validate the file.Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com2tag:blogger.com,1999:blog-6083392550327202392.post-69670922764071402642019-08-24T16:41:00.001-07:002019-08-24T16:42:14.578-07:00Manage Mule Runtime Using Linux Services On RHEL 7<H1>Introduction</H1>
<p>
This article describes the procedures to enable Mule runtime as systemd service for Mule standalone runtime clustering on-premises or in private clouds. Since RHEL 7, the systemd init system is a must. The traditional init.d approach is obsolete. For details about the systemd and Unit files, you may refer to <a href="https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files">this article</a>.
<H1>Assumptions</H1>
<ul>
<li>Mule standalone runtime is installed at /opt/mule/runtime/current
<li>Mule runtime can be started and stoped by the command /opt/mule/runtime/current/bin/mule start | stop
<li>mule user has sudo permission
</ul>
<H1>Create Unit File</H1>
First, we need to create a file, mule.service at /etc/systemd/system, with the following contents:
<hr/>
<pre class="brush:bash">
# file: /etc/systemd/system/mule.service
# Systemd unit file for mule standalone runtime
[Unit]
Description=Mule Runtime Standalone Runtime
After=syslog.target network.target
[Service]
Type=forking
WorkingDirectory=/opt/mule/runtime/current
Environment=JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64
Environment=MULE_HOME=/opt/mule/runtime/current
TasksMax=infinity
LimitNOFILE=65335
ExecStart=/opt/mule/runtime/current/bin/mule start
ExecStop=/opt/mule/runtime/current/bin/mule stop
User=mule
Group=mule
RestartSec=10
Restart=always
[Install]
WantedBy=multi-user.target
</pre>
<hr/>
<pre class="brush:bash">
$ cd /etc/systemd/system/
$ sudo chmod 644 mule.service
$ sudo systemctl daemon-reload
$ sudo systemctl enable mule.service
</pre>
The above command will enable the mule.service. This procedure simply creates a line in the folder of: /etc/systemd/system/system-update.target.wants
<pre class"brush:plain">
systemd-readahead-drop.service -> /usr/lib/systemd/system/systemd-readahead-drop.service
</pre>
Now, we are ready to start the mule runtime as a service. To do this, we must first stop the running Mule runtime:
<pre class"brush:bash">
$ cd /opt/mule/runtime/current
$ bin/mule stop
</pre>
Now, run the following command:
<pre class="brush:bash">
$ sudo systemctl start mule.service
</pre>
It may take sometime before we get the prompt back. After that we can check whether the mule runtime is running or not by the following command:
<pre class="brush:bash">
$ sudo systemctl status mule.service
● mule.service - Mule Runtime Standalone Runtime
Loaded: loaded (/etc/systemd/system/mule.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2019-08-24 15:46:04 CDT; 2h 45min ago
Main PID: 17555 (wrapper-linux-x)
CGroup: /system.slice/mule.service
├─17555 /opt/mule/runtime/current/lib/boot/exec/wrapper-linux-x86-64 /opt/mule/runtime/current/conf/wrapper.conf wrapper.s...
└─17569 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/bin/java -Dmule.home=/opt/mule/runtime/current -D...
Aug 24 15:45:41 wp37mulerte01.aci.awscloud systemd[1]: Starting Mule Runtime Standalone Runtime...
Aug 24 15:45:41 wp37mulerte01.aci.awscloud mule[17439]: MULE_HOME is set to /opt/mule/runtime/current
Aug 24 15:45:41 wp37mulerte01.aci.awscloud mule[17439]: MULE_BASE is set to /opt/mule/runtime/current
Aug 24 15:45:42 wp37mulerte01.aci.awscloud mule[17439]: Starting Mule Enterprise Edition...
Aug 24 15:46:04 wp37mulerte01.aci.awscloud mule[17439]: Waiting for Mule Enterprise Edition.......................
Aug 24 15:46:04 wp37mulerte01.aci.awscloud mule[17439]: running: PID:17555
Aug 24 15:46:04 wp37mulerte01.aci.awscloud systemd[1]: Started Mule Runtime Standalone Runtime.
</pre>
As you can see, the mule runtime is running. If you want to know the deatils about the mule runtime, you can use the following command:
<pre class="brush:bash">
$ sudo systemctl -l status mule.service
</pre>
The -l option will print out the arguments passed to the JVM. To view the history of stop/start of the Mule runtime, we can use the following command:
<pre class="brush:bash">
$ sudo journalctl -u mule.service
-- Logs begin at Tue 2019-08-06 23:11:10 CDT, end at Sat 2019-08-24 18:39:13 CDT. --
Aug 22 21:40:03 wd35mulerte01.aci.awscloud systemd[1]: Starting Mule Runtime Standalone Runtime...
Aug 22 21:40:03 wd35mulerte01.aci.awscloud mule[31293]: MULE_HOME is set to /opt/mule/runtime/current
Aug 22 21:40:03 wd35mulerte01.aci.awscloud mule[31293]: MULE_BASE is set to /opt/mule/runtime/current
Aug 22 21:40:05 wd35mulerte01.aci.awscloud systemd[1]: mule.service: control process exited, code=exited status=1
Aug 22 21:40:05 wd35mulerte01.aci.awscloud systemd[1]: Failed to start Mule Runtime Standalone Runtime.
Aug 22 21:40:05 wd35mulerte01.aci.awscloud systemd[1]: Unit mule.service entered failed state.
Aug 22 21:40:05 wd35mulerte01.aci.awscloud systemd[1]: mule.service failed.
Aug 22 21:40:14 wd35mulerte01.aci.awscloud systemd[1]: Stopped Mule Runtime Standalone Runtime.
Aug 22 21:52:39 wd35mulerte01.aci.awscloud systemd[1]: Starting Mule Runtime Standalone Runtime...
Aug 22 21:52:39 wd35mulerte01.aci.awscloud mule[32610]: MULE_HOME is set to /opt/mule/runtime/current
Aug 22 21:52:39 wd35mulerte01.aci.awscloud mule[32610]: MULE_BASE is set to /opt/mule/runtime/current
Aug 22 21:52:40 wd35mulerte01.aci.awscloud mule[32610]: Starting Mule Enterprise Edition...
Aug 22 21:53:03 wd35mulerte01.aci.awscloud mule[32610]: Waiting for Mule Enterprise Edition.........................
Aug 22 21:53:04 wd35mulerte01.aci.awscloud mule[32610]: running: PID:32750
Aug 22 21:53:04 wd35mulerte01.aci.awscloud systemd[1]: Started Mule Runtime Standalone Runtime.
Aug 22 21:54:15 wd35mulerte01.aci.awscloud systemd[1]: Stopping Mule Runtime Standalone Runtime...
Aug 22 21:54:15 wd35mulerte01.aci.awscloud mule[633]: MULE_HOME is set to /opt/mule/runtime/current
Aug 22 21:54:15 wd35mulerte01.aci.awscloud mule[633]: MULE_BASE is set to /opt/mule/runtime/current
Aug 22 21:54:15 wd35mulerte01.aci.awscloud mule[633]: Stopping Mule Enterprise Edition...
Aug 22 21:54:18 wd35mulerte01.aci.awscloud systemd[1]: Stopped Mule Runtime Standalone Runtime.
Aug 22 21:55:43 wd35mulerte01.aci.awscloud systemd[1]: Starting Mule Runtime Standalone Runtime...
Aug 22 21:55:44 wd35mulerte01.aci.awscloud mule[912]: MULE_HOME is set to /opt/mule/runtime/current
Aug 22 21:55:44 wd35mulerte01.aci.awscloud mule[912]: MULE_BASE is set to /opt/mule/runtime/current
Aug 22 21:55:45 wd35mulerte01.aci.awscloud mule[912]: Starting Mule Enterprise Edition...
Aug 22 21:56:08 wd35mulerte01.aci.awscloud mule[912]: Waiting for Mule Enterprise Edition.........................
Aug 22 21:56:08 wd35mulerte01.aci.awscloud mule[912]: running: PID:1091
Aug 22 21:56:08 wd35mulerte01.aci.awscloud systemd[1]: Started Mule Runtime Standalone Runtime.
</pre>
<p>
That is it. It is very helpful to study the commands of systemctl and journalctl.
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com1tag:blogger.com,1999:blog-6083392550327202392.post-50344353782997185252019-08-18T15:10:00.000-07:002019-08-18T15:10:08.841-07:00Two-Way SSL In Mule Application - Part 2<H1>Introduction</H1>
<p>
In <a href="https://dzone.com/articles/two-way-ssl-in-mule-application">my previous article in DZone</a> or <a href="https://ggl-consulting.blogspot.com/2019/08/two-way-ssl-in-mule-application.html">here</a>, I omitted the procedure to create a trust store for the application. This is important if the applications are deployed to the CloudHub.
<p>In this article, I will describe the procedures to create the trust store and how to configure the HTTPS request for Mule application
<H1>Create A Trust Store FOR MULE HTTPS Request</H1>
<p>
The procedures to import the server's PEM certificate to a trust store are the following.
<p>
First, we will create a trust store using the following command:
<pre class="brush:bash">
keytool -genkey -keyalg RSA -alias cyberark-poc -keystore truststore.ks
</pre>
Enter anything. They are not important as we will delete it.
<p>
Second, delete the content of the trust store just created:
<pre class="brush:bash">
keytool -delete -alias cyberark-poc -keystore truststore.ks
</pre>
Third, import the server's certificate:
<pre class="brush:bash">
keytool -import -v -trustcacerts -alias cyberark-server -file SERVER-CERT.pem -keystore truststore.ks
</pre>
Now, copy the truststore.ks to Mule application project /src/main/resources
<H1>HTTPS Request Configuration</H1>
The following is the complete HTTPS Request configuration:
<pre class="brush:xml">
<http:request-config name="HTTPS_Request_configuration" doc:name="HTTP Request configuration" doc:id="489bd416-2a79-4817-9968-627aaa6ee553" >
<http:request-connection protocol="HTTPS" host="${cyberark.host}" port="${cyberark.port}" >
<tls:context >
<tls:trust-store path="truststore.ks" password="changeit" type="jks" />
<tls:key-store type="pkcs12" path="client.pfx" keyPassword="gary" password="gary" />
</tls:context>
</http:request-connection>
</http:request-config>
</pre>
Note: I put both client.pfx and truststore in the directory of /src/main/resources. You may put them into different directory. In that case, you need to give the full path relative to the Mule Application Project, such as ssh/cert/client.pfx.
<H1>The Key Takeaways</H1>
The best practice for certificates manipulation is:
<ol>
<li>If the deployment is on-prem, import servers' certificates to cacert. In this way if the server's certificate is expired, we just need to reimport, not code change is required.
<li>If the deployment is CloudHub, we have to import the servers' certificate to a truststore as described in this article.
<li>Use JKS format for the trust store used in the HTTPS request. It is most popular one.
</ol>Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com0tag:blogger.com,1999:blog-6083392550327202392.post-47637956444867072312019-08-11T13:56:00.000-07:002019-08-11T13:56:25.941-07:00Two-Way SSL In Mule Application<H1>Introduction</H1>
<p>In <a href="https://dzone.com/articles/mutual-authentication-two-way-ssl-explained-using">my previous article</a>, I have explained how Two-Way SSL works with the context of Mule Application. Many people have asked the question about how to setup HTTPS request in Mule application. This article provide the details about the procedures to invoke HTTPS services which require Two-Way SSL or Mutual Authentication. Before we dive into the detail procedures, lets review how Two-Way SLL works between clients and servers.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAolMGIcPHlhyphenhyphenFkW1Tc_HSeae4Jw8Qq09Y0NFbZIwDxgEr8JnnIPKEBBUzEy13dZqFGUIlaVfdkpe8XZk1vQ4YCunIXjHNjyO2uLFAMdJGFW0CkKDaLoVlsnwLr4ioqxeLDsiqD71tIfbW/s1600/Blank+Diagram+%25281%2529.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAolMGIcPHlhyphenhyphenFkW1Tc_HSeae4Jw8Qq09Y0NFbZIwDxgEr8JnnIPKEBBUzEy13dZqFGUIlaVfdkpe8XZk1vQ4YCunIXjHNjyO2uLFAMdJGFW0CkKDaLoVlsnwLr4ioqxeLDsiqD71tIfbW/s400/Blank+Diagram+%25281%2529.png" width="400" height="212" data-original-width="1406" data-original-height="744" /></a></div>
The gist of Two-Way SSL is to exchange certificates between clients and servers. The details are pretty complicated and they beyond the scope of this article. Basically, here are the high level scheme of the exchange of certificates:
<ol>
<li>Client send a ClientHello message to a server
<li>Server replies with ServerHello, Server's certificate, and Request for Client's certificate
<li>Client its certificate other information like cipher scheme, server's certificate verification, etc.
<li>Server replies with cipher scheme.
<li>Start to exchange information
</ol>
Now, how do we setup Mule Application as client?
<H1>Client's Certificate Generation</H1>
In general, IT admin will generate client certificates similar as I describe in <a href="https://ggl-consulting.blogspot.com/2019/08/mule-4-enable-https-connector-using.html">my blog here</a>
Let's assume that is the way for now so that we can describe how to setup Mule HTTPS Request.
Before we continue, we need to obtain server's certificate in advance. The certificate can be in many forms like JKS, PKCS12, PEM, etc. Mule HTTPS request support three forms:
<ul>
<li>JKS
<li>PKCS12
<li>JCEKS
</ul>
Let's say if we got PEM format from the server. We need to do one of the two things depending on the deployment pattern.
<ul>
<li>if it is on-prem deployment, the best way is to import the cert to JVM cacerts
<li>if it is deployed to MuleSoft CloudHub, we need to convert the PEM to PKCS12.
</ul>
If it is on-prem deplopment, we can import the the PEM certificate directly into cacerts here is the procedure (Make sure you have sudo permission, and server's cert is named like SERVER_CERT.pem)
<pre class="brush:bash">
cd ${JAVA_HOME}/jre/lib/security
cp SERVER_CERT.pem
sudo keytool -import -alias mule1-cyberark -keystore cacerts -file SERVER_CERT.pem
</pre>
To be sure that server's cert is in pem format, you can use the following command:
<pre class="brush:bash">
$ openssl x509 -in SERVER_CERT.pem -text
</pre>
If it is CloudHub deployment, we need to convert the pem file to PKCS12 format. Here is the command:
<pre class="brush:bash">
$ openssl pkcs12 -export -nokeys -in SERVER_CERT.pem -out SERVER_CERT.pfx
</pre>
<p>Note the option of "-nokeys". This means I do not have the private key of the certificate.
Now we have server's certificates being taken care of. We need to convert the client's certificate to PKCS12. Here is the command to do so:
<pre class="brush:bash">
openssl pkcs12 -export -in cacert.pem -inkey cakey.pem -out identity.p12 -name "mykey"
</pre>
<p>Note the above procedure will ask the password. Make sure you remember it.
<H1>Setup Mule Flow</H1>
The following diagram shows the simple Mule flow
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXN5hCtBd6nJW0vigEY9GtCK1O7f4H5-ipNhJD1N2YnGSXD0S-oxR9X0XdR3Ll46bqGYN2AvzMq4E7T9W-SwhjZw0LgJ4Ezs3TW4eivLyUZIBHsy9aPIAJ6un1094o5wdyMQjTuEnlZI__/s1600/two-way-ssl-request.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXN5hCtBd6nJW0vigEY9GtCK1O7f4H5-ipNhJD1N2YnGSXD0S-oxR9X0XdR3Ll46bqGYN2AvzMq4E7T9W-SwhjZw0LgJ4Ezs3TW4eivLyUZIBHsy9aPIAJ6un1094o5wdyMQjTuEnlZI__/s400/two-way-ssl-request.PNG" width="400" height="290" data-original-width="432" data-original-height="313" /></a></div>
The https request configuration is the following:
<pre class="brush:xml">
<http:request-config name="HTTPS_Request_configuration" doc:name="HTTP Request configuration" doc:id="489bd416-2a79-4817-9968-627aaa6ee553" >
<http:request-connection protocol="HTTPS" host="two-way-ssl.server.com" port="443" >
<tls:context >
<tls:key-store type="pkcs12" path="identity.p12" keyPassword="gary" password="gary" />
</tls:context>
</http:request-connection>
</http:request-config>
</pre>
The import point here is that client's certificate is
<pre class="brush:xml">
<tls:key-store type="pkcs12" path="identity.p12" keyPassword="gary" password="gary" />
</pre>
and server's certificates is
<pre class="brush:xml">
<tls:trust-store type="pkcs12" path="SERVER_CERT.p12" keyPassword="gary" password="gary" />
</pre>
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com4tag:blogger.com,1999:blog-6083392550327202392.post-51903201130250517322019-08-09T16:32:00.001-07:002019-08-18T15:11:23.289-07:00How To Pass MuleSoft Certified Developer - Level 1 (Mule 4)<H1>Congratulation Gary!</H1>
First of all, I must congratulate myself for getting this done. For almost a year, I have been thinking to take the exam, but my project schedules have been crazy. I hardly find time to prepare the certification. Three weeks ago, I decided to give a shot. I studied two weekends, and spend about 1 hour each day. And today, I did! I must say that I feel it is a kind of accomplishment. This test is not easy!
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwDd5diiJ9Sm-dTsYzk0WgXyk8iXxf-Eieq1PIEaI_z3unKM4DdbQbDYSSgQsGfPnGmjxPTbAYn1jPHysGa_zyTMmuViqIGtCQ-AcOSqI9JWYZiLD38uUl4TwxikTpBmPErXz_u9xeYWBx/s1600/MCD.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwDd5diiJ9Sm-dTsYzk0WgXyk8iXxf-Eieq1PIEaI_z3unKM4DdbQbDYSSgQsGfPnGmjxPTbAYn1jPHysGa_zyTMmuViqIGtCQ-AcOSqI9JWYZiLD38uUl4TwxikTpBmPErXz_u9xeYWBx/s400/MCD.png" width="400" height="283" data-original-width="1600" data-original-height="1132" /></a></div>
Here I will try to summarized my feeling about the test and how I prepared the test. Hopefully, it will provided some help to those who are thinking to take the test.
<H1>The Procedures</H1>
<p>
First of all, go to Mule training website and paid 250 USD online and schedule a test at the same time. I did my test at my local test center. There, I have to lock away my cell phone, watch, wallet, even my hat! It is very quiet and comfortable place. Not many people there either.
<p>
The exam is 2 hours long. That is plenty time to ponder each question carefully. At the first 10 minutes, MuleSoft did a survey about my experience in Mule, how I prepared the test, what role I play, etc. I am not sure why they should ask these questions at all. One thing really makes me suspicious is that if you are a very experience developer, you may get harder questions. It is just my guess!
<H1>How Was My Test Results?</H1>
<p>It took me 88 minutes to submit the answer. I know I had plenty time, so that I read each question very carefully and did not plan to review them once I am done with all the questions.
<p>
The following are the results I got:
<pre class="brush:js">
Creating Application Networks: 100.00%
Designing APIs: 100.00%
Building API Implementation Interfaces: 100.00%
Deploying and Managing APIs and Integrations: 75.00%
Accessing and Modifying Mule Events: 83.33%
Structuring Mule Applications: 66.66%
Routing Events: 80.00%
Handling Errors: 80.00%
Troubleshooting and Testing Mule Applications: 66.66%
Writing DataWeave Transformations: 100.00%
Using Connectors: 83.33%
Processing Records: 100.00%
Result: PASS
</pre>
Roughly, my overall score is about 89%. I think I did pretty OK given that I did not have enough time to prepare. I have no idea why my trouble shooting score is only 66.66%. I thought this is my strongest area.
<H1>How Do I Feel About The Questions?</H1>
Overall, I think the questions are very good, but many of them are very hard to answer with confidence. About 15% of the questions are really hard.
<p>
The challenges come from several fronts.
Firstly, most of the questions are very long. You must read the question at least two times before you answer the question. This really tests your English as the questions are very tricky.
Secondly, most questions are really difficult to 100% sure from the first glance. You have to read them very carefully. Sometimes the brackets, comma, and semicolon make the difference.
Thirdly, some questions are rare to encounter in real life, such as exporting artifacts from Anypoint Studio. We normally don't do this.
All in all, I think MuleSoft training department did a good job. It seems the Mule 4 MCD is a bit harder the Mule 3 one.
<H1>How I Prepared My Test </H1>
<p>I really don't have the time to go through all the training materials. Definitely no time to go through all the DIY.
<p>Here is the procedure I took. I think if you are experience developer and want to pass the test at the first shot. You can follow my way.
<ol>
<li>Go to the final Quiz at the last Module of the training material, and try to answer them. They are pretty difficult actually. Many of them, I really had trouble to answer at the first time. The good news is that for each question, if you did not answer correctly, the website tell you the right answer. Thus you can figure out why.
<li>Quickly go through each chapter and all the slides. Notes down what the chapter is about.
<li>After the first two steps, I start to create many mini-code to really understand the details about connectors, error handling, different scopes, etc.
<li>Take notes. Take a lot of notes. These notes really help me to remember the nitty-mitty details. Remember, to pass the test, you must pay attention to tiny details.
</ol>
<H1>Some Thoughts About The Certification</H1>
<p>Definitely, it worths the time and energy to prepare and take the Exam. It really lets us to start paying attention to details and try to think about the reason about the way MuleSoft implements connectors, design patterns, and other systems. It is not for beginner anyway.
<p>Don't take the delta test. To me, it is really better to challenge ourselves and take Mule 4 MCD.
<p>The questions have a lot of room to improve.
<ul>
<li>It should really focus on how our daily development works.
<li>Questions should more focus on development of Mule flows, less memorization of syntaxes.
<li>More questions on design and troubleshooting.
<li>More questions on problem solving skills.
</ul>
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com13tag:blogger.com,1999:blog-6083392550327202392.post-87644525442171373142019-08-06T15:57:00.000-07:002019-08-06T15:57:01.711-07:00Mule 4: Enable HTTPS Connector Using openssl<H1>Introduction</H1>
This article demonstrate the procedures using openssl to generate self-signed certificates, and how to use the private key to configure HTTPS connector.
<H1>Generate Private Key And Public Cert Using openssl</H1>
<pre class="brush:bash">
$ openssl req -newkey rsa:2048 -x509 -keyout cakey.pem -out cacert.pem -days 3650
Generating a RSA private key
....+++++
...................................................+++++
writing new private key to 'cakey.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:Texas
Locality Name (eg, city) [Default City]:Dallas
Organization Name (eg, company) [Default Company Ltd]:GGL Consulting Inc
Organizational Unit Name (eg, section) []:EA
Common Name (eg, your name or your server's hostname) []:Gary Liu
Email Address []:gary.liu1119@gmail.com
</pre>
The above command will generate two files:
<ol>
<li>cakey.pem
<li>cacert.pem
</ol>
Mulesoft HTTPS TLS configuration support 3 format:
<ol>
<li>JKS -- <a href="https://en.wikipedia.org/wiki/Java_KeyStore">Java Keystore</a>
<li>PKCS12 -- for details refer <a href="https://en.wikipedia.org/wiki/PKCS_12">this page</a>
<li>JCEKS -- Stands for Java Cryptography Extension KeyStore
</ol>
We need to convert the RAS format to PKCS12 using the following command:
<pre class="brush:bash">
$ openssl pkcs12 -export -in cacert.pem -inkey cakey.pem -out identity.p12 -name "mykey"
Enter pass phrase for cakey.pem:
Enter Export Password:
Verifying - Enter Export Password:
</pre>
The above command generate a file namely: identity.p12 with the alias mykey.
Now we can configure the HTTPS Connector.
<H1>Configure HTTPS Connector</H1>
The xml configuration will be like the following:
<pre class="brush:xml">
<http:listener-connection protocol="HTTPS" host="0.0.0.0" port="443" >
<tls:context >
<tls:key-store type="pkcs12" path="identity.p12" alias="mykey" keyPassword="gary" password="gary" />
</tls:context>
</http:listener-connection>
</http:listener-config>
</pre>
The follow snapshots show the procedures using Anypoint Studio:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixh52p6MFVFEFy8KlPUdZUkv4JFy6qhNdA0aECp5BNBhCW2qxgx-LODtAILz-PG1zRKZPjh-KB2zZoFjpoAsMg6C3PMCC7Cxa4e2TJfEBZW30H270Fdi541l05hva-nxL7_NSfRGaeGzww/s1600/https.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixh52p6MFVFEFy8KlPUdZUkv4JFy6qhNdA0aECp5BNBhCW2qxgx-LODtAILz-PG1zRKZPjh-KB2zZoFjpoAsMg6C3PMCC7Cxa4e2TJfEBZW30H270Fdi541l05hva-nxL7_NSfRGaeGzww/s400/https.PNG" width="400" height="399" data-original-width="616" data-original-height="615" /></a></div>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgO_5wcrQC_ZETEDAhGm8EPYF54JJOdOitQyH6il87omQuuEWPpVzAc1_4Q_UgBPWvbSSUcDuFS3MRWBVeYX5HaJk_GVtuuI28PVfGLum-NBE4ip9XRwCph4OC2UUck4An3LnWXUD0mMBbg/s1600/tls.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgO_5wcrQC_ZETEDAhGm8EPYF54JJOdOitQyH6il87omQuuEWPpVzAc1_4Q_UgBPWvbSSUcDuFS3MRWBVeYX5HaJk_GVtuuI28PVfGLum-NBE4ip9XRwCph4OC2UUck4An3LnWXUD0mMBbg/s400/tls.PNG" width="311" height="400" data-original-width="708" data-original-height="911" /></a></div>
<H1>Invoke The Service</H1>
To test the service we can use the following curl command:
<pre class="brush:js">
$ curl -k -XGET https://localhost/helloworld
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 31 100 31 0 0 31 0 0:00:01 0:00:01 --:--:-- 29
{
"message": "Hello, World"
}
</pre>
Note -k option is to tell curl to accepted self-signed certificates.
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com5tag:blogger.com,1999:blog-6083392550327202392.post-67436899215168185542019-08-04T12:45:00.003-07:002019-08-04T12:50:04.942-07:00Dataweave Tricks: Extract Keys and Values From HashMap<H1>Introduction</H1>
I have a requirement to extract the keys and values from a dataset like the following:
<pre class="brush:js">
{
"account1" : {
"accountID" : "1234",
"name" : "Mary Loo",
"balance" : 234.32
},
"account2" : {
"accountID" : "1234",
"name" : "Lauren Flor",
"balance" : 234.32
},
"account3" : {
"accountID" : "1234",
"name" : "Mary Loo",
"balance" : 234.32
}
}
</pre>
Apparently, the above data is a Map. Now we need to produce two dataset of values and keys from the input (Map) as the following:
<pre class="brush:js">
[
{
"accountID": "1234",
"name": "Mary Loo",
"balance": 234.32
},
{
"accountID": "1234",
"name": "Lauren Flor",
"balance": 234.32
},
{
"accountID": "1234",
"name": "Mary Loo",
"balance": 234.32
}
]
</pre>
and
<pre class="brush:js">
[
"account1",
"account2",
"account3"
]
</pre>
<H1>Understanding Dataweave pluck Function</H1>
Dataweave has devised a function particularly for this kind of requirement. Here is the solution to extract values of a HashMap
<pre class="brush:js">
%dw 2.0
output application/json
---
payload pluck (item, key, index) -> item
</pre>
The shorthand version:
<pre class="brush:js">
%dw 2.0
output application/json
---
payload pluck $
</pre>
And to extract the keys:
<pre class="brush:js">
%dw 2.0
output application/json
---
payload pluck (item, key, index) -> key
</pre>
The shorthand version:
<pre class="brush:js">
%dw 2.0
output application/json
---
payload pluck $$
</pre>
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com0tag:blogger.com,1999:blog-6083392550327202392.post-6436430560965656652019-08-03T15:57:00.000-07:002019-08-03T16:26:18.915-07:00Dataweave 2.0 Tricks: Sorting and Grouping<H1>The Challenges</H1>
<p>I have Accounts retrieved from Salesforce like the following:
<pre class="brush:js">
[
{
"LastModifiedDate": "2015-12-09T21:29:01.000Z",
"Id": "0016100000Kngh3AAB",
"type": "Account",
"Name": "AAA Inc."
},
{
"LastModifiedDate": "2015-12-09T20:16:47.000Z",
"Id": "0016100000KnXKhAAN",
"type": "Account",
"Name": "AAA Inc."
},
{
"LastModifiedDate": "2015-12-12T02:06:48.000Z",
"Id": "0016100000KqonvAAB",
"type": "Account",
"Name": "AAA Inc."
},
...
]
</pre>
The dataset contains many accounts which have the same name. These accounts with the same account are regarded as duplicates Eventually I want to delete the duplicates and just leave one in the SFDC. Before I delete the duplicates, I need to create an output for review like the following:
<pre class="brush:js">
{
"AAA Inc.": [
"0016100000Kngh3AAB",
"0016100000KnXKhAAN",
"0016100000KqonvAAB",
"0016100000KnggyAAB",
"0016100000KngflAAB",
"0016100000KqalVAAR",
"0016100000Kngh8AAB",
"0016100000KnVUKAA3",
"0016100000Kngh5AAB",
"0016100000KnVXdAAN",
"0016100000KnVh4AAF",
"0016100000KnVs6AAF",
"0016100000KnggAAAR",
"0016100000KnlokAAB",
"0016100000KnggKAAR"
],
"Adam Smith": [
"0016100000L7sDjAAJ"
],
"Alice John Smith": [
"0016100000L7x29AAB"
],
"Alice Smith.": [
"0016100000L7sDiAAJ"
],
...
</pre>
<H1>Solutions</H1>
I device a two-stage solution. The first transform will create LinkedHashMap which will contain account name as key and the value as array of Account as shown below:
<pre class="brush:js">
%dw 2.0
output application/java
---
//payload groupBy $.Name orderBy $$
(payload groupBy (account) -> account.Name) orderBy (item, key) -> key
</pre>
The second stage of transformation is to extract the account ID as the following:
<pre class="brush:js">
%dw 2.0
output application/java
---
payload mapObject (item, key, index) -> {
(key) : (item map (value) -> value.Id)
}
</pre>
Of course, I can put the two Dataweave scripts into one like the following:
<pre class="brush:js">
%dw 2.0
output application/java
---
//payload groupBy $.Name orderBy $$
((payload groupBy (account) -> account.Name) orderBy (item, key) -> key)
mapObject (item, key, index) -> {
(key) : (item map (value) -> value.Id)
}
</pre>
<H1>Key Learnings</H1>
<p>
The key concept of the above use case is to group the accounts with the same name and sort them in alphabetic order. The Mulesoft document about the groupBy and orderBy together with other core functions of dataweave can be found <a href="https://docs.mulesoft.com/mule-runtime/4.2/dw-core-functions-orderby">here</a> The groupBy and orderBy have the similar signature:
<pre class="brush:js">
1. groupBy(Array<T>, (item: T, index: Number) -> R): { (R): Array<T> }
2. groupBy({ (K)?: V }, (value: V, key: K) -> R): { (R): { (K)?: V } }
3. groupBy(Null, (Nothing, Nothing) -> Any): Null
</pre>
The first function indicates that it can take array as input. The usage will like the following:
<pre class="brush:js">
//payload groupBy $.Name
//payload groupBy (account, index) -> account.Name
payload groupBy (account) -> account.Name
</pre>
The above code are the same. The first one is a short-cut version. The second and third lines are to show the lambda style. As a good developer, you should know all the syntax.
<p>In my solution of the second stage, I use mapObject function as the following:
<pre class="brush:js">
payload mapObject (item, key, index) -> {
(key) : (item map (value) -> value.Id)
}
</pre>
This is because the payload is a LinkedHashMap and the value of each HashMap entry is an array. That is why I have to use map function inside the mapObject function.
<p>In my work, I also need to remove the type attribute in the Account object.
<pre class="brush:js">
[
{
"LastModifiedDate": "2015-12-09T21:29:01.000Z",
"Id": "0016100000Kngh3AAB",
"type": "Account",
"Name": "AAA Inc."
},
...
]
</pre>
Here is the transform to remove the type:
<pre class="brush:js">
(payload orderBy (item) -> item.Name) map (account) -> {
(account -- ['type'])
}
</pre>
As you can see I have used function of --.
<H1>Summary</H1>
The key thinking of solve this kind of problem how to group, sort, and extract values from the Array or LinkedHashMap. Thus I have used the following core Dataweave functions:
<ul>
<li>groupBy</li>
<li>orderBy</li>
<li>map</li>
<li>mapObject</li>
</ul>
Also, we should know the short-hand way and Lambda style for using the Dataweave functions. The short-hand way is to use build-in variable $, $$, $$$. </br>
If the payload is Array
<ul>
<li> $ - item</li>
<li> $$ - index</li>
</ul>
If the payload is LinkedHashMap
<ul>
<li> $ - value </li>
<li> $$ - key </li>
<li> $$$ - index. </li>
</ul>
The Lambda style is like the following: </br>
<pre class="brush:js">
(payload orderBy (item) -> item.Name) map (account, index) -> {
(account -- ['type'])
}
</pre>
<pre class="brush:js">
mapObject (item, key, index) -> {
(key) : (item map (value) -> value.Id)
}
</pre>
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com2tag:blogger.com,1999:blog-6083392550327202392.post-45586807475169193962019-08-03T11:02:00.001-07:002019-08-03T11:02:43.812-07:00Mule Application Hacking: Reveal Details Of A Connector's Connection<H1>Introduction</H1>
<p>
In my <a href="Hacking Mule Application - View Source Code">last hacking post</a>, I described the method to view the source code of Mule connectors using the ECD. In this article, I am go demonstrate the procedure to reveal the details about connection and communication within the Mule Connector. This is very useful for the purpose of trouble-shooting.
<p>
I will use Salesforce connector as example to demonstrate how to read connection details, query, etc.
<H1>Details</H1>
First, looking for the connector as shown in the following snapshots:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6gInDXe61d2qXu2-w4zqmpj6emljFsjYN6tuYFgHLtR6S2wEYBkp6T0l72F-cS-uY5aEPSo3D-81RD1bjXlEhQz-6qLTcKR-7K9fdE0r8z9N9SXygyy8yoch2kdSRwpBIV3vPnILsCKkI/s1600/Screen+Shot+2019-08-03+at+12.49.31+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6gInDXe61d2qXu2-w4zqmpj6emljFsjYN6tuYFgHLtR6S2wEYBkp6T0l72F-cS-uY5aEPSo3D-81RD1bjXlEhQz-6qLTcKR-7K9fdE0r8z9N9SXygyy8yoch2kdSRwpBIV3vPnILsCKkI/s400/Screen+Shot+2019-08-03+at+12.49.31+PM.png" width="400" height="309" data-original-width="490" data-original-height="378" /></a></div>
As you can see, I have added Salesforce connection version 9.7.7. Now, we need to expand the connector. Then looking for mule-salesforce-connector-9.7.7-mule-pluging.jar and expand the jar file as shown below:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlaCuM4Z9O517r2Vge3T8qTb5ZIDqyV11gPuujD3BG3HC-gZlg2OL7dAMH56GwReCDpd0OXXfJPxWH555nABPEX0rK58Oan1j5CssGP_ClIXJ9IZQvAZ-F5jrtTxwEjq3JlI50NE1M_Xff/s1600/Screen+Shot+2019-08-03+at+12.55.11+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlaCuM4Z9O517r2Vge3T8qTb5ZIDqyV11gPuujD3BG3HC-gZlg2OL7dAMH56GwReCDpd0OXXfJPxWH555nABPEX0rK58Oan1j5CssGP_ClIXJ9IZQvAZ-F5jrtTxwEjq3JlI50NE1M_Xff/s400/Screen+Shot+2019-08-03+at+12.55.11+PM.png" width="266" height="400" data-original-width="477" data-original-height="717" /></a></div>
Now we can see that the java class packages all under org.mule.extension.salesforce.
<p>
The next step is to add the package to log4j.xml in the dir of src/main/resources as shown below:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0mnMHvDaSxBbpATQh3xLlFHirMecTipLnAdmyxBtrJXoYkmFNS33iS4DXUcqf3c3Txp4daWuFjE4iEi0aLS7lMI2XN70njBObpC0YRbJstBcQ5I0VmrUgcO6I2as0T5NDmzy7KdvS8bg7/s1600/Screen+Shot+2019-08-03+at+12.57.50+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0mnMHvDaSxBbpATQh3xLlFHirMecTipLnAdmyxBtrJXoYkmFNS33iS4DXUcqf3c3Txp4daWuFjE4iEi0aLS7lMI2XN70njBObpC0YRbJstBcQ5I0VmrUgcO6I2as0T5NDmzy7KdvS8bg7/s400/Screen+Shot+2019-08-03+at+12.57.50+PM.png" width="400" height="321" data-original-width="468" data-original-height="376" /></a></div>
Now add the package of org.mule.extension.salesforce into the Loggers section of the log4j.xml file as shown below
<pre class="brush:xml;">
<Loggers>
<!-- Http Logger shows wire traffic on DEBUG. -->
<!--AsyncLogger name="org.mule.service.http.impl.service.HttpMessageLogger" level="DEBUG" /-->
<AsyncLogger name="org.mule.service.http" level="WARN"/>
<AsyncLogger name="org.mule.extension.http" level="WARN"/>
<AsyncLogger name="org.mule.extension.salesforce" level="DEBUG"/>
<!-- Mule logger -->
<AsyncLogger name="org.mule.runtime.core.internal.processor.LoggerMessageProcessor" level="INFO"/>
<AsyncRoot level="INFO">
<AppenderRef ref="file" />
</AsyncRoot>
</Loggers>
</pre>
Now, if you run the project, the console will deplay the debugging information about the connection information, request, and response from Salesforce connectors.
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com3tag:blogger.com,1999:blog-6083392550327202392.post-70282431481579468162019-07-27T10:35:00.000-07:002019-07-27T10:35:47.167-07:00Hacking Mule Application - View Source Code<H1>Introduction</H1>
<p>
To become a real good Mule developer, we need to understand the source code of Mule connectors, components, and other internal source code. This helps us to learn the internal data model and interfaces of Mule classes. I have written a <a href="http://ggl-consulting.blogspot.com/2018/05/install-and-configure-java-decompiler.html">blog</a> on how to compile and install the ECD in Anypoint Studio. However, that building system is broken for Mule 4. Hopefully, it will be fixed soon.
<p>
This article demonstrate another way to view the source code. It is not the best solution yet, but it help. The idea is to use Eclipse IDE and ECD plugin to review the java source code.
<P>
ECD is so far the best Java Decompiler available for Eclipse. The details can be found <a href="https://ecd-plugin.github.io/ecd/">at here</a>
<H1> Install Eclipse & ECD Plugin </H1>
First download the Eclipse EE from <a href="https://www.eclipse.org/downloads/packages/release/kepler/r/eclipse-ide-java-ee-developers">this site.</a>
Second, create a java project.
Third, install ECD Eclipse Plugin. go to <a href="https://marketplace.eclipse.org/category/free-tagging/ecd">Eclipse Market Place of ECD</a></br>
Drag the install to your project.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNpDa1TUlUF6ctlDQfvPRlct6gg0NMuq4Yb40IMxNRpu6S0DfXF96l958_fSVe_U41jSmPPFfHegWKwlUGjohDQJjEjUkFtUoyhR4ZK69FjVH4xKyxMJQmPMfT9ll71vgXXDI2hXIuJ9aO/s1600/Screen+Shot+2019-07-27+at+12.21.12+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNpDa1TUlUF6ctlDQfvPRlct6gg0NMuq4Yb40IMxNRpu6S0DfXF96l958_fSVe_U41jSmPPFfHegWKwlUGjohDQJjEjUkFtUoyhR4ZK69FjVH4xKyxMJQmPMfT9ll71vgXXDI2hXIuJ9aO/s400/Screen+Shot+2019-07-27+at+12.21.12+PM.png" width="400" height="201" data-original-width="1543" data-original-height="777" /></a></div>
Now to preferences, you should see the Decompiler under Java as shown belog:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHlCDebmaUIIUvvxuFIgvmlJABo-5xwJ5pd-C7rw6YP0H-NCuQLXN1-b1rWuPFrbBKctc1MK5-QnUdSreUHcirChZeqpwQmk33hbHzOhSZh9UZgVw7tPeQnhQO9EGDMXIF977akSjx3pt7/s1600/Screen+Shot+2019-07-27+at+12.24.48+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHlCDebmaUIIUvvxuFIgvmlJABo-5xwJ5pd-C7rw6YP0H-NCuQLXN1-b1rWuPFrbBKctc1MK5-QnUdSreUHcirChZeqpwQmk33hbHzOhSZh9UZgVw7tPeQnhQO9EGDMXIF977akSjx3pt7/s400/Screen+Shot+2019-07-27+at+12.24.48+PM.png" width="400" height="344" data-original-width="624" data-original-height="537" /></a></div>
Fourth, import jar archive. right click the project --> select Achive File
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuAYJMWQDD11dIWkJUpQBWn_FpuQ-3h_d2qiOUOX9UqOtfZSLI95JzTsGaZKf1IunMS4yC0eVKSyGesjDVyoOEJOlb35loGRPbCvx4uZZxFkdK8TcfHy5CuT3ceqOY_6V5EBv2IfOiIAr_/s1600/Screen+Shot+2019-07-27+at+12.11.38+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuAYJMWQDD11dIWkJUpQBWn_FpuQ-3h_d2qiOUOX9UqOtfZSLI95JzTsGaZKf1IunMS4yC0eVKSyGesjDVyoOEJOlb35loGRPbCvx4uZZxFkdK8TcfHy5CuT3ceqOY_6V5EBv2IfOiIAr_/s400/Screen+Shot+2019-07-27+at+12.11.38+PM.png" width="384" height="400" data-original-width="529" data-original-height="551" /></a></div>
browser the file from your local maven repository. In my case, it is at .m2/repository/org/mule/connectors/mule-http-connector/1.5.3. </br>
Import the jar file to the newly created project. Drill down the classed you are interested as shown in the following snapshot:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhyphenhypheneUCk14EqimLDCOrr2usfccIBnszkd4TlbHDNzFNKMHUyvs0pGzGYalcQN1pY7ylFCpN9evLcR4V9zzmuCeC8gzByIgeVJZHWhcC3rqzldK0L5BSgK5Iyw91tCRQSIQZ_a5nJ8G7oagl/s1600/Screen+Shot+2019-07-27+at+12.17.10+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhyphenhypheneUCk14EqimLDCOrr2usfccIBnszkd4TlbHDNzFNKMHUyvs0pGzGYalcQN1pY7ylFCpN9evLcR4V9zzmuCeC8gzByIgeVJZHWhcC3rqzldK0L5BSgK5Iyw91tCRQSIQZ_a5nJ8G7oagl/s400/Screen+Shot+2019-07-27+at+12.17.10+PM.png" width="400" height="248" data-original-width="1600" data-original-height="992" /></a></div>
Note that you have to chose the decompiler by right click java class --> open with, select as shown below:
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkfI0iPfD6I4ua87Q2i1XrPPJl8hyphenhyphen7MfSUHXmyosFOZJl5AKpah7uwSXtLr7ZEdOwNn8dfeF3TYBlcRZLQHjAun8PIF9l5gVyMQlBkYpRNabzWDXKDL-JOia9EvhLu9ShNu23lwqxxjk1K/s1600/Screen+Shot+2019-07-27+at+12.28.48+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkfI0iPfD6I4ua87Q2i1XrPPJl8hyphenhyphen7MfSUHXmyosFOZJl5AKpah7uwSXtLr7ZEdOwNn8dfeF3TYBlcRZLQHjAun8PIF9l5gVyMQlBkYpRNabzWDXKDL-JOia9EvhLu9ShNu23lwqxxjk1K/s400/Screen+Shot+2019-07-27+at+12.28.48+PM.png" width="319" height="400" data-original-width="496" data-original-height="621" /></a></div>
At this point, we can review the source code. The next step is to decompiler the whole jar file and rebuild it with source. In this we can debug the source code and modify the behavior of the connectors. I will cover the procedure later.Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com4tag:blogger.com,1999:blog-6083392550327202392.post-82210621431367036702019-06-29T14:06:00.000-07:002019-07-11T16:10:03.655-07:00SSL Handshake Failure Connecting To Mulesooft Anypoint Exchange In Corporate Environment<H1>The Issue</H1>
As a Mulesoft developer, we will need to download connectors from Anypoint exchange periodically.
When we try to connect to Mulesoft Anypoint Exchange, which is the repository for Mulesoft related connectors and other libraries, we may get SSH Handshake exception, in particular, using corporate provided laptop.
Here is the top part of the exception message:
<pre class="brush:js">
eclipse.buildId=unknown
java.version=1.8.0_212
java.vendor=Oracle Corporation
BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=en_US
Command-line arguments: -os win32 -ws win32 -arch x86_64
org.mule.tooling.core
Error
Thu Jul 11 17:49:09 CDT 2019
The following exceptions were encountered while resolving dependency com.mulesoft.connectors:mule-salesforce-connector:9.7.6: java.lang.RuntimeException: There was an issue resolving the dependency tree for the bundleDescriptors [[BundleDescriptor{groupId='com.mulesoft.connectors', artifactId='mule-salesforce-connector', baseVersion='null', version='9.7.6', type='jar', classifier=Optional[mule-plugin]}, BundleDescriptor{groupId='org.mule.connectors', artifactId='mule-objectstore-connector', baseVersion='null', version='1.0.0', type='jar', classifier=Optional[mule-plugin]}]]
at org.mule.maven.client.internal.AetherMavenClient.resolvePluginBundleDescriptorsDependencies(AetherMavenClient.java:322)
at org.mule.tooling.core.m2.internal.MuleMavenClientResolver.resolvePluginDependencies(MuleMavenClientResolver.java:80)
at org.mule.tooling.core.module.internal.runner.DownloadTask.doRun(DownloadTask.java:76)
at org.mule.tooling.core.module.internal.runner.Task.run(Task.java:65)
at org.mule.tooling.core.module.internal.runner.DownloadTask.run(DownloadTask.java:1)
at org.mule.tooling.core.module.internal.runner.ArtifactResolvingRunner$ArtifactJob.run(ArtifactResolvingRunner.java:212)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:56)
Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to collect dependencies at com.mulesoft.connectors:mule-salesforce-connector:jar:mule-plugin:9.7.6 -> com.mulesoft.connectors:mule-connector-commons:jar:2.1.1
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:291)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies(DefaultRepositorySystem.java:316)
at org.mule.maven.client.internal.AetherMavenClient.doResolveDependencies(AetherMavenClient.java:408)
at org.mule.maven.client.internal.AetherMavenClient.resolvePluginBundleDescriptorsDependencies(AetherMavenClient.java:314)
... 6 more
Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for com.mulesoft.connectors:mule-connector-commons:jar:2.1.1
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:282)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:198)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.resolveCachedArtifactDescriptor(DefaultDependencyCollector.java:535)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.getArtifactDescriptorResult(DefaultDependencyCollector.java:519)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:409)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:504)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:254)
... 9 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.mulesoft.connectors:mule-connector-commons:pom:2.1.1 from/to mulesoft-releases (https://repository.mulesoft.org/releases/): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:267)
... 20 more
</pre>
This article describes the procedures to fix this kind of issues.
<H1>Find The Root Cause</H1>
<p>Problems solving skills are really about to find the root cause of the issue. In this issue, if we look the error message carefully, we will find the following:
</p>
<pre class="brush:bash">
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.mulesoft.connectors:mule-connector-commons:pom:2.1.1 from/to mulesoft-releases (https://repository.mulesoft.org/releases/): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
</pre>
<p>
What this error message is saying that Java process was trying to transfer data from host of: repository.mulesoft.org. The problem is the SSL Handshake. To resolve this kind of problem we need to import the certificate from the site to cacerts.
</p>
<H1>Tasks</H1>
<p>In this article, I only import 3 certificates from the following site:
<ul>
<li>anypoint.mulesoft.com</li>
<li>maven.anypoint.mulesoft.com</li>
<li>release.anypoint.mulesoft.com</li>
</ul>
</p>
<H1>Prerequisites</H1>
<ol>
<li>On Windows install Cygwind</li>
<li>Have Admin privilege of the laptop</li>
</ol>
<H1>Solutions</H1>
When we connect to Mulesoft Anypoint Exchange, AnypointStudio needs to go through SSL Handshake procedure before we can see the download page. If this process failed, typically, AnypointStudio (a java process) could not store the certificate), the SSHHandshakeException will be thrown by the studio.
The following steps will fix the issue:
<br/><br/>
Step One: Download certificate from anypoint.mulesoft.com
<pre class="brush:bash;">
openssl s_client -connect anypoint.mulesoft.com:443 -showcerts </dev/null 2>/dev/null |openssl x509 -outform PEM >anypoint.pem
</pre>
Step Two: Download certificate from maven.anypoint.mulesoft.com
<pre class="brush:bash;">
openssl s_client -connect maven.anypoint.mulesoft.com:443 -showcerts </dev/null 2>/dev/null | openssl x509 -outform PEM >mulesoft.maven.pem
</pre>
Step Three: Download certificate from repository.anypoint.mulesoft.com
<pre class="brush:bash;">
openssl s_client -connect repository.mulesoft.org:443 -showcerts </dev/null 2>/dev/null | openssl x509 -outform PEM >mulesoft.repo.pem
</pre>
Step Four: Copy the 3 certificates
<pre class="brush:bash;">
cp *.pem /cygdrive/c/’Program Files’/Java/jdk1.8.0_212/jre/lib/security
</pre>
As you can see that I am using cygwin on Windows. On Macbook Pro, the JAVA_HOME is may be different. In this case the JAVA_HOME is under:
<pre class="brush:bash;">
/cygdrive/c/’Program Files’/Java/jdk1.8.0_212
</pre>
Step Five: Import the certificate to cacerts
<pre class="brush:bash;">
/cygdrive/c/’Program Files’/Java/jdk1.8.0_212/jre/lib/security
keytool -import -alias anypoint -keystore cacerts -file anypoint.pem
</pre>
Repeat the same procedure for the other 2 certificaes.
Step Fix: Restart Anypoint Studio
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com4tag:blogger.com,1999:blog-6083392550327202392.post-722292377216434642019-03-16T12:07:00.001-07:002019-03-19T18:49:28.545-07:00Email Address Validation Using Dataweave Regex<H1>Introduction</H1>
This short article is about how to use regex with Dataweave 2.0 with regard to email validation. Regex is used in Mulesoft language very wide with regard two functions, matchs(...) and match(...). It is very important to master the regular expression in order to be professional in Mulesoft integration projects.
The are a lot of reference available one. Here are few:
<ul>
<li>
<a href="https://www.rexegg.com/regex-quickstart.html">Regex Cheat Sheet for more experience developer</a>
<li>
<a href="https://docs.microsoft.com/en-us/dotnet/standard/base-types/regular-expression-language-quick-reference">Reference from Microsoft</a>
<li>
<a href="https://www.regular-expressions.info/reference.html">More complete reference</a>
</ul>
<H1>Use Case</H1>
We expect the output of the dataweave transformation as the following depending on the validity of the email address:
<pre class="brush:js;">
[
{
...
"invalidEmail": "johndo@yahoo"
...
},
{
...
"PersonEmail": "john.smith@google.com"
...
},
...
]
</pre>
<H1>Invalid Emails</H1>
The following types of emails are invalid:
<ol>
<li>beginning with a dot: .gary.liu@google.com</li>
<li>ending with a dot: gary.liu@google.com.</li>
<li>double dots: gary.liu@google..com</li>
<li>domain name contains underscore: gary.liu@att_rr.com</li>
<li>domain name contains space: gayr.liu@att rr.com</li>
<li>domain name contains and of the following: ,<>/[]</li>
<li>no organization email: gary@google</li>
</ol>
<H1>Solution</H1>
<pre class="brush:js;">
%dw 2.0
output application/json
var regexEmail = /^[^.][a-zA-Z0-9.!#$%&’*+\/=?^_`{|}~-]+@[a-zA-Z0-9-](?!.*?\.\.)[^_ ; ,<>\/\\]+(?:\.[a-zA-Z0-9-]+)[^.]*$/
---
payload map using (email = $.email) {
(validEmail: email) if (email matches regexEmail),
(invalidEmail: email) if ( not (email matches regexEmail))
}
</pre>
The above dataweave script is self-explanatory. Few explanation is required if you are not very familiar with regular expression:
<ol>
<li>negation: [^_;,\.] this expression means if the email domain contains underscore _ , semi-coma, etc. is not valid email</li>
<li> simple ^ and $ represent the beginning and end of the line</li>
<li>[a-zA-Z0-9] mean any charater A a, Bb ... Zz, or 0 to 9 digits are valid</li>
<li> + sign means to match one for more </li>
<li> * sign matches 0 or more </li>
<li> ?! means not include, (?!.*?\.\.) --> not include double dots: ..</li>
</ol>Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com5tag:blogger.com,1999:blog-6083392550327202392.post-51329058357110941282019-03-02T09:44:00.004-08:002019-03-02T09:46:39.633-08:00How To Resolve Issue With: "General SSLEngine problem"<H1>The Background</H1>
This happens when you enable the HTTPS with your own certificates. In my case, I have configured Anypoint runtime fabrics with self generated certification using the following command:
<pre class="brush:bash">
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365
</pre>
The above command generates two files: cert.pem and key.pem. The purpose of them is beyond the scope of the article.
The error will occur when the local mule flow call the remote application which is deployed in the Anypoint Runtime Fabrics.
<H1>Solution</H1>
To resolve this problem, we just need to import the cert.pem to cacerts file. The command is (on MacOs):
<pre class="brush:bash">
cd /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/security
sudo keytool -import -trustcacerts -keystore cacerts -storepass changeit -alias rogers-poc-cert -file /Users/gl17/anypoint/certs/poc/cert.pem
</pre>
Make sure restart Anypoint Studio.
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com2tag:blogger.com,1999:blog-6083392550327202392.post-91008488804021582982019-01-14T08:01:00.001-08:002020-01-22T02:40:32.649-08:00Deploy Mule 4 Application To Anypoint Runtime Fabric Using Maven Plugin<H1>Introduction</H1>
In CI/CD process, it is very common to use Mule Maven plugin to build and deploy application to the Cloudhub, on-premise private cloud like AWS, AZure, Google Cloud, etc. Since Mule 4, a lot of changes related to the deployment has changed, in particular, related to the Mule Runtime Fabric (RTF). Actually, RTF is a completely new infrastructure for Mule application deployment. I will cover more on that topic later.
In this article, I am going to cover the following topics related to the deployment to Anypoint Runtime Fabric (RTF):
<ol>
<li>Prepare pom.xml setup to deploy mule project to Anypoint RTF</li>
<li>Encrypt password</li>
<li>Troubleshooting</li>
</ol>
<p>
If everything works, at the end, we should be able to achieve the following goals:
<ul>
<li>deploy mule projects (assets) to Anypoint Exchange</li>
<li>deploy mule projects to Anypoint Runtime Fabric</li>
</ul>
</p>
<p>
In order for mule maven plugin to work, we need change the following two files:
<ul>
<li>pom.xml</li>
<li>./m2/settings.xml</li>
</ul>
</p>
<p>
The complete project for this article can be find at <a href="https://github.com/garyliu1119/mule-maven-plugin/tree/master/mule-maven-plugin">my github repository</a>
</p>
<H1>Settings.xml</H1>
In order to deploy mule applications using mule maven plugin, we need to set the Anypoint credentials, which are the ones we use to login to anypoint portal (http://anypoint.mulesoft.com). The best way to do this is to add server information in the .m2/settings.xml as the following:
<pre class="brush: xml;">
<servers>
<server>
<id>ExchangeRepository</id>
<username>gary_liu_client</username>
<password>{5XLgXNDBGBIHa99xNAyJ6gL+ZxyUiyIJNHWu0H7Ctew=}</password>
</server>
</servers>
</pre>
The password is encrypted. To encrypt password we need to add another file namely: ~/.m2/settings-security.xml:
<pre class="brush: xml;">
<settingsSecurity>
<master>{Vq5Aso1ZkO4HdJrUscJTZEii4BcFy+khGiGxDNVNgc4=}</master>
</settingsSecurity>
</pre>
The encrypt master password in the above is created by the following commands:
<pre class="brush:bash;">
$ mvn --encrypt-master-password MasterPassword
{+4nnH6EW9HcHAHYBGnloFCAZZHSC4W3Xp9Zls0LvBqk=}
</pre>
Once we encrypted the master password, we can encrypt the password to the anypoint portal as the following:
<pre class="brush:bash;">
mvn --encrypt-password AnypointPortalPassword
</pre>
For more details about the maven password encryption, you may refer to the following page: https://maven.apache.org/guides/mini/guide-encryption.html
<H1>pom.xml</H1>
<pre class="brush: xml; ">
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>fea874ca-11d9-4779-b1ce-90d49f738259</groupId>
<artifactId>mule-maven-plugin</artifactId>
<version>1.0.2</version>
<packaging>mule-application</packaging>
<name>mule-maven-plugin</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<mule.maven.plugin.version>3.2.3</mule.maven.plugin.version>
<anypoint.uri>https://anypoint.mulesoft.com</anypoint.uri>
<anypoint.provider>MC</anypoint.provider>
<deployment.environment>QA</deployment.environment>
<!-- <deployment.target>qa-azure-rtf</deployment.target> -->
<app.runtime>4.1.4</app.runtime>
<app.name>gary-deployment</app.name>
<app.cores>100m</app.cores>
<app.memory>500Mi</app.memory>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.mule.tools.maven</groupId>
<artifactId>mule-maven-plugin</artifactId>
<version>${mule.maven.plugin.version}</version>
<extensions>true</extensions>
<configuration>
<runtimeFabricDeployment>
<uri>${anypoint.uri}</uri>
<provider>${anypoint.provider}</provider>
<environment>${deployment.environment}</environment>
<target>${deployment.target}</target>
<muleVersion>${app.runtime}</muleVersion>
<server>ExchangeRepository</server>
<applicationName>${app.name}</applicationName>
<deploymentSettings>
<replicationFactor>${deployment.replica}</replicationFactor>
<cpuReserved>${app.cores}</cpuReserved>
<memoryReserved>${app.memory}</memoryReserved>
</deploymentSettings>
</runtimeFabricDeployment>
<classifier>mule-application</classifier>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>${maven.properties}</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<distributionManagement>
<repository>
<id>ExchangeRepository</id>
<name>Corporate Repository</name>
<url>https://maven.anypoint.mulesoft.com/api/v1/organizations/${groupId}/maven</url>
<layout>default</layout>
</repository>
</distributionManagement>
<dependencies>
......
</dependencies>
......
</project>
</pre>
The details are explained in the next section. But one item I must point out here: <groupId>fea874ca-11d9-4779-b1ce-90d49f738259</groupId>. The groupId is your Anypoint platform organization ID.
<H1>Explanations</H1>
<p>The configuration in both pom.xml and settings.xml are pretty straightforward. Few items worths to explain. First, what is the latest version of mule maven plugin?
This can be found at: https://docs.mulesoft.com/release-notes/mule-maven-plugin/mule-maven-plugin-release-notes. Currently, the latest verson is 3.2.3.</p>
<p>
Secondly, I use another plugin in addition to the mule maven plugin, namely, properties-maven-plugin. This plugin allow us to pass a property file to the pom.xml.
</p>
<pre class="brush: xml;">
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>${maven.properties}</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
</pre>
Note that in the configuration, we put ${maven.properties} variable. This allow as to pass the property file as:
<pre class="brush:bash;">
mvn clean deploy -DmuleDeploy -Dmaven.properties=src/main/resources/mule.rtf.deploy.properties
</pre>
Thirdly, note that in the mule maven plugin, I use <server>ExchangeRepository</server> as shown below. The ExchangeRespository is the server defined in the ~/.m2/settings.xml
<pre class="brush: xml">
<configuration>
<runtimeFabricDeployment>
<uri>https://anypoint.mulesoft.com</uri>
<provider>MC</provider>
<environment>QA</environment>
<target>qa-azure-rtf</target>
<muleVersion>4.1.4</muleVersion>
<server>ExchangeRepository</server>
<applicationName>${app.name}</applicationName>
<deploymentSettings>
<replicationFactor>1</replicationFactor>
<cpuReserved>100m</cpuReserved>
<memoryReserved>500Mi</memoryReserved>
</deploymentSettings>
</runtimeFabricDeployment>
<classifier>mule-application</classifier>
</configuration>
</pre>
The details for the parameters can be referred at: https://docs.mulesoft.com/mule-runtime/4.1/runtime-fabric-deployment-mmp-reference.
<H1>Publish Assets To Exchange</H1>
In order to deploy mule application to RTF using Mule Maven Plugin, we must publish the application (asset to Anypoint Exchange). To me this is extra step unnecessary.
To publish an artifact (asset) to the Anypoint Exchange, we run the following command:
<pre class="brush:bash;">
mvn clean package deploy -Dmaven.properties=src/main/resources/mule.rtf.deploy.properties
</pre>
After it completes the publishing, you can verify the result by the Anypoint Port using the following web address: <br/>
https://anypoint.mulesoft.com/exchange/{groupId}/{artifactId} <br/>
Here is an my example: <br>
https://anypoint.mulesoft.com/exchange/fea874ca-11d9-4779-b1ce-90d49f738259/mule-maven-plugin/
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5nLeIKv0EIeqNeZmIWyV3HiNVlPsMHEYdJU6QTyBymstm4ks5r86lHkZ3aU8r94qwWpeP1jYkrAA2nDD50LAeAYEOJzOSaouqCFzentbb_CNhZjYpBE_WoEy6VGmUHEH9XxiaVDXuf5QQ/s1600/mule-maven-plugin-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5nLeIKv0EIeqNeZmIWyV3HiNVlPsMHEYdJU6QTyBymstm4ks5r86lHkZ3aU8r94qwWpeP1jYkrAA2nDD50LAeAYEOJzOSaouqCFzentbb_CNhZjYpBE_WoEy6VGmUHEH9XxiaVDXuf5QQ/s320/mule-maven-plugin-1.png" width="320" height="171" data-original-width="1600" data-original-height="855" /></a></div>
<H1>Deploy Application To RTF</H1>
Deployment to RTF could be slow and sometimes could take very long time if it failed. I think the plugin should be improved on this by using asynchronous deployment instead of waiting and constantly checking with the runtime manager. Any way the deployment to RTF can be done using the following command:
<pre class="brush:bash;">
mvn clean package deploy -DmuleDeploy -Dmaven.properties=src/main/resources/mule.rtf.deploy.properties
</pre>
<H1>Take Aways</H1>
The commands used in the process:
<pre class="brush:bash;">
mvn --encrypt-master-password GaryLiu1234
mvn --encrypt-password GaryLiu1234
mvn clean package deploy-Dmaven.properties=src/main/resources/mule.rtf.deploy.properties
mvn clean package deploy -DmuleDeploy -Dmaven.properties=src/main/resources/mule.rtf.deploy.properties
</pre>
Web address to check assets in the Anypoint Exchange in the Anypoint Portal: <br>
https://anypoint.mulesoft.com/exchange/{groupId}/{artifactId}
Key Considerations using the Mule Maven Plugin in the CI/CD:
<ul>
<li>Make sure don't wait for the deployment to finish as it can take very long time</li>
<li>There are rest API you can check the deployment process. You should use the APIs to check the deploy</li>
<li>At the moment, anypoint-cli is not working for RTF</li>
</ul>
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com7tag:blogger.com,1999:blog-6083392550327202392.post-8155285652882462522018-12-15T14:26:00.001-08:002018-12-15T15:02:47.279-08:00Mule 4 Integration With Cassandra<div dir="ltr" style="text-align: left;" trbidi="on">
<h1>
Introduction</h1>
Mule 4 Cassandra Database Connector, current version of 3.10, has made significant improvement over the previous version. It is fairly straight forward to setup the integration to the Cassandra Database using the connector. This article is an introduction of the Cassandra Connector to Cassandra cluster.
<br />
<h1>
Cassandra Cluster Configuration</h1>
I have created 2 a two node cluster as shown the in following diagram. The details on how to setup the Cassandra clustering will be covered in another post. Basically, I opened the native transport port 9042, which is the default vale on both nodes.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnzCyhwsoZMcHBCl2YUTjfs61Po7w4W8q3bYin6XQCwA3N2Rk4zQQE3BfI7tuhmAUzJBL4E-0wRv25soo8H2s47lgW_fYDNeNRpJfSawKQOFatqo1R_l3XsJoqEVcZvS_yAH0l7WBDtrY7/s1600/REAL+Architecture+-+Cassandra+%25281%2529.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="675" data-original-width="1163" height="232" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnzCyhwsoZMcHBCl2YUTjfs61Po7w4W8q3bYin6XQCwA3N2Rk4zQQE3BfI7tuhmAUzJBL4E-0wRv25soo8H2s47lgW_fYDNeNRpJfSawKQOFatqo1R_l3XsJoqEVcZvS_yAH0l7WBDtrY7/s400/REAL+Architecture+-+Cassandra+%25281%2529.png" width="400" /></a></div>
We need to some initial setup using cqlsh too by run the following command:
<br />
<pre class="brush:js">$ cqlsh -u cassandra -p casandra
cassandra@cqlsh> create keyspace if not exists dev_keyspace with replication = {'class' : 'SimpleStrategy', 'replication_factor' : 2};
</pre>
The above command will create a keyspace, namely dev_keyspace. We can query the keyspaces by the following command:
<br />
<pre class="brush:js">cassandra@cqlsh> desc keyspaces;
</pre>
You should see the following:
<br />
<pre class="brush:js">system_schema system system_distributed
system_auth dev_keyspace system_traces
</pre>
The next step is to create emp table by the following command:
<br />
<pre class="brush:js">cassandra@cqlsh> create table emp (empid int primary key, emp_first varchar, emp_last varchar, emp_dept varchar);
</pre>
And insert a row:
<br />
<pre class="brush:js">create table emp (empid int primary key, emp_first varchar, emp_last varchar, emp_dept varchar);
insert into emp (empid, emp_first, emp_last, emp_dept) values (1, 'Gary','liu','consulting');
</pre>
That is all and we are ready to do the integration.<br />
<h1>
Integration Using Mule 4 Cassandra Connector</h1>
<h2>
Add Cassandra Connector</h2>
First, we need to search the Exchange and add the Cassandra Connection as shown in the following snapshot:
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4XMkEKP-tHGKJRMafazLARv28hvEB42cWbtzbRJiVDG85BSNsrbzs7Ek-xudio9d7mUASXNcm-1cCuIyOUDtVfT3WrTo9EJRFd_LM_rewHrjhGLq2f573qOZD4QJnXaVKnm1M-IdKSxta/s1600/Add+Cassandra+Module.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="538" data-original-width="828" height="260" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4XMkEKP-tHGKJRMafazLARv28hvEB42cWbtzbRJiVDG85BSNsrbzs7Ek-xudio9d7mUASXNcm-1cCuIyOUDtVfT3WrTo9EJRFd_LM_rewHrjhGLq2f573qOZD4QJnXaVKnm1M-IdKSxta/s400/Add+Cassandra+Module.png" width="400" /></a></div>
<h2>
Create CassandraDB Config</h2>
Create a mule configuration, namely, global-config.xml. Then create CasandraDB Config as the following:
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipYi3STQ6zZap3AMlx_Ayza4GYKJ9nhY-DUJlrGpb3y_NpuK6fGrWu3qqQRvOHEJKcNZ-2P_5_tMUcIB0RCcHb3XpNhCzpS4UB3QZUjO9NGWbw64LqAjkXPr5u0_5H26nOUGF8F-YN3vF-/s1600/Screen+Shot+2018-12-15+at+4.10.28+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="626" data-original-width="627" height="399" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipYi3STQ6zZap3AMlx_Ayza4GYKJ9nhY-DUJlrGpb3y_NpuK6fGrWu3qqQRvOHEJKcNZ-2P_5_tMUcIB0RCcHb3XpNhCzpS4UB3QZUjO9NGWbw64LqAjkXPr5u0_5H26nOUGF8F-YN3vF-/s400/Screen+Shot+2018-12-15+at+4.10.28+PM.png" width="400" /></a></div>
Enter the General setting as the following. Note: Leave the Host empty as we use cluster configuration.
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtdAEwSf5lrfmvFZFSu0qWe_kJtLZEElV8RZzd9LgE3kaSBtJPtvxOcuyES47vf55UYRPFrpXCwQpDMDL98yWjRPenvdPzduNc6IocliNlnjGUtqHYHHLgVzJbP43KqU4aueBLbb6HWR6_/s1600/Screen+Shot+2018-12-15+at+4.08.41+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="624" data-original-width="626" height="399" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtdAEwSf5lrfmvFZFSu0qWe_kJtLZEElV8RZzd9LgE3kaSBtJPtvxOcuyES47vf55UYRPFrpXCwQpDMDL98yWjRPenvdPzduNc6IocliNlnjGUtqHYHHLgVzJbP43KqU4aueBLbb6HWR6_/s400/Screen+Shot+2018-12-15+at+4.08.41+PM.png" width="400" /></a></div>
Now, click the "Advanced Settings" tab and enter the information as the following:
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYF0xWVGosnB3ITe_VF0XUvCvPaaea-zCnwsUIQWDiaGOeMmP1TUie5b-QF5tXZXqYPL7m8Gza8BZOriAtmM7WY_YZG-Y9XPcKL_HVs180MbGLzm-TFstCbgC_8CgNFjWB-FGsa-zyVMrK/s1600/Screen+Shot+2018-12-15+at+4.15.51+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="627" data-original-width="629" height="399" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYF0xWVGosnB3ITe_VF0XUvCvPaaea-zCnwsUIQWDiaGOeMmP1TUie5b-QF5tXZXqYPL7m8Gza8BZOriAtmM7WY_YZG-Y9XPcKL_HVs180MbGLzm-TFstCbgC_8CgNFjWB-FGsa-zyVMrK/s400/Screen+Shot+2018-12-15+at+4.15.51+PM.png" width="400" /></a></div>
<br />
As you can see, we can enter the ip addresses separated by comma. In this way, we can achieve high availability from client side of view. Now, we test the connectivity. If the port of 9042 is exposed correctly, it should work fine. I will explain more in the next article on how to make sure we expose the native transport port correctly.
<br />
<h2>
Create Integration Flows</h2>
<h3 style="text-align: left;">
Read Flow</h3>
The read flow is very straight forward. The cassandra-db:select operation uses payload as the whole query. We just need to set payload. In this case, it is "select * from emp;"
<br />
<br />
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">flow</span><span class="s1"> </span><span class="s4">name</span><span class="s1">=</span>"select-objecsFlow"<span class="s1"> </span><span class="s4">doc:id</span><span class="s1">=</span>"8bdcc4fe-cf4e-49be-b2f9-2dfacbddaf4b"<span class="s1"> </span><span class="s2">></span></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">http:listener</span><span class="s1"> </span><span class="s4">doc:name</span><span class="s1">=</span>"Listener"<span class="s1"> </span><span class="s4">doc:id</span><span class="s1">=</span>"ab57d8ab-2623-468b-b635-a0c6efd6c829"<span class="s1"> </span><span class="s4">config-ref</span><span class="s1">=</span>"HTTP_Listener_config"<span class="s1"> </span><span class="s4">path</span><span class="s1">=</span>"/cassandra/emp"<span class="s2">/></span></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">set-payload</span><span class="s1"> </span><span class="s4">value</span><span class="s1">=</span>"select * from emp;"<span class="s1"> </span><span class="s4">doc:name</span><span class="s1">=</span>"Set Payload"<span class="s1"><span class="Apple-converted-space"> </span></span><span class="s2">/></span></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">cassandra-db:select</span><span class="s1"> </span><span class="s4">doc:name</span><span class="s1">=</span>"Select"<span class="s1"><span class="Apple-converted-space"> </span></span><span class="s4">config-ref</span><span class="s1">=</span>"CassandraDB_Config_cluster"<span class="s1"> </span><span class="s2">/></span></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">ee:transform</span><span class="s1"> </span><span class="s4">doc:name</span><span class="s1">=</span>"Transform Message"<span class="s1"><span class="Apple-converted-space"> </span></span><span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span>ee:message<span class="s1"> </span><span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span>ee:set-payload<span class="s1"> </span><span class="s2">><![CDATA[</span><span class="s1">%dw 2.0</span></span></div>
<div class="p3">
<span style="font-family: "courier new" , "courier" , monospace;">output application/json</span></div>
<div class="p3">
<span style="font-family: "courier new" , "courier" , monospace;">---</span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1">payload</span><span class="s2">]]></</span>ee:set-payload<span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"></</span>ee:message<span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"></</span>ee:transform<span class="s2">></span></span></div>
<div class="p1">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">logger</span><span class="s1"> </span><span class="s4">level</span><span class="s1">=</span>"INFO"<span class="s1"> </span><span class="s4">doc:name</span><span class="s1">=</span>"Logger"<span class="s1"> </span><span class="s4">doc:id</span><span class="s1">=</span>"e6097436-1909-4d8b-8c81-27be82c11714"<span class="s1"> </span><span class="s4">message</span><span class="s1">=</span>"#[payload]"<span class="s1"> </span><span class="s2">/></span></span></div>
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s1"><span class="Apple-tab-span"> </span></span><span class="s2"></</span>flow<span class="s2">></span></span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #2a00ff}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #3f7f7f}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco}
span.s1 {color: #000000}
span.s2 {color: #008080}
span.s3 {color: #3f7f7f}
span.s4 {color: #7f007f}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p2">
<span style="font-family: "courier new" , "courier" , monospace;"><span class="s2"></</span>mule<span class="s2">></span></span></div>
<br />
<h3 style="text-align: left;">
Insert A Row</h3>
<div>
To insert a row, we need to use insert operation as the following:<br />
<br />
<div class="p1">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">flow</span><span class="s1"> </span><span class="s4">name</span><span class="s1">=</span>"insert-objecsFlow"<span class="s1"> </span><span class="s4">doc:id</span><span class="s1">=</span>"8bdcc4fe-cf4e-49be-b2f9-2dfacbddaf4b"<span class="s1"> </span><span class="s2">></span></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">http:listener</span><span class="s1"> </span><span class="s4">doc:name</span><span class="s1">=</span>"Listener"<span class="s1"> </span><span class="s4">doc:id</span><span class="s1">=</span>"ab57d8ab-2623-468b-b635-a0c6efd6c829"<span class="s1"> </span><span class="s4">config-ref</span><span class="s1">=</span>"HTTP_Listener_config"<span class="s1"> </span><span class="s4">path</span><span class="s1">=</span>"/cassandra/emp/create"<span class="s2">/></span></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">ee:transform</span><span class="s1"> </span><span class="s4">doc:name</span><span class="s1">=</span>"Transform Message"<span class="s1"> </span><span class="s4">doc:id</span><span class="s1">=</span>"0d484493-fa23-4a56-9547-58f0bb54dbd1"<span class="s1"> </span><span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span>ee:message<span class="s1"> </span><span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span>ee:set-payload<span class="s1"> </span><span class="s2">><![CDATA[</span><span class="s1">%dw 2.0</span></span></div>
<div class="p3">
<span style="font-family: Courier New, Courier, monospace; font-size: small;">output application/java</span></div>
<div class="p3">
<span style="font-family: Courier New, Courier, monospace; font-size: small;">---</span></div>
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1">payload</span><span class="s2">]]></</span>ee:set-payload<span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"></</span>ee:message<span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"></</span>ee:transform<span class="s2">></span></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">cassandra-db:insert</span><span class="s1"> </span><span class="s4">table</span><span class="s1">=</span>"emp"<span class="s1"> </span><span class="s4">doc:name</span><span class="s1">=</span>"Insert"<span class="s1"> </span><span class="s4">doc:id</span><span class="s1">=</span>"71cb7584-981a-4475-a839-ea82ed3b9832"<span class="s1"> </span><span class="s4">config-ref</span><span class="s1">=</span>"CassandraDB_Config_vm1"<span class="s1"> </span><span class="s4">keyspaceName</span><span class="s1">=</span>"rogers_dev"<span class="s2">/></span></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">ee:transform</span><span class="s1"> </span><span class="s4">doc:name</span><span class="s1">=</span>"Transform Message"<span class="s1"><span class="Apple-converted-space"> </span></span><span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span>ee:message<span class="s1"> </span><span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span>ee:set-payload<span class="s1"> </span><span class="s2">><![CDATA[</span><span class="s1">%dw 2.0</span></span></div>
<div class="p3">
<span style="font-family: Courier New, Courier, monospace; font-size: small;">output application/json</span></div>
<div class="p3">
<span style="font-family: Courier New, Courier, monospace; font-size: small;">---</span></div>
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1">payload</span><span class="s2">]]></</span>ee:set-payload<span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"></</span>ee:message<span class="s2">></span></span></div>
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"></</span>ee:transform<span class="s2">></span></span></div>
<div class="p1">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s2"><</span><span class="s3">logger</span><span class="s1"> </span><span class="s4">level</span><span class="s1">=</span>"INFO"<span class="s1"> </span><span class="s4">doc:name</span><span class="s1">=</span>"Logger"<span class="s1"> </span><span class="s4">doc:id</span><span class="s1">=</span>"e6097436-1909-4d8b-8c81-27be82c11714"<span class="s1"> </span><span class="s4">message</span><span class="s1">=</span>"#[payload]"<span class="s1"> </span><span class="s2">/></span></span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #2a00ff}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #3f7f7f}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco}
span.s1 {color: #000000}
span.s2 {color: #008080}
span.s3 {color: #3f7f7f}
span.s4 {color: #7f007f}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p2">
<span style="font-family: Courier New, Courier, monospace; font-size: small;"><span class="s1"><span class="Apple-tab-span"> </span></span><span class="s2"></</span>flow<span class="s2">></span></span></div>
<br /></div>
<div style="text-align: left;">
Currently, we can only insert one row at a time. To insert multiple rows, we can use for loop or use batch processes for large volumes. </div>
<h3 style="text-align: left;">
About CassandraDB Connector</h3>
<div>
The detailed information can be found at the following Mulesoft Document:</div>
<div>
https://docs.mulesoft.com/connectors/cassandra/cassandra-connector</div>
<div>
<br /></div>
<h3 style="text-align: left;">
Important Cassandra Information</h3>
<div>
When you using cqlsh to connect Cassandra cluster, you should notice the following:</div>
<div>
<br /></div>
<div>
cqlsh -u cassandra -p cassandra<span style="font-family: "courier new" , "courier" , monospace;"><br />Connected to DevelopmentCluster at 127.0.0.1:9042.<br />[cqlsh 5.0.1 | Cassandra 3.11.3 | CQL spec 3.4.4 | Native protocol v4]</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The v4 is the current native protocol version, which is required to configure the connector.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #2a00ff}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco; color: #3f7f7f}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco}
span.s1 {color: #000000}
span.s2 {color: #008080}
span.s3 {color: #3f7f7f}
span.s4 {color: #7f007f}
span.Apple-tab-span {white-space:pre}
</style></div>
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com5tag:blogger.com,1999:blog-6083392550327202392.post-77935431362852478682018-08-04T10:16:00.000-07:002018-08-04T10:29:03.309-07:00Mule 4 : Dataweave 2 In Action - Use Function Modules<H1>Introduction</H1>
<p>
In Mule 4, the mel2 functionalities are replaced by the Dataweave 2 ones. For those who worked in Mule 3, you will find the mule 4 ways of using function modules are much more advanced than the old way (<a href="https://ggl-consulting.blogspot.com/2017/05/mule-application-development-advanced.html">see my post</a>). In this article, I am going to present Mule 4 ways to define and use global functions.
</p>
<H1>Define Functions In Dataweave Modules</H1>
<p>
Here is my project layout:
</p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBuFhqw4g-ezxtURVc8DJO16euJHgfEesjhbOgugRXIdhnSfp2pgCTHp-2xEd4n1MvObwCmm5TdVdmGnQ6_KkvFXJRuPCSYtPAQabVwmHtKcdfl0L4tpHMOSLpB-Mknb_noiB5km7w5Hqj/s1600/Screen+Shot+2018-08-04+at+11.59.01+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBuFhqw4g-ezxtURVc8DJO16euJHgfEesjhbOgugRXIdhnSfp2pgCTHp-2xEd4n1MvObwCmm5TdVdmGnQ6_KkvFXJRuPCSYtPAQabVwmHtKcdfl0L4tpHMOSLpB-Mknb_noiB5km7w5Hqj/s400/Screen+Shot+2018-08-04+at+11.59.01+AM.png" width="311" height="400" data-original-width="330" data-original-height="424" /></a></div>
<p>
As you can see, I place the function modules in the dir: src/main/resources/modules. And the dataweave function module is defined in CommonFunc.dwl.
</p>
<pre class="brush:js">
%dw 2.0
fun concatName(aPerson) = aPerson.firstName ++ ' ' ++ aPerson.lastName
fun stringToDateUTC(dateString) = ((dateString as String {format: "yyyy-MM-dd'T'HH:mm:ss.SSSZ"} >> "UTC"
)) as DateTime {format: "yyyy-MM-dd'T'HH:mm:ss.SSS"}
</pre>
I defined two functions: concatName, stringToDateUTC. The purposes are self-explained.
<H1>Use Function Modules</H1>
<p>
I place the normal dataweave modules in the dir of src/main/resources/dataweave. The usage of the functions is demonstrated at the following dwl file:
</p>
<pre class="brush:js">
%dw 2.0
import * from modules::CommonFunc
output application/json
---
{
"Name" : concatName(payload),
"CreatedDate" : stringToDateUTC(payload.createdDate)
}
</pre>
<p>
Here are few points:
<p>
<ul>
<li>import functions</li>
<ol>
<li>import modules::CommonFuc</li>
<li>import * from modules::CommonFunc</li>
<li>import concatName stringToDateUTC from modules::CommonFunc</li>
</ol>
<li>use functions in dwl</li>
<ol>
<li>CommonFunc::concatName(payload)</li>
<li>concatName(payload)</li>
</ol>
</ul>
<p>
For those who know python, you may find the syntax is very similar between the two languages, in terms of modules, and references.
</p>
<H1>Testing Data</H1>
Here is the complete flow:
<pre class="brush:js">
<flow name="use-functionsFlow" doc:id="ccb2b5cc-b78f-4db2-9337-f96fed2de729" >
<http:listener doc:name="Listener" doc:id="e13b3ae1-ac0c-4200-badf-0acb5039efe1" config-ref="HTTP_Listener_config" path="/mule4/func/one">
<ee:repeatable-file-store-stream />
</http:listener>
<ee:transform doc:name="Use Global Functions" doc:id="be1466e6-6c0d-44c3-b876-c191b5a0b2a3" >
<ee:message >
<ee:set-payload resource="dataweave/useFunctionModules.dwl" />
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" doc:id="9c689ad2-f9bd-47ec-86ba-8d6a7d278e88" message="#[payload]" category="mule4-datweave-in-action"/>
</flow>
</pre>
Here is testing data:
<pre class="brush:js">
{
"firstName" : "Gary",
"lastName" : "Liu",
"createdDate":"2018-07-17T16:18:03+00:00"
}
</pre>
Gary Liuhttp://www.blogger.com/profile/01594108188537664037noreply@blogger.com6