Anypoint Studio Error: The project is missing Munit lIbrary to run tests

Anypoint Studio 7.9 has a bug. Even if we following the article: https://help.mulesoft.com/s/article/The-project-is-missing-MUnit-libraries-to-run-tests-Mule-4, it still would not fix the proble. The only solution at the moment is to add the followiing dependency:

		
			org.mule.weave
			assertions
			1.0.2
			test
		

Access Multiple Github Accounts From One Computer

Step 1: Step 1: Generate SSH and Add it to ssh-agent

Note: if you have two github accounts, you need to generate two SSH Key

Step 2: Configure ~/.ssh/config

Note: In order to access the different account, you must use the hostname specified inside the ~/.ssh/config file. Here is my example:

$ cat .ssh/config
Host *
ServerAliveInterval 240

# garyliu1119 account
Host github1119.com
  HostName github.com
  AddKeysToAgent yes
  IdentityFile ~/.ssh/id_ed25519
  IdentitiesOnly yes

# garyliu19 account
Host github19.com
  HostName github.com
  AddKeysToAgent yes
  IdentityFile ~/.ssh/id_rsa
  IdentitiesOnly yes

When I clone the github repo, I use the following command:

git clone -b develop git@github19.com:garyliu19/FN-External-API.git

Note: the domain name is: github19.com, instead of github.com

Dataweave 2.0: Convert Calendar Date To Julian Date

A Julian date is commonly used in main frame. It's format is YYDDD. DDD is the date since begining of the year. In DetaWeave 2.0 the code should be as the following:

	julianNow: now() as String {format: "YYD"},
	julianAt: "07-10-2020" as Date {format: "dd-MM-yyyy"} as String {format: "YYD"}

Mulesoft Send Email With CSV Attachment

Introduction

This blog explain the tricks to convert csv data to plain text for the purpuse of send csv as attachment. The Mulesoft document can be found at here

  
  
      
          
      
      
          Hello this is an important message"]]>
      
      
        #[{
          'csv-attachment' : payload
        }]
      
  

Code Details

As shown in the follow snapshot, we need to perform two transformations:

• from csv to base64 binary
• from base64 binary to plain text>

The challenge is that we cannot direct convert to "application/csv" to "text/plain".

The details are shown below:

Step 1: Initial code JSON to CSV

%dw 2.0
output application/csv separator=","
---
[
	{
		name: "Gary Liu",
		ssn: "1234"
	},
	{
		name: "John Smith",
		ssN: "4567"
	}
]

Step 2: CSV to Base64 Binary

%dw 2.0
import * from dw::core::Binaries
output text/plain
---
toBase64(write (payload, 'application/csv'))

Step 3: Base64 Binary to CSV Plain Text

%dw 2.0
import fromBase64 from dw::core::Binaries
output text/plain
---
fromBase64(payload)

Take Aways

The line of thinking is that Mule runtime engine internally keeps that data as array of records. This java object cannot direct convert to plain text. The code is really mimic the way we store the data to file and then read the data from file to plain text.

Mule Develop Tricks: Get The Phone Number

The Requirement

The phone number from customer web site is free string. It can take the following forms:

1.  +1 (219) 555-8888
2.  +1 219-555-8888
3.  (219) 555 8888
4.  219 555 8888
5.  219 555-8888
6.  219-555-8888
7.  -1-219-555-8888

There could be other form. 

We need to use dataweave 2.0 to extract 10 digits of the phone number to: 2195558888.

Solution

%dw 2.0

output application/json

---

{

mobile: (payload.mobilePhone replace /([^0-9.]+)/ with(""))[-10 to -1]

}

Take Aways

1. dataweave built in function replace / / with ()
2. substring [-10 to -1] take last 10 digits

Prototyping Mule OAuth2 Client Application

Introduction

This article explains how to prototype Mule OAuth 2.0 client application with grant type of "Client Credentials", which is most popular grant type for Mule integration. Most modern APIs enforce OAuth2.0 security policies. OAuth2.0 has the following Grant Types (The detailed explanation can be found here):

• Authorization Code
• PKCE
• Client Credentials
• Device Code
• Refresh Token
• Legacy: Implicit Flow
• Legacy: Password Grant

This post will cover the following topics:

• Using postman to retrieve access token
• Prototyping retrieve access token using cURL
• Mule flow to retrieve access token
• Sample flow using the access token with caching scope

OAuth 2.0 With Client Credential Grant Type

In order to access the OAuth2.0 enabled APIs, we first have to retrieve the access token from the Identity Providers. Then we can access the API by passing the access token. The parameters for Client Credential can be the following:

• grant_type (required)
• scope (optional)
• client_id (required)
• client_secret (required)

In most cases, cilent_id and client_secret are encrypted as Basic Authentication. The encryption can be done using base64 or openssl as the following:

echo -n "${CLIENT_ID}:${CLIENT_SECRET}" | base64

echo -n "${CLIENT_ID}:${CLIENT_SECRET}" | openssl enc -base64

Note: the "-n" option of echo is for not printing the trailing newline character. This is very important.

Using Postman to Retrieve Access Token

The postman is the best tool to do prototyping for the OAuth 2 client. The following snapshot shows the setup of the Postman:
for body:

for Headers:

for Authorization:

cURL Solution

Once we have the postman, the solution of cUrl is very straight forward.

$ cat oauth2-client.sh

#!/bin/bash
#
CLIENT_ID=MY-CLIENT-ID-GOES-HERE-WITHOUT-QUOTE
CLIENT_SECRET=YOUR-CLIENT-SECRET-GOES-HERE-WITHOUT-QUOTE

OAUTH_HEADER=$(echo -n "${CLIENT_ID}:${CLIENT_SECRET}" | base64)

curl -d "grant_type=client_credentials&scope=https://graph.microsoft.com/.default" \
     -H "Content-Type: application/x-www-form-urlencoded" \
     -H "Authorization: Basic ${OAUTH_HEADER}" \
     -XPOST https://login.microsoftonline.com/keurig.onmicrosoft.com/oauth2/v2.0/token

Mule Application Solution - Retrieve access_token

The mule application flow for retrieving access token is the following:

The Data-Weave transformation code is the following:

%dw 2.0
output application/x-www-form-urlencoded
---
{
 grant_type: "client_credentials",
 scope: "https://graph.microsoft.com/.default"
}

The request configuration is as the following:

The HTTPS connector configuration referred in the request is the following:

The xml configuration is the following:

As you can see, we pass the client_id and client_secret as the username and password of the basic authentication. This is just base64 encoded string. Here is an example of the response from the retrieval of access token.

{
    "token_type": "Bearer",
    "expires_in": 3599,
    "ext_expires_in": 3599,
    "access_token": "eyJ0eXAi......"
}

Mule Application Solution - Use access_token

The following diagram shows the usage of the access_token. The access_token is passed to the server as header.

As you can see, we need to use cache scope. This allow us to avoid calling the server if the token is not expired. In this case, the token will expire in one hour. Thus our object store TTL should be less then 60 minutes.

Anypoint Runtime Fabric : Part 1 - Pre-Installation Network Checking

Introduction

In the next few weeks, I will write more details about Mule runtime fabric installation, configuration, and troubleshooting. This article is about to verify the network connectivity before install Anypoint Runtime Fabric (RTF). In order to install Anypoint RTF, the network firewall must open. The details about the port and host whitelisting can be find here: https://docs.mulesoft.com/runtime-fabric/1.4/install-port-reqs.

Tools for Network Connectivity Checking

The following tools should be installed and configure on the Linux system:

• nslookup
• curl
• nc
• openssl
• chrony

nslookup must configured properly. To verify nslookup, you can use the following command:

nslookup anypoint.mulesoft.com

The above command should print out something like:

Name: anypoint-prod-936665732.us-east-1.elb.amazonaws.com
Address: 52.87.103.123
Name: anypoint-prod-936665732.us-east-1.elb.amazonaws.com
Address: 3.217.25.79
...

This means it is working. If you find a problem, most likely the file /etc/resolv.conf has the incorrect content. It should look like the following:

$ cat /etc/resolv.conf
# Generated by NetworkManager
search gmcr.com
nameserver 10.64.2.13
nameserver 10.124.240.104

The next tool is the well-known cURL. run the following command:

$ curl -sk https://anypoint.mulesoft.com:443

You should see something like the following:

301 Moved Permanently
nginx

This is good. If it hung, then we have problem. I have covered the usage of nc and openssl in my previous post

NTP - Network Time Protocol

In order for Anypoint RTF to work, we must make sure NTP is working in the system. Linux use chrony to perform NTP. Run the following command:

chronyc tracking

If the NTP is working correctly, it should print out the something like the following:

Reference ID    : A3EDDA13 (toddadev.your.org)
Stratum         : 2
Ref time (UTC)  : Mon Feb 03 00:18:55 2020
System time     : 0.000235624 seconds slow of NTP time
Last offset     : -0.000094732 seconds
RMS offset      : 0.000081353 seconds
Frequency       : 16.987 ppm slow
Residual freq   : -0.003 ppm
Skew            : 0.062 ppm
Root delay      : 0.025243049 seconds
Root dispersion : 0.000095447 seconds
Update interval : 514.2 seconds
Leap status     : Normal

Note the Leas status must be "Normal", otherwise, Anypoint RTF will be not working. The real for the Leap status is not normal is the network firewal is not open to the RedHat's time server. In this case, we need to use company's internal time server. To do this, modify the file: /etc/chrony.conf

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
#server 0.rhel.pool.ntp.org iburst
#server 1.rhel.pool.ntp.org iburst
#server 2.rhel.pool.ntp.org iburst
#server 3.rhel.pool.ntp.org iburst
server server 10.70.0.200
...

The line: server server 10.70.0.200 is company's private time server. Make sure comment out all the RedHat's entries. After the modification of the file of /etc/chrony.conf, make sure run the following command:

sudo systemctl restart chronyd
sudo systemctl enable chronyd

You also may verify the sources of the Ntp by the following command:

chronyc sources

This will restart the chrony daemon.

Network Connectivity Test Scripts

Mulesoft provided a scripts totest the network connectivity at here If you have gone through the test in the previous sections, this script should run with the following results.

Once the above procedures are all done, we are read to perform the installation for Anypoint Runtime Fabric. I will cover that in the following articles.