Saturday, April 30, 2016

MuleSoft Dataweave Tutorial Part IV: Using mapObject, pluck, function

Introduction

DataWeave comes with map, mapOjbect, pluck, and other build in functions. The map function is mostly used in the message transformation. There are a lot of other functions provided by the Dataweave component, such as split, trim, uppercase, lower case, etc.

This tutorial is an example on how to use map, mapObject, plunk, and joinBy functions to achieve a fairly complicated message transformation.

Use Cases

Input Data


   
   
      
         venkat
         api-user-jwilson
         4JuGSrVb1E7k8a1datKuGV0fqma58c2B
         VC84632147254611111111
         5000
         
            John Smith
            12 Main St
            Raheja
            30301
            Denver
            CO
            USA
            6099026676
            gg@gmail.com
            10.121.123.112
         
         
            John Smith
            12 Main St
            SR Nagar
            30301
            Denver
            CO
            USA
         
         
            111
            false
            V12345
            MOTO
            true
            true
            H
            3000
            1000
            false
            
               
                  1
                  1
                  T1234522
                  100
                  localsales
                  A10004
                  AA01
                  100
                  400
                  300
               
               
                  02
                  1
                  120
                  120
                  30
                  t-shirt
                  each
                  300
                  0001
               
                
                  001
                  1
                  120
                  120
                  30
                  t-shirt
                  each
                  300
                  0001

Output

{
  "result": "(productcode=02;quantity=1;price=120;tax=120;discount=30;productdescription=t-shirt;unitofmeasure=each;taxamount=300;taxidentifier=0001)(productcode=001;quantity=1;price=120;tax=120;discount=30;productdescription=t-shirt;unitofmeasure=each;taxamount=300;taxidentifier=0001)"
}

We need to create key-value pairs for level-e-field. Each key-value pair is separated by semi-colon, and each object is demarked by round braket. (key1=value1;key2=value2;...;keyN=valueN)().

The Solution

%dw 1.0
%output application/json
%function enrich(val) '(' ++ val ++ ')' 
---
result: payload.Envelope.Body.preauth.optional-info.level-2-3-fields.*level-3-field
map ((value,index) -> (
      value mapObject ((oValue, oKey) -> { 
          '$oKey': oKey ++ '=' ++ oValue        
      }) pluck $ joinBy ';'
)) 
map ((val, key) -> enrich(val)) joinBy ''

The first code line 5 is easy to understand. We need to access the xml object at level-3-file. The only trick we need to know is that we need to add a star *. As we have multiple level-3-field object, we need to use map. map function is just a look. It takes two parameters. The first is value of the object, and the second is the index. If you have an array or list, you can think the first is the value of the array and the second is the index, which starts with 0 as java array.

In our code, we use twice joinBy function. The first joins the array with ';' semi-colon. The second join the level-3-field objects together.

pluck is another function which create an array of keys or values. The value is $, and the keys are $$.

at 2 comments:

Sunday, April 17, 2016

MuleSoft Dataweave Tutorial: Part III - Multiple Level Iteration of Arrays

Introduction

I must admit the MuelSoft's Dataweave component is a critical and very powerful component of the Mule application development, because the data transformation is one of the critical part of enterprise integration. In this short article, I am going to explain the usage of multiple level of mapping.

The build-in function map is used to iterate a list of objects. This is mostly used function with Dataweave component. Given that, let explain my use cases.

Use Cases

Here is the input: 
[
  {
    "TX": [
      {
        "name": "John Smith"
      },
      {
        "name": "Robert Goll"
      }
    ]
  },
  {
    "OK": [
      {
        "name": "Framk Chambers"
      },
      {
        "name": "Billie Chambers"
      }
    ]
  }
]
The output should be [Note: this is just a simple example to illustrate the concept]: 
[
  {
    "name": "John Smith"
  },
  {
    "name": "Robert Goll"
  },
  {
    "name": "Framk Chambers"
  },
  {
    "name": "Billie Chambers"
  }
]

Logical Thinking

If we looking the input, we can see that this is List of List. There are 2 levels. Somehow, if we can create an array of array like this: 

[
   [
   ],
   [
   ]
]

then, we can use the build-in function: flatten, to get the result we want. To achieve this, we need to replace the middle keys like "TX", "OK" to a same name, tempName. This can be achieved by the following code: 

payload map {
     ($ map tempName: $)
 }

The result of the above code produces the following result: 

[
  {
    "tempName": [
      {
        "name": "John Smith"
      },
      {
        "name": "Robert Goll"
      }
    ]
  },
  {
    "tempName": [
      {
        "name": "Framk Chambers"
      },
      {
        "name": "Billie Chambers"
      }
    ]
  }
]

Here the outer loop (map) is easy to understand, but inner loop ($ map tempName: $) is a bit difficult. The dataweave component ignore the key "TX" and loop the inner array.

if we understand the inner loop, the problem should be easily understood now.

The Solution One: Using Temporary Variable

%dw 1.0
%output application/json
---

 flatten (payload map {
     ($ map tempName: $)
 }).tempName

The Solution Two: Using reduce function

Another more elegant solution is to use reduce function. 

%dw 1.0
%output application/json
---

payload reduce ((value, acc = {}) -> acc ++ value[0])

Key Learning

In this exercise, we learn the key concept of map for inner array. In order to flatten the inner array of objects, we have to replace the keys of the inner array with a temporary name, then use the temporary name to get the connects, and further, flatten the array.

Apparently, the build-in function "flatten" is a recursive function. Now, a question comes: can we recursively map the multiple levels of array?

at 5 comments:

Saturday, April 16, 2016

MuleSoft Dataweave Tutorial: Part II - Functions

User Defined Functions

Here is a example from Mule document: 
%dw 1.0
%output application/json
%function toUser(user) { firstName: user.givenName, lastName: user.sn }
---
{
  "user" : toUser({ givenName : "Annie", sn : "Point" })
}
Output from the result: 
{
  "user": {
    "firstName": "Annie",
    "lastName": "Point"
  }
}

Mule document did not provide much explanation. What does this function do? 

 %function toUser(user) { firstName: user.givenName, lastName: user.sn }
First, we can see it declares the user object, which has {givenName, sn}. When we use the function in the body, we parse an object of user in the form of: 
 toUser({ givenName : "Annie", sn : "Point" })
Where 
 { givenName : "Annie", sn : "Point" }
is an user object. This tell us how you can initialize an object and pass to a function.

Using Build-in Function: GroupBy

If we want to group object by attribute, we can groupBy and pluck together. Here is the simple Dataweave code: 

%dw 1.0
%output application/json
---
payload groupBy $.state

Here is the input data: 
[
 {
  "name": "John Smith",
  "street": "1234 JS Ln",
  "city": "Allen",
  "state": "TX",
  "zip": "75123" 
 },
 {
  "name": "Framk Chambers",
  "street": "1234  Colleage Rd",
  "city": "Stillwater",
  "state": "OK",
  "zip": "73001" 
 },
 {
  "name": "Robert Goll",
  "street": "1234 Bush Dr",
  "city": "Plano",
  "state": "TX",
  "zip": "75127" 
 },
  {
  "name": "Billie Chambers",
  "street": "1234 College Rd",
  "city": "Stillwater",
  "state": "OK",
  "zip": "73001" 
 }
]
And here is the output: 
[
  {
    "name": "John Smith"
  },
  {
    "name": "Robert Goll"
  },
  {
    "name": "Framk Chambers"
  },
  {
    "name": "Billie Chambers"
  }
]
at 10 comments:

Monday, April 11, 2016

MuleSoft Dataweave Tutorial: Part I - JSON To XML Example

Use Case One

Input

{
    "medication": {
        "name": "Natrecor"
    },
    "genericequivalents": [
        {
            "name": "Narecor",
            "label": "Erlotinib2"
        }
    ]
}

DW Code

%dw 1.0
%output application/xml
---
response: {
 medication : payload.medication, 
 genericequivalents: { 
  (payload.genericequivalents map {
    names : {
     label: $.label,
     name: $.name 
    }
  }) 
 } 
}

Use Case Two

Input

{
    "medication": "praxel",
    "genericequivalents": {
        "name": [
            "Praxe",
            "Praxedes",
            "Paxel",
            "Pragel",
            "Fraxel",
            "Parexel"
        ]
    }
}

DW Code


%dw 1.0
%output application/xml
---
{
 response: {
  medication: payload.medication,
  genericequivalents: {
   (payload.genericequivalents.name map ((name , indexOfName) -> {
    name: name
   }))
  }
 }
}

Output


   praxel
   
      Praxe
      Praxedes
      Paxel
      Pragel
      Fraxel
      Parexel
at 2 comments:

Wednesday, March 30, 2016

Build Dynamic Query

Use Cases

The following is a code example on how to use Lambda express to build dynamic query the where clause. Here is the use cases:

  • client provide a set of query parameters in JSON format or XML format
  • Mule flow will transform the request payload to java LinkedHashMap
  • We need to write custom transformer to build the database query's where clause
  • In the custom transformer, we need to filter the entries with null or empty string value
  • The transform should return the string in the format of: 
        ID='1243124' and Name='John Smith'

Code Example

package com.ggl.util;

import java.util.HashMap;
import java.util.LinkedHashMap;

public class DynamicQueryBuilder {
 public static void main(String[] argv) {
  System.out.println("--> Build HashMap Now! <--");
  HashMap source = new LinkedHashMap<>();
  source.put("ID", "123456789");
  source.put("NAME", "Gary Liu");
  source.put("Address", null);
  source.put("Sex", "");
  
  StringBuilder builder = new StringBuilder();
  
  source.entrySet()
   .parallelStream()
   .filter(e->e.getValue() != null)
   .filter(e->e.getValue().length() != 0)
   .forEachOrdered(e -> append(builder, e.getKey(), e.getValue()));
  
  String tmpResult = builder.toString();
  
  System.out.println(tmpResult);
  System.out.println (tmpResult.replaceFirst(" and " + "$", ""));
 }
 
 private static void append(StringBuilder builder, String key, String value) {
  builder.append(key + "='" + value + "' and " );
 }

}
at 1 comment:

MuleSoft Application Development With JMS Queue Explained: Part Four

Introduction

In my previous 3 blogs about integration using JMS Queue, have covered:

  1. JMS connector setup, inbound and outbound jms queues, and message selector
  2. JMS request and response patterns
  3. Request and response pattern using request-reply scope

In this article, I will cover how to set the outbound JMS message properties.

Set Single Property

To set a single property for the JMS message, we can use the "property" transformer as should in the following diagram:

Set Multiple Properties

To set multiple properties, we need to use message property transfromer:
Graphically, you can do the following:
And add the properties in the global property transform:

Complete Flow
at 3 comments:

Thursday, March 24, 2016

MuleSoft Application Development: Using Idempotent Message Filter

Introduction

Here is the question:
How can we prevent a GUI user from submitting the same order multiple times with a short period of time?

Idempotent Message Filter is the answer. This filter will reject the submission of orders with same ID. This ID can be defined in MEL. This article explains how to setup idempotent message filter in a mule application flow.



Setup A Mule Flow

The diagram below shows the flow setup. Note that, we need to convert payload from stream to string in order to have the filter works properly.

Idempotent Message Filter Configuration

The follow two diagrams show how to configure the objectstore required for Idempotent Message Filter. The reason we persist the object store is that if for some reason, the mule runtime crushes, once the runtime starts again, the filter will still work.

The TTL should be less than 1 minutes. If it is too long, we may block the legitimate update cases. If it is too short, we may have the collisions.

Test Idempotent Message Filter

Here is the simple json payload for the testing: 
{ 
    "name"   : "John Smith"
}

Use Postman send the first request, we should get message back with "YAY!". If you send again within few seconds, you will get the message of: "You cannot enter the same request with 60 seconds". You should also see the exception as the following: 


ERROR 2016-03-24 19:04:51,112 [[poc-basic].HTTP_Listener_Configuration.worker.01] org.mule.exception.CatchMessagingExceptionStrategy: 
********************************************************************************
Message               : Message has been rejected by filter. Message payload is of type: String
Type                  : org.mule.api.routing.filter.FilterUnacceptedException
Code                  : MULE_ERROR--2
JavaDoc               : http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/routing/filter/FilterUnacceptedException.html
Payload               : { 
    "name"   : "John Smith"
}
********************************************************************************
Exception stack is:
1. Message has been rejected by filter. Message payload is of type: String (org.mule.api.routing.filter.FilterUnacceptedException)
  org.mule.processor.AbstractFilteringMessageProcessor:69 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/routing/filter/FilterUnacceptedException.html)
********************************************************************************
Root Exception stack trace:
org.mule.api.routing.filter.FilterUnacceptedException: Message has been rejected by filter. Message payload is of type: String
 at org.mule.processor.AbstractFilteringMessageProcessor.filterUnacceptedException(AbstractFilteringMessageProcessor.java:69)

This means the filter is working!


Conclusion

  • The purpose of Idempotent Message Filter is to prevent the same message passing through the flow. It is very import if you don't want the POST, UPDATA operation to submit multiple times within a short period of time
  • The message id cannot be stream. It must be serializable Java objects, such as String.
  • The TTL should be less than 1 minute

Reference

MuleSoft Document: https://docs.mulesoft.com/mule-user-guide/v/3.7/idempotent-filter
at 2 comments:
Subscribe to: Posts (Atom)

Anypoint Studio Error: The project is missing Munit lIbrary to run tests

Anypoint Studio 7.9 has a bug. Even if we following the article: https://help.mulesoft.com/s/article/The-project-is-missing-MUnit-libraries-...