Saturday, April 30, 2016

MuleSoft Dataweave Tutorial Part IV: Using mapObject, pluck, function

Introduction

DataWeave comes with map, mapOjbect, pluck, and other build in functions. The map function is mostly used in the message transformation. There are a lot of other functions provided by the Dataweave component, such as split, trim, uppercase, lower case, etc.

This tutorial is an example on how to use map, mapObject, plunk, and joinBy functions to achieve a fairly complicated message transformation.

Use Cases

Input Data

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
<soapenv:envelope xmlns:ns="http://taxi.uhc.com/1.0" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
   <soapenv:header>
   <soapenv:body>
      <ns:preauth>
         <ns:ssi>venkat</ns:ssi>
         <ns:custid>api-user-jwilson</ns:custid>
         <ns:password>4JuGSrVb1E7k8a1datKuGV0fqma58c2B</ns:password>
         <ns:billingid>VC84632147254611111111</ns:billingid>
         <ns:amount>5000</ns:amount>
         <ns:personal-info>
            <ns:name>John Smith</ns:name>
            <ns:address1>12 Main St</ns:address1>
            <ns:address2>Raheja</ns:address2>
            <ns:zip>30301</ns:zip>
            <ns:city>Denver</ns:city>
            <ns:state>CO</ns:state>
            <ns:country>USA</ns:country>
            <ns:phone>6099026676</ns:phone>
            <ns:email>gg@gmail.com</ns:email>
            <ns:ip>10.121.123.112</ns:ip>
         </ns:personal-info>
         <ns:ship-info>
            <ns:shipto_name>John Smith</ns:shipto_name>
            <ns:shipto_address1>12 Main St</ns:shipto_address1>
            <ns:shipto_address2>SR Nagar</ns:shipto_address2>
            <ns:shipto_zip>30301</ns:shipto_zip>
            <ns:shipto_city>Denver</ns:shipto_city>
            <ns:shipto_state>CO</ns:shipto_state>
            <ns:shipto_country>USA</ns:shipto_country>
         </ns:ship-info>
         <ns:optional-info>
            <ns:cvv>111</ns:cvv>
            <ns:avs>false</ns:avs>
            <ns:ticket>V12345</ns:ticket>
            <ns:type>MOTO</ns:type>
            <ns:iias_lookup>true</ns:iias_lookup>
            <ns:partialauth>true</ns:partialauth>
            <ns:marketindicator>H</ns:marketindicator>
            <ns:additionalamount1>3000</ns:additionalamount1>
            <ns:additionalamount2>1000</ns:additionalamount2>
            <ns:demo>false</ns:demo>
            <ns:level-2-3-fields>
               <ns:level-2-field>
                  <ns:numitems>1</ns:numitems>
                  <ns:purchaselevel>1</ns:purchaselevel>
                  <ns:purchaseordernum>T1234522</ns:purchaseordernum>
                  <ns:tax>100</ns:tax>
                  <ns:taxidentifier>localsales</ns:taxidentifier>
                  <ns:vatnum>A10004</ns:vatnum>
                  <ns:commoditycode>AA01</ns:commoditycode>
                  <ns:discount>100</ns:discount>
                  <ns:duty>400</ns:duty>
                  <ns:shippinghandling>300</ns:shippinghandling>
               </ns:level-2-field>
               <ns:level-3-field>
                  <ns:productcode>02</ns:productcode>
                  <ns:quantity>1</ns:quantity>
                  <ns:price>120</ns:price>
                  <ns:tax>120</ns:tax>
                  <ns:discount>30</ns:discount>
                  <ns:productdescription>t-shirt</ns:productdescription>
                  <ns:unitofmeasure>each</ns:unitofmeasure>
                  <ns:taxamount>300</ns:taxamount>
                  <ns:taxidentifier>0001</ns:taxidentifier>
               </ns:level-3-field>
                <ns:level-3-field>
                  <ns:productcode>001</ns:productcode>
                  <ns:quantity>1</ns:quantity>
                  <ns:price>120</ns:price>
                  <ns:tax>120</ns:tax>
                  <ns:discount>30</ns:discount>
                  <ns:productdescription>t-shirt</ns:productdescription>
                  <ns:unitofmeasure>each</ns:unitofmeasure>
                  <ns:taxamount>300</ns:taxamount>
                  <ns:taxidentifier>0001</ns:taxidentifier>
               </ns:level-3-field>                  
            </ns:level-2-3-fields>
            <ns:custom-fields>
               <ns:custom-field sequence="7" value="venkat">
            </ns:custom-field></ns:custom-fields>
         </ns:optional-info>
      </ns:preauth>
   </soapenv:body>
</soapenv:header></soapenv:envelope>

Output

?
1
2
3
{
  "result": "(productcode=02;quantity=1;price=120;tax=120;discount=30;productdescription=t-shirt;unitofmeasure=each;taxamount=300;taxidentifier=0001)(productcode=001;quantity=1;price=120;tax=120;discount=30;productdescription=t-shirt;unitofmeasure=each;taxamount=300;taxidentifier=0001)"
}

We need to create key-value pairs for level-e-field. Each key-value pair is separated by semi-colon, and each object is demarked by round braket. (key1=value1;key2=value2;...;keyN=valueN)().

The Solution

?
1
2
3
4
5
6
7
8
9
10
11
%dw 1.0
%output application/json
%function enrich(val) '(' ++ val ++ ')'
---
result: payload.Envelope.Body.preauth.optional-info.level-2-3-fields.*level-3-field
map ((value,index) -> (
      value mapObject ((oValue, oKey) -> {
          '$oKey': oKey ++ '=' ++ oValue       
      }) pluck $ joinBy ';'
))
map ((val, key) -> enrich(val)) joinBy ''

The first code line 5 is easy to understand. We need to access the xml object at level-3-file. The only trick we need to know is that we need to add a star *. As we have multiple level-3-field object, we need to use map. map function is just a look. It takes two parameters. The first is value of the object, and the second is the index. If you have an array or list, you can think the first is the value of the array and the second is the index, which starts with 0 as java array.

In our code, we use twice joinBy function. The first joins the array with ';' semi-colon. The second join the level-3-field objects together.

pluck is another function which create an array of keys or values. The value is $, and the keys are $$.

at 2 comments:

Sunday, April 17, 2016

MuleSoft Dataweave Tutorial: Part III - Multiple Level Iteration of Arrays

Introduction

I must admit the MuelSoft's Dataweave component is a critical and very powerful component of the Mule application development, because the data transformation is one of the critical part of enterprise integration. In this short article, I am going to explain the usage of multiple level of mapping.

The build-in function map is used to iterate a list of objects. This is mostly used function with Dataweave component. Given that, let explain my use cases.

Use Cases

Here is the input:
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
[
  {
    "TX": [
      {
        "name": "John Smith"
      },
      {
        "name": "Robert Goll"
      }
    ]
  },
  {
    "OK": [
      {
        "name": "Framk Chambers"
      },
      {
        "name": "Billie Chambers"
      }
    ]
  }
]
The output should be [Note: this is just a simple example to illustrate the concept]:
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
[
  {
    "name": "John Smith"
  },
  {
    "name": "Robert Goll"
  },
  {
    "name": "Framk Chambers"
  },
  {
    "name": "Billie Chambers"
  }
]

Logical Thinking

If we looking the input, we can see that this is List of List. There are 2 levels. Somehow, if we can create an array of array like this:

?
1
2
3
4
5
6
[
   [
   ],
   [
   ]
]

then, we can use the build-in function: flatten, to get the result we want. To achieve this, we need to replace the middle keys like "TX", "OK" to a same name, tempName. This can be achieved by the following code:

?
1
2
3
payload map {
     ($ map tempName: $)
 }

The result of the above code produces the following result:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
[
  {
    "tempName": [
      {
        "name": "John Smith"
      },
      {
        "name": "Robert Goll"
      }
    ]
  },
  {
    "tempName": [
      {
        "name": "Framk Chambers"
      },
      {
        "name": "Billie Chambers"
      }
    ]
  }
]

Here the outer loop (map) is easy to understand, but inner loop ($ map tempName: $) is a bit difficult. The dataweave component ignore the key "TX" and loop the inner array.

if we understand the inner loop, the problem should be easily understood now.

The Solution One: Using Temporary Variable

?
1
2
3
4
5
6
7
%dw 1.0
%output application/json
---
 
 flatten (payload map {
     ($ map tempName: $)
 }).tempName

The Solution Two: Using reduce function

Another more elegant solution is to use reduce function.

?
1
2
3
4
5
6
%dw 1.0
%output application/json
---
 
payload reduce ((value, acc = {}) -> acc ++ value[0])

Key Learning

In this exercise, we learn the key concept of map for inner array. In order to flatten the inner array of objects, we have to replace the keys of the inner array with a temporary name, then use the temporary name to get the connects, and further, flatten the array.

Apparently, the build-in function "flatten" is a recursive function. Now, a question comes: can we recursively map the multiple levels of array?

at 5 comments:

Saturday, April 16, 2016

MuleSoft Dataweave Tutorial: Part II - Functions

User Defined Functions

Here is a example from Mule document:
?
1
2
3
4
5
6
7
%dw 1.0
%output application/json
%function toUser(user) { firstName: user.givenName, lastName: user.sn }
---
{
  "user" : toUser({ givenName : "Annie", sn : "Point" })
}
Output from the result:
?
1
2
3
4
5
6
{
  "user": {
    "firstName": "Annie",
    "lastName": "Point"
  }
}

Mule document did not provide much explanation. What does this function do? 

 %function toUser(user) { firstName: user.givenName, lastName: user.sn }
First, we can see it declares the user object, which has {givenName, sn}. When we use the function in the body, we parse an object of user in the form of: 
 toUser({ givenName : "Annie", sn : "Point" })
Where 
 { givenName : "Annie", sn : "Point" }
is an user object. This tell us how you can initialize an object and pass to a function.

Using Build-in Function: GroupBy

If we want to group object by attribute, we can groupBy and pluck together. Here is the simple Dataweave code:

?
1
2
3
4
%dw 1.0
%output application/json
---
payload groupBy $.state

Here is the input data:
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[
 {
  "name": "John Smith",
  "street": "1234 JS Ln",
  "city": "Allen",
  "state": "TX",
  "zip": "75123"
 },
 {
  "name": "Framk Chambers",
  "street": "1234  Colleage Rd",
  "city": "Stillwater",
  "state": "OK",
  "zip": "73001"
 },
 {
  "name": "Robert Goll",
  "street": "1234 Bush Dr",
  "city": "Plano",
  "state": "TX",
  "zip": "75127"
 },
  {
  "name": "Billie Chambers",
  "street": "1234 College Rd",
  "city": "Stillwater",
  "state": "OK",
  "zip": "73001"
 }
]
And here is the output:
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
[
  {
    "name": "John Smith"
  },
  {
    "name": "Robert Goll"
  },
  {
    "name": "Framk Chambers"
  },
  {
    "name": "Billie Chambers"
  }
]
at 10 comments:

Monday, April 11, 2016

MuleSoft Dataweave Tutorial: Part I - JSON To XML Example

Use Case One

Input

?
1
2
3
4
5
6
7
8
9
10
11
{
    "medication": {
        "name": "Natrecor"
    },
    "genericequivalents": [
        {
            "name": "Narecor",
            "label": "Erlotinib2"
        }
    ]
}

DW Code

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
%dw 1.0
%output application/xml
---
response: {
 medication : payload.medication,
 genericequivalents: {
  (payload.genericequivalents map {
    names : {
     label: $.label,
     name: $.name
    }
  })
 }
}

Use Case Two

Input

?
1
2
3
4
5
6
7
8
9
10
11
12
13
{
    "medication": "praxel",
    "genericequivalents": {
        "name": [
            "Praxe",
            "Praxedes",
            "Paxel",
            "Pragel",
            "Fraxel",
            "Parexel"
        ]
    }
}

DW Code

?
1
2
3
4
5
6
7
8
9
10
11
12
13
%dw 1.0
%output application/xml
---
{
 response: {
  medication: payload.medication,
  genericequivalents: {
   (payload.genericequivalents.name map ((name , indexOfName) -> {
    name: name
   }))
  }
 }
}

Output

?
1
2
3
4
5
6
7
8
9
10
11
<response>
   <medication>praxel</medication>
   <genericequivalents>
      <name>Praxe</name>
      <name>Praxedes</name>
      <name>Paxel</name>
      <name>Pragel</name>
      <name>Fraxel</name>
      <name>Parexel</name>
   </genericequivalents>
</response>
at 2 comments:
Subscribe to: Posts (Atom)

Anypoint Studio Error: The project is missing Munit lIbrary to run tests

Anypoint Studio 7.9 has a bug. Even if we following the article: https://help.mulesoft.com/s/article/The-project-is-missing-MUnit-libraries-...