Mule Dev Tricks: DataWeave Using mapObject - Part I

Introduction

Here is a problem I am facing. An incoming JSON payload misses a piece of information, the transaction type. This piece of information can be deducted from the existing information. Actually, the really application is more more complicated. In this Mule Dev Trick, I am going to demonstrate how to add new information to the payload.

This small piece of code involves several key concept of Dataweave:

• map
• mapObject
• Condition statement
• Lamda sytax

Input Payload

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

[   {     "worker_id": "AMAGWK00002673",     "position_id": "35119973",     "requisition_id": "R-32353",     "applicant_id": "199505",     "staff_id": "199504"   },   {     "worker_id": "AMAGWK00002674",     "position_id": "35119943",     "requisition_id": "R-32353",     "applicant_id": "199505",     "staff_id": "199506"   },   {     "worker_id": "AMAGWK00002675",     "position_id": "35119975",     "requisition_id": "R-32354",     "applicant_id": "199507",     "staff_id": ""   } ]

The Desired Output

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

[   {     "worker_id": "AMAGWK00002673",     "position_id": "35119973",     "requisition_id": "R-32353",     "applicant_id": "199505",     "staff_id": "199504",     "type": "re-hire"   },   {     "worker_id": "AMAGWK00002674",     "position_id": "35119943",     "requisition_id": "R-32353",     "applicant_id": "199505",     "staff_id": "199506",     "type": "re-hire"   },   {     "worker_id": "AMAGWK00002675",     "position_id": "35119975",     "requisition_id": "R-32354",     "applicant_id": "199507",     "staff_id": "",     "type": "new-hire"   } ]

As you can see that the "type" attribute is introduced based on whether the staff_id is empty or not. If the staff_id attribute is not empty, I set the value of this attribute to "re-hire", otherwise, it will be "new-hire".

Solution

?

1 2 3 4 5 6 7 8 9

%dw 1.0 %output application/json --- payload map ((worker) ->{  (worker mapObject (value, key) -> {   "$key" : value  }),  type : "new-hire" when worker.staff_id == null or worker.staff_id == '' otherwise 're-hire' })

Note: the $key is used here. There is another syntax is (key). Thus, the above code can be as the following:

?

1 2 3 4 5 6 7 8 9

%dw 1.0 %output application/json --- payload map ((worker) ->{  (worker mapObject (value, key) -> {   (key) : value  }),  type : "new-hire" when worker.staff_id == null or worker.staff_id == '' otherwise 're-hire' })

Key Learning Points

The above script seems pretty simple. It has used many key concepts of the DataWeave component. The key of the map function is to iterate a List or Array and produce a new array. And the main usage of mapObject is to iterate the attributes of an object. Both functions use Lamda expression which takes two value, $$, $. By default, $ represent the value of the current object the iterator is point to, and $$ is the index or key of the object.

Another worth noting point is that map can take single argument as well.

Mule Dev Tricks: DataWeave Using Reduction

Introduction

In this post, I am going to demonstrate how to convert an array of JSON payload to a single JSON payload.

Input

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

[   {     "worker_id": "AMAGWK00002673",     "position_id": "35119973",     "requisition_id": "R-32353",     "applicant_id": "199505",     "staff_id": "199504"   },   {     "worker_id": "AMAGWK00002674",     "position_id": "35119943",     "requisition_id": "R-32353",     "applicant_id": "199505",     "staff_id": "199506"   },   {     "worker_id": "AMAGWK00002675",     "position_id": "35119975",     "requisition_id": "R-32354",     "applicant_id": "199507",     "staff_id": "199508"   } ]

Desired Output

?

1 2 3 4 5

{   "AMAGWK00002673": "199504",   "AMAGWK00002674": "199506",   "AMAGWK00002675": "199508" }

Solution

The first step is to use dataweave to extrace the worker_id. This is very straight forward. The only trick about this piece of script is to use value as key in the transformed JSON result. This is achieved by using ($.key) : $.otherKey. here key = worker_id and the otherKey is staff_id.

?

1 2 3

payload map ((value, index) -> {      (value.worker_id): value.staff_id })

The above dataweave script will produce the following result:

?

1 2 3 4 5 6 7 8 9 10 11

[   {     "AMAGWK00002673": "199504"   },   {     "AMAGWK00002674": "199506"   },   {     "AMAGWK00002675": "199508"   } ]

In order to produce the final result, we need to use reduction reduce($$ + $).

?

1 2 3

payload map ((value, index) -> {      (value.worker_id): value.staff_id }) reduce($$ ++ $)

Note: You have to use reduce ($$ ++$). The round bracket is important! The MuleSoft documentation missed that which causes many confusions.

Mule Dev Tricks: Using xpath3

Introduction

Even though today most web services using RESTful architecture, there are still a lot of legacy systems using SOAP. Handling xml file is challenging, if not difficult. Fortunately, Mule Application platform provide several utilities to alleviate the pain. The most powerful component is Dataweave, which is very handle to build SOAP request and handle responses. However, some cases, xpath3 is an easier solution. Sometimes it may be the best solution, for instance, if we want to split the xml payload based on certain node.

In this short article, I will introduce two use cases:

1. Extract values from xml payload
2. Split the payload

Use Case One: Extract Value Of An Element

Input Example

Here is the input:

?

1 2 3 4 5 6 7 8 9 10

<!--xml version="1.0" encoding="UTF-8"?--> <wd:create_position_response xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wd="urn:com.workday/bsvc" wd:version="v27.0">     <wd:event_reference wd:descriptor="Create Position: SOA Architect">         <wd:id wd:type="WID">31bf3ce3d6bb1025d71346c6ec276bd6</wd:id>     </wd:event_reference>     <wd:position_reference wd:descriptor="SOA Architect (Unfilled)">         <wd:id wd:type="WID">31bf3ce3d6bb1025d71348e2f1c26bdc</wd:id>         <wd:id wd:type="Position_ID">35119960</wd:id>     </wd:position_reference> </wd:create_position_response>

I want to extract the value of Position_ID in the line of:

?

1	<wd:id wd:type="Position_ID">35119960</wd:id>

Solution

There are two solutions in this case. The first is:

?

1	#[xpath3('/*:Create_Position_Response/*:Position_Reference/*:ID[2]')]

The above MEL will assign position_id to 35119960. Note: XML node count starting from 1 not from 0 as in Java array.

The other better solution is the following:

?

1	#[xpath3('/*:Create_Position_Response/*:Position_Reference/*:ID[@*:type=\'Position_ID\']')]

Here is the trick is \/*:ID[@*:type=\'Position_ID\'].

1. Refer element attribute using [@*:type]
2. Escape the single quote with backslash \'

Use Case Two: Use xpath3 To Split Payload

Example Input

In this use case, I would like to extract the node of StockMovementData, so that I can process the data.

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

<!--xml version="1.0"?--> <stockmovementdatarequest xmlns:p1="urn:ams.com.au:dynamo:3pl:am:SAP_AM_I_005:StockMovement" xmlns:a="http://www.edi.com.au/EnterpriseService">     <header>         <from>Warehouse</from>         <to>Location</to>         <unique_id>idm1468201212</unique_id>         <datetimestamp>2016-04-13T11:55:30.263+10:00</datetimestamp>     </header>     <stockmovementdata>         <serialised_material>YES</serialised_material>         <datetime>2016-04-13T11:55:30.263+10:00</datetime>         <from_location>30-80</from_location>         <to_location>client</to_location>         <material>7CAGL3G00</material>         <serial>700030011</serial>         <quantity>100</quantity>     </stockmovementdata>     <stockmovementdata>         <serialised_material>YES</serialised_material>         <datetime>2016-04-13T11:55:30.263+10:00</datetime>         <from_location>30-80</from_location>         <to_location>client</to_location>         <material>7CAGL3G00</material>         <serial>700029911</serial>         <quantity>100</quantity>     </stockmovementdata> </stockmovementdatarequest>

Here is the mule flow:

Solution

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14

<flow name="use_xpath3_splitter-Flow">         <http:listener config-ref="HTTP_Listener_Configuration" path="/advanced/splitter" doc:name="HTTP">         <splitter expression="#[xpath3('//StockMovementDataRequest/StockMovementData',payload,'NODESET')]" doc:name="Splitter">         <mulexml:dom-to-xml-transformer doc:name="DOM to XML">         <logger level="INFO" doc:name="Logger">         <collection-aggregator failontimeout="true" doc:name="Collection Aggregator">         <dw:transform-message doc:name="Transform Message">             <dw:set-payload><!--[CDATA[%dw 1.0 %output application/json --- payload]]--></dw:set-payload>         </dw:transform-message>         <logger message="#[payload]" level="INFO" doc:name="Logger">     </logger></collection-aggregator></logger></mulexml:dom-to-xml-transformer></splitter></http:listener></flow>

The above flow, split the xml payload using the following xpath3 expression:

?

1 2	<splitter expression="#[xpath3('//StockMovementDataRequest/StockMovementData',payload,'NODESET')]" doc:name="Splitter"> </splitter>

Complete Mule Flow Configuration

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

<!--xml version="1.0" encoding="UTF-8"?-->   <mule xmlns:dw="http://www.mulesoft.org/schema/mule/ee/dw" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns:mulexml="http://www.mulesoft.org/schema/mule/xml" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemalocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd http://www.mulesoft.org/schema/mule/xml http://www.mulesoft.org/schema/mule/xml/current/mule-xml.xsd http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd http://www.mulesoft.org/schema/mule/ee/dw http://www.mulesoft.org/schema/mule/ee/dw/current/dw.xsd">     <flow name="use_xpath3_extract_value-Flow">         <http:listener config-ref="HTTP_Listener_Configuration" path="/advanced/xpath3" doc:name="HTTP">         <mulexml:dom-to-xml-transformer doc:name="DOM to XML">         <set-variable variablename="position_id" value="#[xpath3('/*:Create_Position_Response/*:Position_Reference/*:ID[2]')]" doc:name="Variable">         <set-variable variablename="position_id2" value="#[xpath3('/*:Create_Position_Response/*:Position_Reference/*:ID[@*:type=\'Position_ID\']')]" doc:name="Variable">         <logger message="#[payload]" level="INFO" doc:name="Logger">     </logger></set-variable></set-variable></mulexml:dom-to-xml-transformer></http:listener></flow>     <flow name="use_xpath3_splitter-Flow">         <http:listener config-ref="HTTP_Listener_Configuration" path="/advanced/splitter" doc:name="HTTP">         <splitter expression="#[xpath3('//StockMovementDataRequest/StockMovementData',payload,'NODESET')]" doc:name="Splitter">         <mulexml:dom-to-xml-transformer doc:name="DOM to XML">         <logger level="INFO" doc:name="Logger">         <collection-aggregator failontimeout="true" doc:name="Collection Aggregator">         <dw:transform-message doc:name="Transform Message">             <dw:set-payload><!--[CDATA[%dw 1.0 %output application/json --- payload]]--></dw:set-payload>         </dw:transform-message>         <logger message="#[payload]" level="INFO" doc:name="Logger">     </logger></collection-aggregator></logger></mulexml:dom-to-xml-transformer></splitter></http:listener></flow> </mule>