Re: Parsing XML Data (especially if it's SOAP)

eliotlandrum · ‎Feb 21, 2020

Recently I needed to be able to parse and handle XML data natively inside of a ThingWorx script, and this XML file happened to have a SOAP namespace as well. I learned a few things along the way that I couldn’t find a lot of documentation on, so am sharing here.

Lessons Learned

The biggest lesson I learned is that ThingWorx uses “E4X” XML handling. This is a language that Mozilla created as a way for JavaScript to handle XML (the full name is “ECMAscript for XML”). While Mozilla deprecated the language in 2014, Rhino, the JavaScript engine that ThingWorx uses on the server, still supports it, so ThingWorx does too. Here’s a tutorial on E4X - https://developer.mozilla.org/en-US/docs/Archive/Web/E4X_tutorial
The built-in linter in ThingWorx will complain about E4X syntax, but it still works.
I learned how to get to the data I wanted and loop through to create an InfoTable. Hopefully this is what you want to do as well.

Selecting an Element and Iterating

My data came inside of a SOAP envelope, which was meaningless information to me. I wanted to get down a few layers. Here’s a sample of my data that has made-up information in place of the customer's original data:

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" headers="">
    <SOAP-ENV:Body>
        <get_part_schResponse xmlns="urn:schemas-iwaysoftware-com:iwse">
            <get_part_schResult>
                <get_part_schRow>
                    <PART_NO>123456</PART_NO>
                    <ORD_PROC_DIV_CD>E</ORD_PROC_DIV_CD>
                    <MFG_DIV_CD>E</MFG_DIV_CD>
                    <SCHED_DT>2020-01-01</SCHED_DT>
                </get_part_schRow>
                <get_part_schRow>
                    <PART_NO>789456</PART_NO>
                    <ORD_PROC_DIV_CD>E</ORD_PROC_DIV_CD>
                    <MFG_DIV_CD>E</MFG_DIV_CD>
                    <SCHED_DT>2020-01-01</SCHED_DT>
                </get_part_schRow>
            </get_part_schResult>
        </get_part_schResponse>
    </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

To get to the schRow data, I need to get past SOAP and into a few layers of XML. To do that, I make a new variable and use the E4X selections to get there:
```
var data = resultXML.*::Body.*::get_part_schResponse.*::get_part_schResult.*;
```
1. Note a few things:
  1. resultXML is a variable in the service that contains the XML data.
  2. I skipped the Envelope tag since that’s the root.
  3. The .* syntax does not mean “all the following”, it means “all namespaces”. You can define and specify the namespaces instead of using .*, but I didn’t find value in that. I found some sample code that theoretically should work on a VMware forum: https://communities.vmware.com/thread/592000.
2. This gives me schRow as an XML List that I can iterate through. You can see what you have at this point by converting the data to a String and outputting it:
```
var result = String(data);
```

Now that I am to the schRow data, I can use a for loop to add to an InfoTable:

for each (var row in data) { 
    result.AddRow({
        PartNumber: row.*::PART_NO,
        OrderProcessingDivCD: row.*::ORD_PROC_DIV_CD,
        ManufacturingDivCD: row.*::MFG_DIV_CD,
        ScheduledDate: row.*::SCHED_DT
    });
}

Shoo! That’s it! Data into an InfoTable! Next time, I'll ask for a JSON API.

PaiChung · ‎Mar 02, 2020

Very cool. should also add the @ part as well when you have multiple elements within a row.

cbaurand · ‎Jan 05, 2021

i,

Does this code still working ?

var data = resultXML.*::Body.*::get_part_schResponse.*::get_part_schResult.*;

If yes, should i add something to make it work ?

PaiChung · ‎Jan 05, 2021

I think the example was developed on Thingworx 8.x not sure what version you are using, but it should still be valid.

eliotlandrum · ‎Jan 05, 2021

Yes, it was developed in 8.5.3. As mentioned in the original post, you will get Linting warnings but it does actually work. I have not tested on any newer versions 8.5.3, but the Rhino engine hasn't changed versions, so I would assume it still works.

eliotlandrum · ‎Apr 06, 2021

I recently needed to get some properties from a Kepware Complex Update (it's under Advanced Tags) and needed a little simpler of an example than what's in the original post.

Here's the XML:

<ComplexUpdate>
    <Item>
        <Name>DCToolCh1.DCToolDev1.LTR.LTR_ID</Name>
        <Value DataType="5">38164</Value>
        <Quality>192</Quality>
        <TimeStamp>132621347700044322</TimeStamp>
    </Item>
    <Item>
        <Name>DCToolCh1.DCToolDev1.LTR.LTR_TIMESTAMP</Name>
        <Value DataType="8">4/5/2021 10:19:30 PM</Value>
        <Quality>192</Quality>
        <TimeStamp>132621347700044322</TimeStamp>
    </Item>
    <Item>
        <Name>DCToolCh1.DCToolDev1.LTR.LTR_TORQUE_VALUE</Name>
        <Value DataType="5">2.25</Value>
        <Quality>192</Quality>
        <TimeStamp>132621347700044322</TimeStamp>
    </Item>
    <Item>
        <Name>DCToolCh1.DCToolDev1.LTR.LTR_TORQUE_STATUS</Name>
        <Value DataType="5">0</Value>
        <Quality>192</Quality>
        <TimeStamp>132621347700044322</TimeStamp>
    </Item>
    <Item>
        <Name>DCToolCh1.DCToolDev1.TOOLDATA.TOOLDATA_TSERIAL</Name>
        <Value DataType="8">238FH0929</Value>
        <Quality>192</Quality>
        <TimeStamp>132621347700044322</TimeStamp>
    </Item>
</ComplexUpdate>

Note that the TimeStamp is a Windows file time, which is not the same as Unix epoch (but similar). My code doesn't parse that at the moment, but it is something that could be done.

Also something I learned since I initially posted is that you can add jshint comments in to help with the linting issues with E4X (and I added the moz: true to stop getting warnings with my for each loop).

Here's the code to take that XML and return it as an InfoTable:

// jshint moz: true

// me.ComplexUpdate is a Kepware-bound property that is getting in the XML bundle
let complexUpdate = new XML(me.ComplexUpdate); 

// jshint ignore:start
let items = complexUpdate.*;
// jshint ignore:end

// CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(TS.Kepware.ComplexUpdate_DS)
let result = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape({
    infoTableName: "InfoTable",
    dataShapeName: "TS.Kepware.ComplexUpdate_DS"
});

// load the XML up into an InfoTable structure
for each (let item in items) {
    result.AddRow({
        Name: item.Name,
        Value: item.Value,
        // jshint ignore:start
        ValueDataType: parseInt(item.Value.@DataType),
        // jshint ignore:end
        Quality:  parseInt(item.Quality),
        TimeStamp:  parseInt(item.TimeStamp) 
    });
}

Hope these examples help someone!

Rocko · ‎Nov 10, 2021

Here's a small service you can use to convert XML to object or JSON without having to touch E4X.

https://community.ptc.com/t5/ThingWorx-Developers/SOAP-Webservices-amp-parsing-XML-amp-JSON/m-p/7576...