IoT Tips

The following script is a component of the Axeda Connected Configuration (CMDB) feature. It is used to provide configuration data for controlling package deployments via Connected Content (SCM). ConfigItem_CRU.groovy *Takes a POST request, not parameters import static com.axeda.sdk.v2.dsl.Bridges.* import com.axeda.drm.sdk.scripto.Request import com.axeda.services.v2.ConfigurationItem import com.axeda.services.v2.ConfigurationItemCriteria import com.axeda.services.v2.AssetConfiguration import com.axeda.services.v2.Asset import com.axeda.services.v2.ExecutionResult import groovy.json.JsonSlurper import net.sf.json.JSONObject import groovy.xml.MarkupBuilder /** * ConfigItem_CRU.groovy * ----------------------- * * Reads in json from an http post request and reads, adds, deletes or updates Configuration Items. * * * @note this parses a post and does not take any additional parameters. * * @author sara streeter <sstreeter@axeda.com> */ def contentType = "application/json" final def serviceName = "ConfigItem_CRU" def response = [:] def writer = new StringWriter() def xml = new MarkupBuilder(writer) try { // BUSINESS LOGIC BEGIN def assetId def validationOnly def validationResponse = "" List<ConfigurationItem> configItemList if (Request?.body != null && Request?.body !="") { def slurper = new JsonSlurper() def request = slurper.parseText(Request?.body) assetId = request.result.assetId validationOnly = request.result.validationOnly?.toBoolean() if (request.result.items != null && request.result.items.size() > 0){ configItemList = request.result.items.inject([]) { target, item -> if (item && item.path != "" && item.key != "" && item.path != null && item.key != null){ ConfigurationItem configItem = new ConfigurationItem() configItem.path = item.path + item.key configItem.value = item.value target << configItem } target } } } if (assetId != null) { def asset = assetBridge.find([assetId])[0] AssetConfiguration config = assetConfigurationBridge.getAssetConfiguration(assetId, "") def itemToDelete if (config == null) { createConfigXML(xml) AssetConfiguration configToCreate = assetConfigurationBridge.fromXml(writer.toString(), asset.id) ExecutionResult result = assetConfigurationBridge.create(configToCreate) AssetConfiguration config2 = assetConfigurationBridge.getAssetConfiguration(asset.id, "") config = config2 itemToDelete = "/Item" } if (configItemList != null && configItemList?.size() > 0){ List<ConfigurationItem> compareList = config.items def intersectingCompareItems = compareList.inject(["save": [], "delete": []]) { map, item -> // find whether to delete def foundItem = configItemList.findAll{ compare -> item?.path == compare?.path && item?.value == compare?.value } map[foundItem.size() > 0 ? "save" : "delete"] << item map } intersectingCompareItems.delete = intersectingCompareItems.delete.collect{it.path} if (itemToDelete){ intersectingCompareItems.delete.add(itemToDelete) } def intersectingConfigItems = configItemList.inject(["old": [], "new": []]) { map, item -> // find whether it's old def foundItem = compareList.findAll{ compare -> item?.path == compare?.path && item?.value == compare?.value } map[foundItem.size() > 0 ? "old" : "new"] << item map } assetConfigurationBridge.deleteConfigurationItems(config, intersectingCompareItems.delete) assetConfigurationBridge.appendConfigurationItems(config, intersectingConfigItems.new) def exResult = assetConfigurationBridge.validate(config) if (exResult.successful){ validationResponse = "success" if (!validationOnly){ assetConfigurationBridge.update(config) } } else { validationResponse = exResult.failures[0]?.details } } response = [ assetId: assetId, items: config?.items?.collect { item -> def origpath = item.path def lastSlash = origpath.lastIndexOf("/") def key = origpath.substring(lastSlash + 1, origpath.length()) def path = origpath.replace("/" + key, "") path += "/" [ path: path, key: key, value: item.value ] }, validationResponse: validationResponse ] } else { throw new Exception("Error: Asset Id must be provided.") } } catch (Exception ex) { logger.error ex response = [ error: [ type: "Backend Application Error" , msg: ex.getLocalizedMessage() ] ] } return ['Content-Type': 'application/json', 'Content': JSONObject.fromObject(response).toString(2)] /** * Create the Success response. * * @param xml : The xml response.<br> * @param info : If this is set to "1" the info element will be included in the response.<br> * @param infos : Collection of information to include within the info element of the response.<br> */ private void createConfigXML(xml) { xml.Item() }

May 12, 2016

In this post, I show how you can downsample time-series data on server side using the LTTB algorithm. The export comes with a service to setup sample data and a mashup which shows the data with weak to strong downsampling. Motivation: Users displaying time series data on mashups and dashboards (usually by a service using a QueryPropertyHistory-flavor in the background) might request large amounts of data by selecting large date ranges to be visualized, or data being recorded in high resolution. The newer chart widgets in Thingworx can work much better with a higher number of data points to display. Some also provide their own downsampling so only the „necessary“ points are drawn (e.g. no need to paint beyond the screen‘s resolution). See discussion here. However, as this is done in the widgets, this means the data reduction happens on client site, so data is sent over the network only to be discarded. It would be beneficial to reduce the number of points delivered to the client beforehand. This would also improve the behavior of older widgets which don’t have support for downsampling. Many methods for downsampling are available. One option is partitioning the data and averaging out each partition, as described here. A disadvantage is that this creates and displays points which are not in the original data. This approach here uses Largest-Triangle-Three-Buckets (LTTB) for two reasons: resulting data points exist in the original data set and the algorithm preserves the shape of the original curve very well, i.e. outliers are displayed and not averaged out. It also seems computationally not too hard on the server. Setting it up: Import Entities from LTTB_Entities.xml Navigate to thing LTTB.TestThing in project LTTB, run service downsampleSetup to setup some sample data Open mashup LTTB.Sampling_MU: Initially, there are 8000 rows sent back. The chart widget decides how many of them are displayed. You can see the rowcount in the debug info. Using the button bar, you determine to how many points the result will be downsampled and sent to the client. Notice how the curve get rougher, but the shape is preserved. How it works: The potentially large result of QueryPropertyHistory is downsampled by running it through LTTB. The resulting Infotable is sent to the widget (see service LTTB.TestThing.getData). LTTB implementation itself is in service downsampleTimeseries Debug mode allows you to see how much data is sent over the network, and how much the number decreases proportionally with the downsampling. LTTB.TestThing.getData; The export and the widget is done with TWX 9 but it's only the widget that really needs TWX 9. I guess the code would need some more error-checking for robustness, but it's a good starting point.

Aug 31, 2022

This script illustrates how to call a Groovy script as an external web service. This example also applies to calling any external web service that relies on a username and password. Parameters: external_username external_password script_name import com.axeda.drm.sdk.Context import com.axeda.drm.sdk.device.DeviceFinder import com.axeda.drm.sdk.data.CurrentDataFinder import com.axeda.drm.sdk.device.Device import com.axeda.drm.sdk.data.HistoricalDataFinder import com.axeda.drm.sdk.device.DataItem import net.sf.json.JSONObject import com.axeda.drm.sdk.device.ModelFinder import groovyx.net.http.* import static groovyx.net.http.ContentType.* import static groovyx.net.http.Method.* /** * CallScriptoAsExternalWebService.groovy * * This script illustrates how to call a Groovy script as an external web service. * * @param external_username - (REQ):Str Username for the external web service. * @param external_password - (REQ):Str Password for the external web service. * @param script_name - (REQ):Str Script Name to call. * * */ def result try { validateParameters(actual: parameters, expected: ["external_username", "external_password", "script_name"]) // authentication tokens (username + password) def auth_tokens = [username: parameters.external_username, password: parameters.external_password] http = new HTTPBuilder( 'http://platform.axeda.com/services/v1/rest/Scripto/execute/'+parameters.script_name ) // pass in dummy parameters to the script for illustration def parammap = [key1: "val1", key2: "val2"] // Call the script http.request (GET, JSON) { uri.query = auth_tokens + parammap response.success = { resp, json -> // traverse the wrapped json response result = json.wsScriptoExecuteResponse.content.$ } response.failure = { resp -> result = response.failure } } } catch (Throwable any) { logger.error any.localizedMessage } return ['Content-Type': 'application/json', 'Content': result] static def validateParameters(Map args) { if (!args.containsKey("actual")) { throw new Exception("validateParameters(args) requires 'actual' key.") } if (!args.containsKey("expected")) { throw new Exception("validateParameters(args) requires 'expected' key.") } def config = [ require_username: false ] Map actualParameters = args.actual.clone() as Map List expectedParameters = args.expected config.each { key, value -> if (args.options?.containsKey(key)) { config[key] = args.options[key] } } if (!config.require_username) { actualParameters.remove("username") } expectedParameters.each { paramName -> if (!actualParameters.containsKey(paramName) || !actualParameters[paramName]) { throw new IllegalArgumentException( "Parameter '${paramName}' was not found in the query; '${paramName}' is a reqd. parameter.") } } }

May 10, 2016

Calling external services from M2M applications is a critical aspect of building end-to-end solutions. Knowing how to apply network timeouts when connecting to external servers can prevent unexpected and problematic network hang-ups. Let's investigate how to create a safe networking flow using HttpClient, HttpBuilder, and Apache’s FTPClient class. Background Custom Objects called from Expression Rules have a configurable maximum execution time. This is set by the com.axeda.drm.rules.statistics.rule-time-threshold property. Without this safeguard in place long running or misbehaved Custom Objects can cause internal processing queues to fill and the server will suffer a performance degradation. In Java (and Groovy) all network calls internally use InputStream.read() to establish the socket connection and to read data from the socket. It is possible for faulty external servers (such as an FTP server) to hang and not properly respond. This means that the InputStream.read() method will continuously wait for the server to respond with data, and the server will never respond. According to the Java spec, InputStream.read() may be uninterruptable while it is waiting for data. This means that if a Custom Object has exceeded the com.axeda.drm.rules.statistics.rule-time-threshold the Rule Sniper will still not be able to interrupt the Custom Object’s execution if it is waiting on InputStream.read(). Because the Custom Object cannot be stopped, the internal processing queues will eventually fill. Even though InputStream.read() is uninterruptable it is still possible to set timeouts for it to be able to give up on a connection. Beyond that, we want to make sure that the connection is completely disconnected. Types of Timeouts There are typically two types of timeouts that should be set when making calls over the web: the Connection Timeout and the Socket Timeout. The Connection Timeout is the maximum amount of time that should be allowed when establishing the bi-directional socket connection between the client and server. Behind the scenes socket connection involves resolving the domain name of the server to an IP address, and then the server opening a port to connect with the client’s port. The Socket Timeout is the timeout that limits the amount of time each socket operation is allowed to take. It limits the amount of time InputStream.read() will listen for a server’s response. If a server is faulty or overloaded it may take a long time (or forever) to respond to a request. This timeout limits the amount of time the client will wait for the server to respond. When making any calls from a Custom Object to an external server (either making WebService calls, or FTP transfers), you should always set the Connection Timeout and the Socket Timeout. Always try to keep the timeouts as reasonably small as possible. Failure to do so could unexpectedly impact your Axeda server. Consider a Custom Object that takes an average of 10 seconds to run is called to make an external WebService call once a minute. This will not cause any issues and the system will be stable. If the external server suddenly has a performance degredation and now the external WebService call takes over a minute to run, the execution queue will eventually fill, causing performance degradation to the Axeda system. To protect against this scenario, set the timeouts to limit the call to one minute, and log whenever the time limit is exceeded. Examples Provided below are examples of properly set timeouts and thorough connection management use HttpClient, HttpBuilder, and FTPClient. All of these examples assume they are being executed from Custom Objects. By default, set the Connection Timeout to 10 seconds. In normal circumstances, connections should not take more then 10 seconds. If they are exceeding this time there is a good chance of networking issues between the client and server. The Socket Timeout can vary per use-case. The examples provided set the Socket Timeout to 30 seconds, which should be sufficient for typical WebService calls and small to medium sized FTP file transfers. Depending exactly on what is being done, the timout may have to be increased. If you expect the call to go over 5 minutes please contact Axeda Support to investigate increasing com.axeda.drm.rules.statistics.rule-time-threshold property (which defaults to 5 minutes). HttpClient //HttpClient import org.apache.http.client.HttpClient import org.apache.http.impl.client.DefaultHttpClient import org.apache.http.client.methods.HttpGet import org.apache.http.HttpResponse import org.apache.http.params.BasicHttpParams import org.apache.http.params.HttpParams import org.apache.http.params.HttpConnectionParams int TENSECONDS = 10*1000 int THIRTYSECONDS = 30*1000 final HttpParams httpParams = new BasicHttpParams() //Establishing the connection should take <10 seconds in most circumstances HttpConnectionParams.setConnectionTimeout(httpParams, TENSECONDS) //The data transfer/call should take <30 seconds. Adjust as necessary if receiving large data sets. HttpConnectionParams.setSoTimeout(httpParams, THIRTYSECONDS) HttpClient hc = new DefaultHttpClient(httpParams) try { //Simply get the contents of http://www.axeda.com and log it to the Custom Object Log HttpGet get = new HttpGet("http://www.axeda.com") HttpResponse response = hc.execute(get) BufferedReader br = new BufferedReader( new InputStreamReader( response.getEntity().getContent())) br.readLines().each { logger.info it } } finally { //Make sure to shutdown the connectionManager hc.getConnectionManager().shutdown() } return true https://gist.github.com/axeda/5189092/raw/2f7b93c5f96ed8f445df4364b885486bc6fa1feb/HttpClientTimeouts.groovy HttpBuilder import groovyx.net.http.HTTPBuilder import static groovyx.net.http.ContentType.* import static groovyx.net.http.Method.* int TENSECONDS = 10*1000; int THIRTYSECONDS = 30*1000; HTTPBuilder builder = new HTTPBuilder('http://www.axeda.com') //HTTPBuilder has no direct methods to add timeouts. We have to add them to the HttpParams of the underlying HttpClient builder.getClient().getParams().setParameter("http.connection.timeout", new Integer(TENSECONDS)) builder.getClient().getParams().setParameter("http.socket.timeout", new Integer(THIRTYSECONDS)) try { //Simply get the contents of http://www.axeda.com and log it to the Custom Object Log builder.request(GET, TEXT){ response.success = { resp, res -> res.readLines().each { logger.info it } } } } finally { //Make sure to always shut down the HTTPBuilder when you’re done with it builder.shutdown() } return true https://gist.github.com/axeda/5189102/raw/66bb3a4f4f096681847de1d2d38971e6293c4c6b/HttpBuilderTimeouts.groovy FtpClient Apache’s FTPClient has a third type of timeout, the Default Timeout. The Default Timeout is a timeout that further ensures that socket timeouts are always used. Note: Default Timeout does not set a timeout for the .connect() method. import org.apache.commons.net.ftp.* import java.io.InputStream import java.io.ByteArrayInputStream String ftphost = "127.0.0.1" String ftpuser = "test" String ftppwd = "test" int ftpport = 21 String ftpDir = "tmp/FTP" int TENSECONDS = 10*1000 int THIRTYSECONDS = 30*1000 //Declare FTP client FTPClient ftp = new FTPClient() try { ftp.setConnectTimeout(TENSECONDS) ftp.setDefaultTimeout(TENSECONDS) ftp.connect(ftphost, ftpport) //30 seconds to log on. Also 30 seconds to change to working directory. ftp.setSoTimeout(THIRTYSECONDS) def reply = ftp.getReplyCode() if (!FTPReply.isPositiveCompletion(reply)) { throw new Exception("Unable to connect to FTP server") } if (!ftp.login(ftpuser, ftppwd)) { throw new Exception("Unable to login to FTP server") } if (!ftp.changeWorkingDirectory(ftpDir)) { throw new Exception("Unable to change working directory on FTP server") } //Change the timeout here for a large file transfer that will take over 30 seconds //ftp.setSoTimeout(THIRTYSECONDS); ftp.setFileType(FTPClient.ASCII_FILE_TYPE) ftp.enterLocalPassiveMode() String filetxt = "Some String file content" InputStream is = new ByteArrayInputStream(filetxt.getBytes('US-ASCII')) try { if (!ftp.storeFile("myFile.txt", is)) { throw new Exception("Unable to write file to FTP server") } } finally { //Make sure to always close the inputStream is.close() } } catch(Exception e) { //handle exceptions here by logging or auditing } finally { //if the IO is timed out or force disconnected, exceptions may be thrown when trying to logout/disconnect try { //10 seconds to log off. Also 10 seconds to disconnect. ftp.setSoTimeout(TENSECONDS); ftp.logout(); //depending on the state of the server the .logout() may throw an exception, //we want to ensure complete disconnect. } catch(Exception innerException) { //You potentially just want to log that there was a logout exception. } finally { //Make sure to always disconnect. If not, there is a chance you will leave hanging sockects ftp.disconnect(); } } return true https://gist.github.com/axeda/5189120/raw/83545305a38d03b6a73a80fbf4999be3d6b3e74e/FtpClientConnectionTimeouts.groovy

May 10, 2016

This script creates a csv file from the audit log filtered by the User Access category, so dates of when users logged in or logged out. *** see update below *** Note: The csv file has the same name as the Groovy script and does NOT have the .csv extension . To get the .csv extension, the Groovy script has to be renamed to AuditEntryToCSV.csv.groovy . Suggestions on how to improve this are welcome. *** Update ***: The download works without the renamed groovy script by returning text instead of an input stream. The script has been modified to illustrate this. Parameters: days - the number of days past to fetch audit logs model_name - the model name of the asset serial_number - the serial number of the asset import com.axeda.drm.sdk.device.ModelFinder import com.axeda.drm.sdk.Context import com.axeda.common.sdk.id.Identifier import com.axeda.drm.sdk.device.Model import com.axeda.drm.sdk.device.DeviceFinder import com.axeda.drm.sdk.device.Device import com.axeda.drm.sdk.audit.AuditCategoryList import com.axeda.drm.sdk.audit.AuditCategory import com.axeda.drm.sdk.audit.AuditEntryFinder import com.axeda.drm.sdk.audit.SortType import com.axeda.drm.sdk.audit.AuditEntry import groovy.xml.MarkupBuilder import com.axeda.platform.sdk.v1.services.ServiceFactory /* * AuditEntryToCSV.groovy * * Creates a csv file from the audit log filtered by the User Access category, so dates of when users logged in or logged out. * * @param days - (REQ):Str number of days to search. * @param model_name - (REQ):Str name of the model. * @param serial_number - (REQ):Str serial number of the device. * * @note - the csv file has the same name as the Groovy script and does NOT have the .csv extension . To get * the .csv extension, the Groovy script has to be renamed to AuditEntryToCSV.csv.groovy . * * @author Sara Streeter <sstreeter@axeda.com> */ def writer = new StringWriter() def xml = new MarkupBuilder(writer) try { def ctx = Context.getUserContext() ModelFinder modelFinder = new ModelFinder(ctx) modelFinder.setName(parameters.model_name) Model model = modelFinder.find() DeviceFinder deviceFinder = new DeviceFinder(ctx) deviceFinder.setSerialNumber(parameters.serial_number) Device device = deviceFinder.find() AuditCategoryList acl = new AuditCategoryList() acl.add(AuditCategory.USER_ACCESS) long now = System.currentTimeMillis() Date today = new Date(now) def paramdays = parameters.days ? parameters.days: 5 long days = 1000 * 60 * 60 * 24 * Integer.valueOf(paramdays) AuditEntryFinder aef = new AuditEntryFinder(ctx) aef.setCategories(acl) aef.setToDate(today) aef.setFromDate(new Date(now - (days))) aef.setSortType(SortType.DATE) aef.sortDescending() List<AuditEntry> audits = aef.findAll() // use a Data Accumulator to store the information def dataStoreIdentifier = "FILE-CSV-audit_log" def daSvc = new ServiceFactory().dataAccumulatorService if (daSvc.doesAccumulationExist(dataStoreIdentifier, device.id.value)) { daSvc.deleteAccumulation(dataStoreIdentifier, device.id.value) } // assemble the response audits.each { AuditEntry audit -> def row = [ audit?.id.value, audit?.user?.username, audit?.date, audit?.category?.bundleKey, audit?.message ] row = row.join(',') row += '\n' daSvc.writeChunk(dataStoreIdentifier, device.id.value, row); } // stream the data accumulator to create the file InputStream is = daSvc.streamAccumulation(dataStoreIdentifier, device.id.value) return ['Content-Type': 'text/csv', 'Content-Disposition':'attachment; filename=AuditEntryCSVFile.csv', 'Content': is.text] } catch (def ex) { xml.Response() { Fault { Code('Groovy Exception') Message(ex.getMessage()) StringWriter sw = new StringWriter(); PrintWriter pw = new PrintWriter(sw); ex.printStackTrace(pw); Detail(sw.toString()) } } logger.info(writer.toString()) }

May 10, 2016

I recently had a customer who wanted to run services on ThingWorx from Power BI to retrieve existing operational data, and we were a bit stumped on how to pass the API key over in the headers, so I did a bit of Googling and pieced together the solution. It's not quite intuitive on the Power BI side, so I thought it would be helpful to share. If you have any other experience with integrating ThingWorx with Power BI, feel free to add a comment. Prepare ThingWorx Create an Application Key that has Run Time execution access to the services you need. Understand the inputs needed for the service you would like. I'll have examples of none, one, an InfoTable, and multiple inputs. Power BI Following the following steps in Power BI: 1. In Power BI, create a new blank query 2. On the left, right click on Query1 and go to the Advanced Editor: 3. Replace all of the body content with the following, replacing your API key, appropriate end point, and base URL as needed (this is an example with NO input parameters, I'll follow with examples of other parameters): let appKey = "your-application-key-here", endpoint = "Things/YourThingNameHere/Services/YourServiceNameHere", baseUrl = "https://YourServerNameHere/Thingworx/", url = Text.Combine({baseUrl,endpoint}), body = "", request = Web.Contents( url, [ Headers = [ appKey = appKey, #"Content-Type" = "application/json", Accept = "application/json" ], Content = Text.ToBinary(body) ] ), Source = Json.Document(request) in Source 4. Click "Done", and now you'll have a warning about how to connect. Click the "Edit Credentials" button. 5. Leave it on Anonymous and click "Connect": 6. You should now see the return data coming from ThingWorx. Note that I had a little trouble with this authentication initially and it saved the wrong method. To clear that out, go to the ribbon bar item "Data source settings" and select the server and clear it out. Other Examples Here is an example for sending a single string parameter: let appKey = "your-application-key-here", endpoint = "Things/YourThingNameHere/Services/YourServiceNameHere", baseUrl = "https://YourServerNameHere/Thingworx/", url = Text.Combine({baseUrl,endpoint}), body = "{""InputParameter"": ""InputValue""}", request = Web.Contents( url, [ Headers = [ appKey = appKey, #"Content-Type" = "application/json", Accept = "application/json" ], Content = Text.ToBinary(body) ] ), Source = Json.Document(request) in Source Here's an example of sending a string and an integer: let appKey = "your-application-key-here", endpoint = "Things/YourThingNameHere/Services/YourServiceNameHere", baseUrl = "https://YourServerNameHere/Thingworx/", url = Text.Combine({baseUrl,endpoint}), body = "{""InputString"": ""Hello, world!"", ""InputNumber"" : 42}", request = Web.Contents( url, [ Headers = [ appKey = appKey, #"Content-Type" = "application/json", Accept = "application/json" ], Content = Text.ToBinary(body) ] ), Source = Json.Document(request) in Source Here is an example for sending an InfoTable. Note that you must supply the dataShape with fieldDefinitions. If you're using an existing Data Shape, you can get the JSON by using the service GetDataShapeMetadataAsJSON() that is on the data shape. let appKey = "your-application-key-here", endpoint = "Things/YourThingNameHere/Services/YourServiceNameHere", baseUrl = "https://YourServerNameHere/Thingworx/", url = Text.Combine({baseUrl,endpoint}), body = "{""propertyNames"": { ""rows"": [ { ""name"": ""FirstEntityName"", ""description"": ""The first entity"" }, { ""name"": ""SecondEntityName"", ""description"": ""The second entity"" }], ""dataShape"": { ""fieldDefinitions"": { ""name"": { ""name"": ""name"", ""aspects"": { ""isPrimaryKey"": true }, ""description"": ""Entity name"", ""baseType"": ""STRING"", ""ordinal"": 0 }, ""description"": { ""name"": ""description"", ""aspects"": {}, ""description"": ""Entity description"", ""baseType"": ""STRING"", ""ordinal"": 0 } } } }}", request = Web.Contents( url, [ Headers = [ appKey = appKey, #"Content-Type" = "application/json", Accept = "application/json" ], Content = Text.ToBinary(body) ] ), Source = Json.Document(request) in Source If I find any more interesting ways to use Power BI with ThingWorx services, I'll add them on here.

Mar 23, 2022

This script finds all the data items both current and historical on all the assets of a model and outputs them as XML. Parameters: model_name from_time to_time import com.axeda.drm.sdk.Context import com.axeda.drm.sdk.device.ModelFinder import com.axeda.drm.sdk.device.Model import com.axeda.drm.sdk.device.DeviceFinder import com.axeda.drm.sdk.data.CurrentDataFinder import com.axeda.drm.sdk.device.Device import com.axeda.drm.sdk.data.HistoricalDataFinder import groovy.xml.MarkupBuilder /* * AllDataItems2XML.groovy * * Find all the historical and current data items for all assets in a given model. * * @param model_name - (REQ):Str name of the model. * @param from_time - (REQ):Long millisecond timestamp to begin query from. * @param to_time - (REQ):Long millisecond timestamp to end query at. * * @note from_time and to_time should be provided because it limits the query size. * * @author Sara Streeter <sstreeter@axeda.com> */ def response = [:] def writer = new StringWriter() def xml = new MarkupBuilder(writer) // measure the script run time def timeProfiles = [:] def scriptStartTime = new Date() try { // getUserContext is supported as of release 6.1.5 and higher final def CONTEXT = Context.getUserContext() // confirm that required parameters have been provided validateParameters(actual: parameters, expected: ["model_name", "from_time", "to_time"]) // find the model def modelFinder = new ModelFinder(CONTEXT) modelFinder.setName(parameters.model_name) Model model = modelFinder.findOne() // throw exception if no model found if (!model) { throw new Exception("No model found for ${parameters.model_name}.") } // find all assets of that model def assetFinder = new DeviceFinder(CONTEXT) assetFinder.setModel(model) def assets = assetFinder.findAll() // find the current and historical data values for each asset //note: since device will be set on the datafinders going forward, a dummy device is set on instantiation which is not actually stored def currentDataFinder = new CurrentDataFinder(CONTEXT, new Device(CONTEXT, "placeholder", model)) def historicalDataFinder = new HistoricalDataFinder(CONTEXT, new Device(CONTEXT, "placeholder", model)) historicalDataFinder.startDate = new Date(parameters.from_time as Long) historicalDataFinder.endDate = new Date(parameters.to_time as Long) // assemble the response xml.Response(){ assets.each { Device asset -> currentDataFinder.device = asset def currentValueList = currentDataFinder.find() historicalDataFinder.device = asset def valueList = historicalDataFinder.find() Asset(){ id(asset.id.value) name( asset.name) serial_number(asset.serialNumber) model_id( asset.model.id.value) model_name(asset.model.name) current_data(){ currentValueList.each{ data -> timestamp( data?.getTimestamp()?.format("yyyyMMdd HH:mm")) name(data?.dataItem?.name) value( data?.asString()) }} historical_data(){ valueList.each { data -> timestamp( data?.getTimestamp()?.format("yyyyMMdd HH:mm")) name(data?.dataItem?.name) value( data?.asString()) }} } } } } catch (def ex) { xml.Response() { Fault { Code('Groovy Exception') Message(ex.getMessage()) StringWriter sw = new StringWriter(); PrintWriter pw = new PrintWriter(sw); ex.printStackTrace(pw); Detail(sw.toString()) } } } return ['Content-Type': 'text/xml', 'Content': writer.toString()] private Map createTimeProfile(String label, Date startTime, Date endTime) { [ (label): [ startTime: [timestamp: startTime.time, readable: startTime.toString()], endTime: [timestamp: endTime.time, readable: endTime.toString()], profile: [ elapsed_millis: endTime.time - startTime.time, elapsed_secs: (endTime.time - startTime.time) / 1000 ] ] ] } private validateParameters(Map args) { if (!args.containsKey("actual")) { throw new Exception("validateParameters(args) requires 'actual' key.") } if (!args.containsKey("expected")) { throw new Exception("validateParameters(args) requires 'expected' key.") } def config = [ require_username: false ] Map actualParameters = args.actual.clone() as Map List expectedParameters = args.expected config.each { key, value -> if (args.options?.containsKey(key)) { config[key] = args.options[key] } } if (!config.require_username) { actualParameters.remove("username") } expectedParameters.each { paramName -> if (!actualParameters.containsKey(paramName) || !actualParameters[paramName]) { throw new IllegalArgumentException( "Parameter '${paramName}' was not found in the query; '${paramName}' is a reqd. parameter.") } } } Sample Output: <Response> <Asset> <id>2864</id> <name>keg24</name> <serial_number>keg24</serial_number> <model_id>1081</model_id> <model_name>Kegerator</model_name> <current_data> <timestamp>20111103 14:44</timestamp> <name>currKegPercentage</name> <value>34.0</value> <timestamp>20111103 14:38</timestamp> <name>currTempF</name> <value>43.0</value> </current_data> <historical_data /> </Asset> <Asset> <id>2861</id> <name>keg28</name> <serial_number>keg28</serial_number> <model_id>1081</model_id> <model_name>Kegerator</model_name> <current_data> <timestamp /> <name>currKegPercentage</name> <value>?</value> <timestamp>20111103 14:21</timestamp> <name>currTempF</name> <value>43.0</value> </current_data> <historical_data /> </Asset> <Asset> <id>2863</id> <name>keg21</name> <serial_number>keg21</serial_number> <model_id>1081</model_id> <model_name>Kegerator</model_name> <current_data> <timestamp /> <name>currKegPercentage</name> <value>?</value> <timestamp>20111103 14:39</timestamp> <name>currTempF</name> <value>42.0</value> </current_data> <historical_data /> </Asset> <Asset> <id>2862</id> <name>keg25</name> <serial_number>keg25</serial_number> <model_id>1081</model_id> <model_name>Kegerator</model_name> <current_data> <timestamp>20111103 14:36</timestamp> <name>currKegPercentage</name> <value>34.0</value> <timestamp /> <name>currTempF</name> <value>?</value> </current_data> <historical_data /> </Asset> <Asset> <id>2867</id> <name>keg29</name> <serial_number>keg29</serial_number> <model_id>1081</model_id> <model_name>Kegerator</model_name> <current_data> <timestamp>20111103 14:48</timestamp> <name>currKegPercentage</name> <value>35.0</value> <timestamp /> <name>currTempF</name> <value>?</value> </current_data> <historical_data /> </Asset> <Asset> <id>2865</id> <name>keg27</name> <serial_number>keg27</serial_number> <model_id>1081</model_id> <model_name>Kegerator</model_name> <current_data> <timestamp>20111103 14:39</timestamp> <name>currKegPercentage</name> <value>34.0</value> <timestamp>20111103 14:44</timestamp> <name>currTempF</name> <value>42.0</value> </current_data> <historical_data /> </Asset> <Asset> <id>2866</id> <name>keg23</name> <serial_number>keg23</serial_number> <model_id>1081</model_id> <model_name>Kegerator</model_name> <current_data> <timestamp>20111103 14:46</timestamp> <name>currKegPercentage</name> <value>34.0</value> <timestamp /> <name>currTempF</name> <value>?</value> </current_data> <historical_data /> </Asset> </Response>

May 10, 2016

This is an example of an advanced apc-metadata.xml file for use with Axeda Artisan that includes examples of how to create different data structures on the platform, including Expression Rules and System Timers. Step 1 - In the upload.xml make sure the artisan-installer is 1.2 <dependencies> <dependency> <groupId>com.axeda.community</groupId> <artifactId>artisan-installer</artifactId> <version>1.2</version> </dependency> </dependencies> Step 2 - <?xml version="1.0" encoding="UTF-8"?>  <apcmetadata> <models> <model> <name>DemoModel</name>  <type>gateway</type> <DataItems> <DataItem> <name>PendingMessage</name>  <type>string</type> <visible>false</visible> <stored>true</stored>  <storageOption>2</storageOption> <readOnly>false</readOnly> </DataItem> <DataItem> <name>ConsumableData</name>  <type>string</type> <visible>false</visible> <stored>true</stored>  <storageOption>2</storageOption> <readOnly>false</readOnly> </DataItem> </DataItems> </model> </models> <ruleTimers> <ruletimer> <name>SFTP Retry</name> <description></description>  <schedule>0 0 0 * * ?</schedule> <rules> <rule>SFTP Retry</rule> </rules> </ruletimer> </ruleTimers> <expressionRules> <rule> <name>SFTP Retry</name> <description></description> <enabled>true</enabled> <applyToAll>true</applyToAll> <type>SystemTimer</type> <ifExpression><![CDATA[true]]></ifExpression> <thenExpression> <![CDATA[ExecuteCustomObject("SFTPRetry")]]></thenExpression> <elseExpression></elseExpression> <consecutive>true</consecutive> <models> <model>DemoMOdel</model> </models> </rule> </expressionRules> <customobjects> <customobject> <name>GetChartData</name> <type>Action</type> <sourcefile>GetChartData.groovy</sourcefile> <params> <param name="username" description="(REQUIRED) The name of the calling user"/> </params> </customobject> <customobject> <name>GetChartData_rss</name> <type>Action</type> <sourcefile>GetChartData_rss.groovy</sourcefile> <params> <param name="username" description="(REQUIRED) The name of the calling user"/> </params> </customobject> <customobject> <name>GetAddress</name> <type>Action</type> <sourcefile>GetAddress.groovy</sourcefile> <params>  </params> </customobject> </customobjects> <applications> <application> <description>Chart Example</description> <applicationId>chartexample</applicationId> <indexFile>index.html</indexFile>  <sourcePath>artisan-starter-html/src/main/webapp</sourcePath> </application> </applications> </apcmetadata>

May 10, 2016

Shown below is example code that when deployed in the appropriate container, will allow an end-user to talk to the Axeda Platform Integration Queue. A customer should supply their unique values for the following properties: queueName user password url import java.util.Properties; import javax.jms.*; import javax.naming.*; public class SampleConsumer { private String queueName = "com.axeda.integration.ACME.queue"; private String user = "system"; private String password = "manager"; //private String url = "ssl://hostname:61616"; private String url = "tcp://hostname:61616"; private boolean transacted; private boolean isRunning = false; public static void main(String[] args) throws NamingException, JMSException { SampleConsumer consumer = new SampleConsumer(); consumer.run(); } public SampleConsumer() { /** For SSL connections only, add the following: **/ // System.setProperty("javax.net.ssl.keyStore", "path/to/client.ks"); // System.setProperty("javax.net.ssl.keyStorePassword", "password"); // System.setProperty("javax.net.ssl.trustStore", "path/to/client.ts"); } public void run() throws NamingException, JMSException { isRunning = true; //JNDI properties Properties props = new Properties(); props.setProperty(Context.INITIAL_CONTEXT_FACTORY, "org.apache.activemq.jndi.ActiveMQInitialContextFactory"); props.setProperty(Context.PROVIDER_URL, url); //specify queue propertyname as queue.jndiname props.setProperty("queue.slQueue", queueName); javax.naming.Context ctx = new InitialContext(props); ConnectionFactory connectionFactory = (ConnectionFactory)ctx.lookup("ConnectionFactory"); Connection connection = connectionFactory.createConnection(user, password); connection.start(); Session session = connection.createSession(transacted, Session.AUTO_ACKNOWLEDGE); Destination destination = (Destination)ctx.lookup("slQueue"); //Using Message selector ObjectClass = ‘AlarmImpl’ MessageConsumer consumer = session.createConsumer(destination, "ObjectClass= 'LinkedList'"); while (isRunning) { System.out.println("Waiting for message..."); Message message = consumer.receive(1000); if (message != null && message instanceof TextMessage) { TextMessage txtMsg = (TextMessage)message; System.out.println("Received: " + txtMsg.getText()); } } System.out.println("Closing connection"); consumer.close(); session.close(); connection.close(); } }

May 10, 2016

Background Getting a performance benchmark of your running application is an important thing to do when deploying and scaling up an application in production. This not only helps focus in on performance issues quickly, but also allows for safely planning for scaling up and resource sizing based on real concrete data. I recently created a tool and made a post about capturing and analysing ThingWorx utilisation statistics to do such an analysis, as well as identifying potential performance bottlenecks. Although they are rich and precise, utilisation statistics fall short in a number of areas however - specifically being able to count and time specific service executions, as well as identifying and sorting based on the host executing the service. Tomcat Access Log Analysis As ThingWorx is a Tomcat web application, Tomcat logs details of the requests being made to the application server and ThingWorx REST API. The default settings include the host (IP address), date/timestamp, and request URI; which can be decoded to reveal relevant details like the calling entities and service executions. Adding 3 key additional variables (%s %B %D) to the server.xml access log value also gives us the HTTP response code, service execution time, and bytes returned from Tomcat. This is super useful as we can now determine exact time of service executions, and run statistics on their execution totals and execution time. Once you have an access log file looking like the one above, you can attempt to load it into the access_log sheet in the analysis Excel workbook that I created. You do this by click on the access_log table, then selecting "Data > Get Data > Data Source Settings". You'll then be prompted with the following or similar pop-up allowing you to navigate to your access_log file to select and then load. It should be noted that you'll have to Refresh the table after selecting the new access_log.txt file so that it is read in and populates the table. You can do this by right-clicking on the table and saying Refresh, or using the Data > Refresh button. This workbook relies on a number of formulas to slice and dice the timestamp, and during my attempts at importing I had significant issues with this due to some of the ways that Excel does things automatically without any manual options. You really need to make sure that the timestamps are imported and converted correctly, or something in the workbook will likely not work as intended. One thing that I had to do was to add 1 second to round up 00:00:00 for the first entries as this was being imported as a date without the time part, and then the next lines imported as a date/time. Depending on how many lines your file is, you'll likely also have to "Fill Down" the formulas on the right side of the sheet which may be empty in the table after importing your new data set. I had the best results by selecting the cells in question on the last row, then going down to the bottom corner, pushing and holding Shift, clicking on the last cell bottom right, and then selecting Home > Fill > Down to pull the formulas down from the top. Once the data is loaded, you'll be able to start poking around. The filters and sorting by the named columns is really helpful as you can start out by doing things like removing a particular host, sorting by longest execution times, selecting execution times greater than 4 seconds, or only showing activity aimed at a particular entity or service. You really need to make sure that the imported data worked fine and looks perfect, as the next steps will totally break if not. With the data loaded, you can now go to the Summary Data table and right-click on one of the tables and select Refresh. This is reload the data in into the pivot table and re-run their calculations. Once the refresh is complete, you should see the table summary like shown here; there are Day, Hour, and Minute expand/collapse buttons. You should also see the Day, Hour, Month fields showing in the Field Definitions on the right. This is the part that is painful -- if the dates are in the wrong format and Excel is unable to auto-detect everything in the same way, then you will not get these automatically created fields. With the data reloaded, and Pivot Tables re-built, you should be able to go over to the Dashboard sheet to start looking at and analysing the graphs. This one is showing the Top 10 services organised into hourly buckets with cumulated service execution times. I'm not going to go into all of the workbooks features, but you can also individually select a set of key services that you want to have a look at together across both the execution count and execution time dimensions. Next you can see the coordinated view of both total service execution time over number or service executions. This is helpful for looking for patterns where a service may be executing longer but being triggered the same amount of times, compared to both being executed and taking more time. I've created a YouTube video (see bottom) which goes through using all of the features as well as providing other pointers to using it. Getting into a finer level of detail, this "bonus" sheet provides a Pivot Table and Pivot Chart which allows for exploring minimum, maximum and average execution time for a specific service. Comparing this with the utilisation subsystem metrics taken during the same period now provide much deeper insight as we can pinpoint there the peaks were, how long they lasted, and where the slow executions were in relation to other services being executed at that time (example: identifying many queries/data processing occurring simultaneously). Without further ado, you can download and play with my ThingWorx Tomcat Access Log Analysis Excel Workbook, and check out the recorded demonstration and explanation for more details on loading and analysis use. [YouTube] ThingWorx Tomcat Access Logs - Service Performance Analysis

Feb 28, 2022

Since the advent of 6.1.6 we've been able to access the body of a post in a Groovy script. This frees us from the tyranny of those pesky predefined parameters and opens up all sorts of Javascript object-passing possibilities. To keep this example as simple as possible, there are only two files: postbody.html TestPostBody.groovy postbody.html <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <title>Scripto Post Body Demo</title> <meta http-equiv="content-type" content="text/html; charset=utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"/> <meta name="apple-mobile-web-app-capable" content="yes"/> <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> <meta http-equiv="refresh" content="3600"> </head> <body> <div id="wrapper"> <div id="header"> <h1>Scripto Post Body Demo</h1> </div> <p> Username:<input type="text" id="username" /><br /> Password:<input type="password" id="password" /><br /><br /> Enter some valid JSON (validate it <a href="http://jsonlint.com/" alt="jsonlint">here</a> if you're not sure): <br /><textarea id="jsoninput" rows=10 cols=20></textarea><br /> Enter arbitrary Text: <br /><textarea id="textinput" rows=10 cols=20></textarea> <input type="submit" value="Go" id="submitbtn" onclick="poststuff();"/> </p> <div id="response"></div> </div> <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script> <script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.16/jquery-ui.min.js"></script> <script type="text/javascript"> var poststuff= function(){ var data = {} var temp if ($("#jsoninput").val() != ""){ try { temp = $.parseJSON($("#jsoninput").val()) } catch (e){ temp = "" } if (temp && temp != ""){ data = JSON.stringify(temp) } } else if ($("#textinput").val() != ""){ data.text = $("#textinput").val() data = JSON.stringify(data) } else data = {"testing":"hello"} if ($("#username").val() != "" && $("#password").val() != ""){ // you need contentType in order for the POST to succeed var promise = $.ajax({ type:"POST", url: "http://dev6.axeda.com/services/v1/rest/Scripto/execute/TestPostBody?username=" + $("#username").val() + "&password=" + $("#password").val(), data: data, contentType:"application/json; charset=utf-8", dataType:"text" }) $.when(promise).then(function(json){ $("#response").html("<p>" + JSON.stringify(json) + "</p><br />Check your console for the object.<br />") console.log($.parseJSON(json)) $("#jsoninput").val("") $("#textinput").val("") }) } } </script> </body> </html> TestPostBody.groovy import net.sf.json.JSONObject import groovy.json.JsonSlurper import static com.axeda.drm.sdk.scripto.Request.*; try { // just get the string body content response = [body: body] response.element = [] // parse the text into a JSON Object or JSONArray suitable for traversing in Groovy // this assumes the body is stringified JSON def slurper = new JsonSlurper() def result = slurper.parseText(body) result.each{ response.element << it } } catch (Exception e) { response = [ faultcode: 'Groovy Exception', faultstring: e.message ]; } return ["Content-Type": "application/json","Content":JSONObject.fromObject(response).toString(2)] The "body" variable is passed in as a standalone implicit object of type String. The key here is that to process the string as a Json object in Groovy, we send stringed JSON from the Javascript, rather than the straight JSON object. FYI: If you happen to be using Scripto Editor, you might like to know that importing the Request class disables the sidebar input of parameters. You can enter the parameters in the sidebar, but if this import is included the parameters will not be visible to the script. To access the POST body through the Request Object, you can also refer to: Using Axeda Scripto

May 10, 2016

Requirements: 6.1.2+ Geofences are geometric shapes drawn virtually on a geographical area that represents a fence that can be crossed by a device. The Axeda Platform has built-in support for mobile locations and geofences, which can be linked to the rules engine to enable notifications based on geofence crossing. What this tutorial covers This tutorial demonstrates the workflow of creating a geofence through to creating the expression rules with notifications, then how the mobile location can trigger the rules. 1) Creating the Geofence 2) Creating the Expression Rule There is currently no user interface built into the Axeda Applications Console which interacts with geofences. For a sample application with a geofence user interface, see Sample Project: Traxeda (TODO). For a single Custom Object that includes all of the functionality described below, see the end of this document. The properties of a geofence are a name, a description, and a series of coordinates based on Well-Known Text (WKT) syntax (see the OpenGIS Simple Features Specification). def addGeofence(CONTEXT, map){ Geofence myGeofence = new Geofence(CONTEXT) myGeofence.name = map.name if(map.type != "polygon" && map.type != "circle") { throw new Exception("Invalid type: need 'polygon' or 'circle', not '$map.type'") } else if(map.type == "polygon") { def geo = map.locs.loc.inject( "POLYGON (("){ str, item -> def lng = item.lng def lat = item.lat str += "$lng $lat," str } //the first location also has to be the last location myGeofence.geometry = geo + map.locs.loc[0].lng + " " + map.locs.loc[0].lat + "))" //Something like this is built: //POLYGON ((-71.082118 42.383892,-70.867198 42.540923,-71.203654 42.495374,-71.284678 42.349394,-71.163829 42.221382,-71.003154 42.266114,-71.082118 42.383892)) } else if(map.type == "circle") { def lng = map.locs.loc[0].lng def lat = map.locs.loc[0].lat myGeofence.geometry = "POINT ($lng $lat)" //POINT (-71.082118 42.383892) myGeofence.buffer = map.radius.toDouble() } myGeofence.description = "ALERT:::$map.alertType:::$map.alert" try { myGeofence.store() } catch (e){ logger.info e.localizedMessage return null } myGeofence } The geofence itself does not interact with devices in any way. Rather it is the Expression Rule that is applied to models and devices and that invokes the geofence when a mobile location is passed in. Creating the Expression Rule The Expression Rule for the Geofence is built as follows: TYPE: MobileLocation IF: Expression set to "InNamedGeofence" for entering and "!InNamedGeofence" for exiting. The following function creates this expression rule: /* Sample call createGeofenceExpressionRule(CONTEXT, "My Geofence", "rule_MyGeofence", "in", "You entered the geofence!", "SDK Generated Geofence Rule", 100) */ def createGeofenceExpressionRule(com.axeda.drm.sdk.Context CONTEXT, String geofencename, String rulename, String alertType, String alertMessage, String ruledescription, int severity){ ExpressionRuleFinder erf = new ExpressionRuleFinder(CONTEXT) erf.setName(rulename) ExpressionRule expressionRule1 = erf.findOne() expressionRule1?.delete() def expressionRule = new ExpressionRule(CONTEXT) expressionRule.setName(rulename) expressionRule.setDescription(ruledescription) expressionRule.setTriggerName("MobileLocation") def ifExpStr = "InNamedGeofence(\"$geofencename\", Location.location)" if(alertType == "out"){ ifExpStr = "!" + ifExpStr } expressionRule.setIfExpression(new Expression(ifExpStr)) expressionRule.setThenExpression(new Expression("CreateAlarm(\"$alertMessage\", severity)")) expressionRule.setEnabled(true) expressionRule.setConsecutive(false) expressionRule.store() expressionRule } Then the rule associations must be created to apply the rule to a model or device. /* Sample call findOrCreateRuleAssociations(CONTEXT, myModel, expressionRule, "EXPRESSION_RULE", "MODEL") Where expressionRule is the rule created in the above example */ def findOrCreateRuleAssociations(Context CONTEXT, Object entity, Object rule, String ruleType, String entityType){ // rule type is whether this is an expression rule ruleType = ruleType ?: "EXPRESSION_RULE" entityType = entityType ?: "DEVICE_INCLUDE" RuleAssociationFinder ruleAssociationFinder = new RuleAssociationFinder(CONTEXT) ruleAssociationFinder.setRuleId(rule.id.value) ruleAssociationFinder.setRuleType(RuleType.valueOf(ruleType)) ruleAssociationFinder.setEntityId(entity.id.value) ruleAssociationFinder.setEntityType(EntityType.valueOf(entityType)) def ruleAssociations = ruleAssociationFinder.findAll() if (!ruleAssociations || ruleAssociations?.size() == 0){ def ruleAssociation = new RuleAssociation(CONTEXT) ruleAssociation.entityId = entity.id.value ruleAssociation.entityType = EntityType.valueOf(entityType) ruleAssociation.ruleType = RuleType.valueOf(ruleType) ruleAssociation.setRuleId(rule.id.value) ruleAssociation.store() ruleAssociations = [ruleAssociation] } return ruleAssociations } The rule will now be triggered when any device of the applied model sends a mobile location within the geofence, which in turn will create an alarm. Here is a custom object with the complete geofence functionality: import com.axeda.drm.sdk.Context import com.axeda.drm.sdk.geofence.Geofence import com.axeda.drm.sdk.geofence.GeofenceFinder import com.axeda.drm.sdk.rules.engine.Expression import com.axeda.drm.sdk.rules.engine.ExpressionRule import com.axeda.drm.sdk.rules.engine.ExpressionRuleFinder import com.axeda.drm.sdk.rules.engine.RuleAssociation import com.axeda.drm.sdk.rules.engine.RuleAssociationFinder import com.axeda.drm.sdk.rules.engine.RuleType import com.axeda.drm.sdk.common.EntityType import com.axeda.drm.sdk.device.Model import com.axeda.drm.sdk.device.ModelFinder try { def Context CONTEXT = Context.getSDKContext() def model = findOrCreateModel(CONTEXT, "FooModel") def sampleCircle = [ "name": "My Circle", "alert": "My Geofence Alert Text", "type": "circle", "alertType": "in", "radius": "65.76", "locs": [ [ "loc": [ "lat": "42.60970621339408", "lng": "-73.201904296875" ] ] ] ] def samplePolygon = [ "name": "My Polygon", "alert": "My Geofence Alert Text", "type": "polygon", "alertType": "out", "locs": [ ["loc": [ "lng": -71.2604999542236, "lat": 42.3384903145478 ]], ["loc": [ "lng": -71.4218616485596, "lat": 42.3242772020001 ]], ["loc": [ "lng": -71.5585041046143, "lat": 42.2653600946699 ]], ["loc": [ "lng": -71.5413379669189, "lat": 42.1885837119108 ]], ["loc": [ "lng": -71.4719867706299, "lat": 42.1137514551207 ]], ["loc": [ "lng": -71.3737964630127, "lat": 42.0398506628541 ]], ["loc": [ "lng": -71.2508869171143, "lat": 42.0311807962068 ]], ["loc": [ "lng": -71.1355304718018, "lat": 42.2084223174036 ]], ["loc": [ "lng": -71.2604999542236, "lat": 42.3384903145478 ]] ] ] // find geofence if it exists def circle = findGeofenceByName(CONTEXT, sampleCircle.name) // create circular geofence if (!circle){ circle = addGeofence(CONTEXT, sampleCircle) } // create rule for circular geofence def circleRule = createGeofenceExpressionRule(CONTEXT, circle.name, "${circle.name}__Rule", sampleCircle.alertType, sampleCircle.alert, "SDK Generated Geofence Rule", 100) // apply rule to new Model findOrCreateRuleAssociations(CONTEXT, model, circleRule, "EXPRESSION_RULE", "MODEL") def polygon = findGeofenceByName(CONTEXT, samplePolygon.name) if (!polygon){ polygon = addGeofence(CONTEXT, samplePolygon) } def polygonRule = createGeofenceExpressionRule(CONTEXT, polygon.name, "${polygon.name}__Rule", samplePolygon.alertType, samplePolygon.alert, "SDK Generated Geofence Rule", 100) // apply rule to new Model findOrCreateRuleAssociations(CONTEXT, model, polygonRule, "EXPRESSION_RULE", "MODEL") } catch (Exception e) { logger.info(e.localizedMessage) } return true def findGeofenceByName(CONTEXT, name){ GeofenceFinder geofenceFinder = new GeofenceFinder(CONTEXT) geofenceFinder.setName(name) def geofence = geofenceFinder.find() geofence } def addGeofence(CONTEXT, map){ Geofence myGeofence = new Geofence(CONTEXT) myGeofence.name = map.name if(map.type != "polygon" && map.type != "circle") { throw new Exception("Invalid type: need 'polygon' or 'circle', not '$map.type'") } else if(map.type == "polygon") { def geo = map.locs.loc.inject( "POLYGON (("){ str, item -> def lng = item.lng def lat = item.lat str += "$lng $lat," str } //the first location also has to be the last location myGeofence.geometry = geo + map.locs.loc[0].lng + " " + map.locs.loc[0].lat + "))" //Something like this is built: //POLYGON ((-71.082118 42.383892,-70.867198 42.540923,-71.203654 42.495374,-71.284678 42.349394,-71.163829 42.221382,-71.003154 42.266114,-71.082118 42.383892)) } else if(map.type == "circle") { def lng = map.locs.loc[0].lng def lat = map.locs.loc[0].lat myGeofence.geometry = "POINT ($lng $lat)" //POINT (-71.082118 42.383892) myGeofence.buffer = map.radius.toDouble() } myGeofence.description = "ALERT:::$map.alertType:::$map.alert" try { myGeofence.store() } catch (e) { logger.info e.localizedMessage return null } myGeofence } def createGeofenceExpressionRule(com.axeda.drm.sdk.Context CONTEXT, String geofencename, String rulename, String alertType, String alertMessage, String ruledescription, int severity) { ExpressionRuleFinder erf = new ExpressionRuleFinder(CONTEXT) erf.setName(rulename) ExpressionRule expressionRule1 = erf.findOne() expressionRule1?.delete() def expressionRule = new ExpressionRule(CONTEXT) expressionRule.setName(rulename) expressionRule.setDescription(ruledescription) expressionRule.setTriggerName("MobileLocation") def ifExpStr = "InNamedGeofence(\"$geofencename\", Location.location)" if(alertType == "out"){ ifExpStr = "!" + ifExpStr } expressionRule.setIfExpression(new Expression(ifExpStr)) expressionRule.setThenExpression(new Expression("CreateAlarm(\"$alertMessage\", severity)")) expressionRule.setEnabled(true) expressionRule.setConsecutive(false) expressionRule.store() expressionRule } def findOrCreateRuleAssociations(Context CONTEXT, Object entity, Object rule, String ruleType, String entityType) { // rule type is whether this is an expression rule ruleType = ruleType ?: "EXPRESSION_RULE" entityType = entityType ?: "DEVICE_INCLUDE" RuleAssociationFinder ruleAssociationFinder = new RuleAssociationFinder(CONTEXT) ruleAssociationFinder.setRuleId(rule.id.value) ruleAssociationFinder.setRuleType(RuleType.valueOf(ruleType)) ruleAssociationFinder.setEntityId(entity.id.value) ruleAssociationFinder.setEntityType(EntityType.valueOf(entityType)) def ruleAssociations = ruleAssociationFinder.findAll() if (!ruleAssociations || ruleAssociations?.size() == 0){ def ruleAssociation = new RuleAssociation(CONTEXT) ruleAssociation.entityId = entity.id.value ruleAssociation.entityType = EntityType.valueOf(entityType) ruleAssociation.ruleType = RuleType.valueOf(ruleType) ruleAssociation.setRuleId(rule.id.value) ruleAssociation.store() ruleAssociations = [ruleAssociation] } return ruleAssociations } def findOrCreateModel(Context CONTEXT, String modelName) { ModelFinder modelFinder = new ModelFinder(CONTEXT) modelFinder.setName(modelName) def model = modelFinder.find() if (!model){ model = new Model(CONTEXT, modelName); model.store(); } return model } https://gist.github.com/axeda/6529288/raw/5ffca58c3c48256b81287d6a6f2d2db63cd5cd2b/AddGeofence.groovy

May 10, 2016

The Axeda Platform provides a few mechanisms for putting user-defined pages or UI modules into the dashboards, or allowing end-users to host AJAX based applications from the same instance their data is retrieved from. This simple application illustrates the use of jQuery to call Scripto and return a JSON formatted array of current data for an Axeda asset. Prerequisites: First steps taken with Axeda Artisan Basic understanding of HTML, JavaScript and jQuery Axeda Platform v6.5 or greater (Axeda Customers and Partners) Artisan project attached to this article Features: Authentication from a Web app Use of CurrentDataFinder API Scripto from jQuery Files of Note (Locations are from the root of Artisan project) index.html – main HTML index page ..\artisan-starter-html\src\main\webapp\index.html app.js – JavaScript code to build application and call Scripto ..\artisan-starter-html\src\main\webapp\scripts\app.js axeda.js – axeda web services JavaScript code ..\artisan-starter-html\src\main\webapp\scripts\axeda.js DataItemsWithScripto.groovy – custom object on Axeda platform ..\artisan-starter-scripts\src\main\groovy\DataItemsWithScripto.groovy Screenshots: Further Reading Developing with Axeda Artisan Extending the Axeda Platform UI - Custom Tabs and Modules

May 10, 2016

Javascript, everyone knows it, at least a little bit. What if I told you that you could do serious data acquisition with just a little bit of Javascript and you may already have the tools to do it, right now on your "Off the Shelf" device. Node.js is a command line implementation of Javascript that can be run on common, credit card sized devices like the Raspberry PI or the Intel Edison. I suspect that if you already know about Node.js, you may have encountered its non-blocking asynchronous, "Call back", style of programming which can be a little different that most other languages which block or wait for commands to complete. While this can be a benefit for increasing performance, it can also be a barrier to entry for new users. This is the problem that Node Red really solves. Node Red is a web based Integrated Development Environment (IDE) that turns the "Call Back" style Javascript programming of Node.js into a series of interconnected Nodes, each Node of which represents a Javascript function which is connected by a callback to another node/function. A simple hello world program in Node Red would look something like this ( with annotations in red) : You can re-create this program using the Node Red IDE yourself. Here is a brief video (with no sound) which should familiarize you with how to create your own hello world flow. Video Link : 1333 How can you install Node Red on your own system to try it out? The good news is, if you have a Raspberry PI 2 with a NOOBS installed on it, Node.js and Node Red come pre-installed. If you do not already have it installed, or want to install it on your own system it is still pretty simple. Here are the steps: 1. Download and install Node.js (https://nodejs.org/en/download/) 2. Run the command: sudo npm install -g --unsafe-perm node-red Omit the sudo on windows (see http://nodered.org/docs/getting-started/installation.html for more info) 3. You now have Node Red. To run it, just type: node-red on your command line. 4. Using your web browser goto http://localhost:1880 and the Node Red IDE will appear in your browser. How about a real hardware integration example? Node Red comes with many built in Nodes and many more nodes you can add to connect to specific peripherals you may have on your device. Rather than provide a complete tutorial on Node Red, I will focus on discussing using this IDE to re-create a hardware integration that I created in the past using the Java SDK, The Raspberry PI, AM2302 Weather Station (see Weather Applications with Raspberry Pi | ThingWorx). This example contains detailed specifics on the attachment of the AM2302 Temperature/Humidity sensor to your Raspberry PI. I am going to assume you have the hardware already attached to your Raspberry PI as described in this tutorial ( https://learn.adafruit.com/dht-humidity-sensing-on-raspberry-pi-with-gdocs-logging/overview ). I am also assuming that you have installed the python based sample program described in this tutorial as well and you now have a python script called "AdafruitDHT.py" installed on your PI that produces the following output when it is run. pi@raspberrypi:~/projects/Adafruit_Python_DHT/examples $ sudo ./AdafruitDHT.py 2302 4 Temp=22.3* Humidity=30.6% pi@raspberrypi:~/projects/Adafruit_Python_DHT/examples $ If you don't have any of this hardware installed, you can still proceed with this example and just create your own temperature and humidity values manually. We are going to connect the output of this python script directly to ThingWorx and sample its output value every 5 seconds. I will start assuming you do not have the Am2302 hardware and create simulated values. I will then replace them with the actual output of the python script as a final step. Polling versus Interrupt Driven Data Collection In the Java SDK version of this example, we are polling for changes in data. Every so many seconds our device will wake up and take a reading. How do we recreate the same effect in Node Red without having to push an inject button every 5 seconds. No. We need an input node that activates on its own every 5 seconds. The Inject Node will do this. Drag out an inject node and configure it as shown below. This is an input node so it will be starting a new flow. It will fire off every 5 seconds from the minute this sheet is deployed. Simulate Data Collection Lets generate a random humidity and temperature value before getting the actual data. For this node we will use a Function node. Drag one out and configure it as shown below. Here is the Javascript for this node so you can cut and paste it into this dialog. var tempF = Math.random() * 40 + 60; var tempC = (tempF-32)/1.8; var humidity = Math.random() * 80 + 20; msg.payload = { "tempF":tempF, "tempC":tempC, "humidity":humidity }; return msg; Remember that the returned message is the message that the next node will receive. The payload property is the standard or default property of a message that most nodes use to pass data between each other. Here, our payload is an object with all of our simulated data in it. Lets Test it Out Connect the two nodes together and add a debug output node and deploy your sheet. The completed flow will look like this. As soon as you deploy you should see the following output in your debug tab and every five seconds another data sample will be generated. So how does this data get to ThingWorx? What we need to do is take this data and deliver it to ThingWorx in the form of a REST web service call. This is easier to do than it sounds. First off, lets create a Thing on your ThingWorx server that looks like this. Now give it these properties. Next, create an Application Key in the application keys section of the composer. Assign it to the "Administrator" user. Your keyId will of course be different. This key will be the credential you need to post your data. Installing the ThingRest Node Red Node To simplify the process of posting the data to ThingWorx, I have created my own custom node to post data. To install a custom node into your Node Red installation you have to find the directory Node Red is using to store your sheets in. By default this is a directory called ".node-red" in your home directory. On a Raspberry PI this directory would be /home/pi/.node-red. If you are running Node Red now, quit it by hitting control-c and cd into the .node-red directory. Below is the sequence of commands you would issue on your PI to install the ThingRest node. cd ~/.node-red npm install git+https://git@github.com/obiwan314/node-red-node-thingrest.git node-red The node package manager (npm) will install this new node automatically into your .node-red directory. Now re-run node-red and go back to your browser and refresh your Node Red IDE. You should now have a "REST Thing" node. Adding a REST Thing node to your flow Drag a REST Thing output node into your flow and configure it as shown below. Remember, your Application Key will be different than the one shown here. Also, your ThingWorx server URL may be different if your server is not on the same machine you are working on. Now connect it as shown below. When you deploy this sheet, you will be posting data to ThingWorx. Go back to your WeatherStation1 Thing in ThingWorx and use the Refresh button shown below to see your data changing. Wait, that is? Thats the whole data collection program? Yes. The flow above is the equivalent of the Java SDK code from the Java weather station example. Now for Some Real Data As promised, we will now replace the simulated data in the Generate Data node with real data obtained from the "~/projects/Adafruit_Python_DHT/examples/AdafruitDHT.py 2302 4" python command on your Raspberry PI using an Exec node. The exec node can be found at the very bottom of your node palette. It executes a command and returns the results as msg.payload to the next node in the flow. You may have noticed it has three outputs instead of one. In order these outputs are your Standard output, Standard Error and the integer return code of the process. Use the first output node to get the results of this command. Now Connect this in place of the Generate Data Node as shown below. At this point, we can't connect the collected data to the WeatherStation1 Thing because it is in the wrong format. It is console output and we need it in the form of a Javascript object. We are going to need a function to parse the console output into a Javascript object. Add the function node shown below. Here is the Javascript for cut and paste convenience. var temphumidArray=msg.payload.split(" "); var tempC = parseFloat(temphumidArray[0].replace("*","").split('=')[1]); var tempF = tempC *1.8 + 32; var humidity = parseFloat(temphumidArray[2].replace("%","").replace("\n","").split('=')[1]); msg.payload = { "humidity":humidity, "tempF":tempF, "tempC":tempC }; return msg; Now msg.payload contains a javascript object identical to the one we were generating at random but now it is using real data. Connect up your nodes so they appear as shown below but when you deploy, don't expect it to work yet because there is still one problem you will have to get around. This python script expects to be run as the root user. How to run Node Red as Root You can start Node Red as root with the following command sudo node-red -u /home/pi/.node-red Note that the -u argument is required to make sure you keep using the pi user's .node-red directory. If you loose your REST Thing node, you are not using the pi user's .node-red directory, but root's instead. If you see any error messages in your debug window, try re-attaching the the debug node to the Collect Data node and see what is being produced by the exec node. Don't forget to verify that your tempC,tempF and humidity properties are updating in ThingWorx. Lets Add a GPS Location You may have noticed that there is a stationLocation property on the WeatherStation1 Thing. Lets set that to a fixed location to complete this example of 40.0568764,-75.6720953,18. Below is the modified Javascript to update in the Parse Data node to add this location. var temphumidArray=msg.payload.split(" "); var tempC = parseFloat(temphumidArray[0].replace("*","").split('=')[1]); var tempF = tempC *1.8 + 32; var humidity = parseFloat(temphumidArray[2].replace("%","").replace("\n","").split('=')[1]); msg.payload = { "humidity":humidity, "tempF":tempF, "tempC":tempC, "stationLocation":"40.0568764,-75.6720953,18" }; return msg; What's Next? Node Red has many more nodes that you can add to your project through the use of the npm command. There is a GPIO node library you can install at https://github.com/monteslu/node-red-contrib-gpio which will give you input and output nodes for the GPIO pins on your PI as well, This library also supports accessing Arduino's attached to the PI over a USB cable which expand the possibilities for data collection and peripheral control.Hopefully this article has exposed you to the many other possibilities for connecting devices to your ThingWorx Server. The Rest Thing node is using the HTTP REST protocol to talk to ThingWorx. In the near future, with the Introduction of the ThingWorx Javascript SDK, a Node Red library can be created that uses ThingWorx AlwaysOn WebSockets protocol to communicate with your ThingWorx server which will offer even more capabilities and better performance.

Mar 4, 2016

In the recent times, one of the frequent questions regarding PostgreSQL is which tools are good with PostgreSQL. With the growing functionality of PostgreSQL, the number of vendors are willing to produce tools for PostgreSQL. There are lot of tools for management, development, data visualization and the list if growing. Here, I'm listing a few tools that might be of interest to Thingworx users. psql terminal: The psql client is a command-line client distributed with PostgreSQL, often called as interactive terminal. psql is a simple yet powerful tool with which you can directly interface with the PostgreSQL server. The psql client comes default with the PostgreSQL database. Key features: Issue queries either through commands or from a file. Provides shell-like features to automate tasks. For more information, refer http://www.postgresql.org/docs/9.5/static/app-psql.html pgAdmin III: pgAdmin III is a GUI based administration and development tool for PostgreSQL database. It delivers the needs of both admin and normal users from writing simple SQL queries to developing complex databases. Key features: Open source and cross-platform support. No additional drivers are required. Supports more than 30 different languages. Note: pgAdmin III comes default with postgreSQL9.4 installer. For more information, refer http://www.pgadmin.org/download/ phpPgAdmin: phpPgAdmin is a web-based client for managing PostgreSQL databases. It provides the user with a convenient way to create databases, create tables, alter tables and query the data using SQL. Key features: Open source and supports PostgreSQL 9.x. Requires webserver. Administer multiple servers. Supports the slony master-slave replication engine. For phpPgAdmin download: http://phppgadmin.sourceforge.net/doku.php?id=download TeamPostgreSQL: TeamPostgreSQL is a browser-based tool for PostgreSQL administration. Using TeamPostgreSQL, database objects can be accessed from anywhere in the web browser. Key features: Open source and cross-platform support. Supports SSH for both the web interface and the database connections. GUI with tabbed SQL editors. For TeamPostgreSQL download: http://www.teampostgresql.com/download.jsp Monitoring Tools pgBadger: pgBadger is a PostgreSQL log analyzer for generating reports from the PostgreSQL log files. It is built in Perl language and uses a javascript and bootstrap libraries. Often seen as a replacement for pgfouine log analyzer. Key features: Open source community project. Autodetects postgreSQL log file formats (stderr, syslog or csvlog). Provides SQL queries related reports and statistics. Can also set limits to only report errors. Generates Pie charts and Time based charts. For more information, refer http://dalibo.github.io/pgbadger/. Git download: https://github.com/dalibo/pgbadger/releases PostgreStats: Postgrestats is a software that has automated scripts to easily view statistics such as commits, rollbacks, user inserts, updates and deletes in a time-based intervals. Postgrestats gets installed and executes on the database server, it customizes the main conf file. Postgrestats also provides an enterprise application for Replication mode and High Availability. Key features: Open source and easy-to-setup installation. Take a snapshot report based on time intervals. Optional email-on-update. Text file Data storage. Also provides enterprise application, PostgreStats Enterprise. For more information, refer: http://www.postgrestats.com/subs/docs.html Slemma: Slemma is a collaborative, data visualization tool for PostgreSQL database. Slemma allows database connections with a near to one-click integration and can generate a dashboard from files. Slemma comes with a commercial license with a $29 per user per month pricing. Key features: Create charts and interactive dashboards by selecting tables. Non-developers can easily create visualizations (with no coding). Email dashboards automatically to clients or your entire team. For more information, refer https://slemma.com/ Ubiq: Ubiq is a web-based buisness intelligence and reporting tool for PostgreSQL server. Ubiq creates reports and online dashboards, providing the feature to export in multiple formats. Ubiq is distributed with a commercial license. Key features: Drag & drop interface to create interactive charts, dashboards and reports. Apply powerful filters and functions to the data. Share your work and schedule email reports. For more information, refer http://ubiq.co/tour

Jan 8, 2016

Applicable Releases: ThingWorx Platform 7.0 to 8.5 Description: Covers how to apply patch upgrades to ThingWorx installation, with the following agenda: How to read ThingWorx version Upgrading to a major/minor version of the platform Focus on upgrading to a patch version of the platform Upgrading extensions Always check the patch release notes for additional information and specific steps

Jul 24, 2020

Oct 31, 2015

Hiya, I recently prepared a short demo which shows how to onboard and use Azure IoT devices in ThingWorx and added some usability tips and tricks to help others who might struggle with some of the things that I did. The good news... I recorded and posted it to YouTube here. •Connect Azure IoT Hub with ThingWorx (to be updated soon for 9.0 release) •Using the Azure IoT Dev Kit with ThingWorx •Getting the Azure IoT Hub Connector Up and Running (V3/8.5) Enjoy, and don't hesitate to comment with your own tips and feedback. Cheers, Greg

Jun 15, 2020

ThingWorx is great for storing large amounts of data coming from your devices but it can also be used like a traditional, row based database for information you would like to integrate with your thing data. Attached to this blog entry is a short example of creating an address book database using a DataTable and a DataShape. It does not focus on creating mashups but sticks with discussing the modeling and service calls you would use to create a simple database.

Oct 31, 2015

May 5, 2020

Axeda Machine Streams enables external Platform integrators to access the current, raw data from connected assets. The Platform can stream the data item, alarm, mobile location, and registration messages from connected assets to an ActiveMQ server or Azure Service Bus endpoint. Streamed data can be used for data analytics or reporting, or simply for storage. This article explains the Machine Streams Data Relay project that Axeda provides. This sample project illustrates how stream consumers can create their own projects to relay Machine Stream messages from ActiveMQ or Azure Service Bus into their environments. The Machine Streams Data Relay project was created using Apache Maven. The project operates by dispatching messages to a log message processor. Each machine streams message is logged to stdout. Note: The "Axeda Features Guide" provides a high level introduction to the Axeda Machine Streams feature. That PDF is available from PTC Support (http://support.ptc.com/).) Downloading and Installing the Project The machine-streams-data-relay project is provided as a tar.gz archive for Linux users and a .zip archive for Windows users. Each archive includes a Maven project with all source code. This page provides downloads and full source for the machine-steams-data-relay Maven project. The Data Relay project files are available from here. Prerequisites To download, build, and compile the machine-streams-data-relay project, you will need the following: Access to an Axeda Platform instance configured to stream asset data (for ActiveMQ endpoint this includes the Axeda provided ActiveMQ machine-streams plugin/overlay (axeda-jms-plugin-r<SVN_REVISION>-machine-streams.zip, which is provided here. ActiveMQ or Azure Service Bus server configured for Machine Streams. Instructions for configuring an ActiveMQ or Azure Service Bus server for Machine Streams are provided in the “Axeda® Machine Streams: A Guide to Setting Up Broker Endpoints", available with all documentation from PTC Support (http://support.ptc.com/). At least one machine stream (Axeda Artisan Machine Streams Archetype) configured to stream data to the ActiveMQ or Azure Service Bus server for your assets. (Complete information about creating machine streams and adding machine stream support to the Axeda Platform is provided in the “Axeda v2 API/Services Developers Reference Guide” available from PTC Support (http://support.ptc.com/).) Access to the ActiveMQ or Azure Service Bus server configured as the endpoint for streamed Machine Streams content Oracle Java JDK 1.7 or greater and java and javac installed and available in your PATH (if you need instructions for this, see http://www.oracle.com/technetwork/java/javase/downloads/index.html) Maven 3.0.4 or greater and mvn installed and available in your PATH (if you need instructions for this, see http://maven.apache.org/download.cgi) Note: For the Machine Streams Data Relay project to work successfully, the Axeda Platform instance and the ActiveMQ or Azure Service Bus server instance must be configured with support for Axeda Machine Streams, and at least one machine stream must be configured to stream data. Complete information about configuring Axeda Platform for Axeda Machine Streams, including the data format for the resulting streams (XML or JSON) is available in the “Axeda v2 API/Services Developers Reference Guide.” Instructions for configuring an ActiveMQ or Azure Service Bus server for Machine Streams are provided in the “Axeda® Machine Streams: A Guide to Setting Up Broker Endpoints” Reference Guide (available from PTC Support (http://support.ptc.com/)). Building the Project This page provides instructions for building the Data Relay project for Linux and for Windows environments. 1. Download and uncompress the project for your environment Linux: Click here for the machine-streams-data-relay-1.0.3-project.tar.gz # tar -zxvf machine-streams-data-relay-1.0.3-project.tar.gz # cd machine-streams-data-relay-1.0.3 Windows: Click here for the machine-streams-data-relay-1.0.3-project.zip Unzip the project to the following directory: C:\machine-streams-data-relay-1.0.3 2. Edit the ActiveMQ or Azure Service Bus configuration file (configAMQ.properties or configASB.properties) in src\main\scripts\ as needed. sample Config.properties files for the MachineStreamsDataRelay component For ActiveMQ broker endpoints - configAMQ.properties # The ActiveMQ broker URL. brokerURL=tcp://localhost:62000 # The ActiveMQ queue name to process messages from. # It can be a single queue: MachineStream.stream01 # Or a wildcard queue: MachineStream.> queueName=MachineStream.> # The username used to connect to the ActiveMQ queue username=axedaadmin # The password used to connect to the ActiveMQ queue password=zQXuLzhQgcyRZ25JCDXYEPBCT2kx48 # The number of ActiveMQ broker connections. numConnections=10 # The number of sessions per connection. Note that each session will create a separate thread. numSessionsPerConnection=5 # The number of concurrent threads used for processing machine streams messages. numProcessingThreads=100 # The type of message listener container. # default = single queue name per connection. # multiDestination = supports multiple queue names per connection messageListenerContainerType=default For Azure Service Bus broker endpoints - configASB.properties # The ASB broker URL. brokerURL=amqps://your-azure-service-bus-namespace.servicebus.windows.net # The ASB queues to process messages from. # It can be a single queue: MachineStream.stream01 # Or multiple queues separated by a comma: MachineStream.stream01,MachineStream.stream02 # Or a queue range defined by the following syntax: MachineStream.stream[01-20] queueName=MachineStream.stream[01-50] # The username used to connect to the ASB queue(s) username=your-azure-service-bus-username # The password used to connect to the ASB queue(s) password=the-password-for-your-azure-service-bus-username # The max number of ASB broker connections. numConnections=10 # The number of concurrent threads used for processing machine streams messages. numProcessingThreads=100 # The type of message listener container. # default = single queue name per connection. # multiDestination = supports multiple queue names per connection messageListenerContainerType=multiDestination Note: messageListenerContainerType is provided because Azure Service Bus does not support wildcard queue names. The configuration details are as follows: Name Description Value brokerURL location of the ActiveMQ or Azure Service Bus (broker) location of the ActiveMQ or Azure Service Bus server (broker) queueName Name of the ActiveMQ or Azure Service Bus queue from which you want to process messages To define a single queue: MachineStream.<insert single queue name here> To define a wildcard queue name for multiple queues:MachineStream. It can be a single queue: MachineStream.stream01 Or multiple queues separated by a comma: MachineStream.stream01,MachineStream.stream02 Or a queue range defined by the following syntax: MachineStream.stream[01-20]: queueName=MachineStream.stream[01-50] (if you have multiple queues and you want to use ASB, then you have to use multiDestination and use the range) username username used to connect to the ActiveMQ or Azure Service Bus queue For ActiveMQ: username=axedaadmin For ASB: username=your-azure-service-bus-username password used to connect to the ActiveMQ or Azure Service Bus queue password used to connect to the ActiveMQ or ASB queue(s) numConnections number of ActiveMQ or Azure Service Bus broker connections Default is 10 broker connections numSessionsPerConnection The number of sessions per connection. Note that each session will create a separate thread. (This key is used infrequently.) APPLICABLE TO ACTIVEMQ ONLY. Default is 5 sessions per connection APPLICABLE TO ACTIVEMQ ONLY. numProcessingThreads The number of concurrent threads used for processing machine streams messages. Default is 100 concurrent threads messageListenerContainerType The type of message listener container. Default is single queue name per connection. Supports multiple queue names per connection 3. Build code using Maven. Use -DskipTests option if you want to skip tests. This will build all source code and produce a bin archive in the target directory. For Linux: # mvn package -DskipTests For Windows: c:\> mvn package -DskipTests 4. Enter the target directory and uncompress *bin.tar.gz archive and enter correct directory For Linux: # cd target # tar -zxvf machine-streams-data-relay-1.0.3-bin.tar.gz # cd machine-streams-data-relay-1.0.3 For Windows: c:\> cd target c:\> unzip machine-streams-data-relay-1.0.3.bin.zip c:\> cd machine-streams-data-relay-1.0.3 5. Start the application. For Linux: # ./machineStreamsDataRelay.sh <config properties file> for example: e.g. ./machineStreamsDataRelay.sh configOfYourChoice.properties For Windows: # ./machineStreamsDataRelay.sh <config properties file> for example: e.g. ./machineStreamDataRelay.bat configOfYourChoice.properties See the two example config files included within the project: configASB.properties (for Azure Service Bus) and configAMQ.properties (for ActiveMQ). 6. Scan the output. If your ActiveMQ configuration is correct, output similar to the following should appear, and no ERRORS should be shown: 2014-03-26 10:27:06.179 [main] INFO [MessageListenerServiceImpl]: Initializing connections to tcp://localhost:62000 username=axedaadmin 2014-03-26 10:27:06.346 [main] INFO [MessageListenerServiceImpl]: Initialized connection 1: queue=MachineStream.> numSessions=5 2014-03-26 10:27:06.351 [main] INFO [MessageListenerServiceImpl]: Initialized connection 2: queue=MachineStream.> numSessions=5 2014-03-26 10:27:06.356 [main] INFO [MessageListenerServiceImpl]: Initialized connection 3: queue=MachineStream.> numSessions=5 2014-03-26 10:27:06.365 [main] INFO [MessageListenerServiceImpl]: Initialized connection 4: queue=MachineStream.> numSessions=5 2014-03-26 10:27:06.369 [main] INFO [MessageListenerServiceImpl]: Initialized connection 5: queue=MachineStream.> numSessions=5 2014-03-26 10:27:06.381 [main] INFO [MessageListenerServiceImpl]: Initialized connection 6: queue=MachineStream.> numSessions=5 2014-03-26 10:27:06.388 [main] INFO [MessageListenerServiceImpl]: Initialized connection 7: queue=MachineStream.> numSessions=5 2014-03-26 10:27:06.402 [main] INFO [MessageListenerServiceImpl]: Initialized connection 8: queue=MachineStream.> numSessions=5 2014-03-26 10:27:06.411 [main] INFO [MessageListenerServiceImpl]: Initialized connection 9: queue=MachineStream.> numSessions=5 2014-03-26 10:27:06.416 [main] INFO [MessageListenerServiceImpl]: Initialized connection 10: queue=MachineStream.> numSessions=5 If your Azure Service Bus configuration is correct, output similar to the following should appear, and no ERRORS should be shown: 2014-10-01 16:51:30.114 [main] INFO [MessageListenerServiceImpl]: Initializing Connections to amqps://acme.servicebus.windows.net username=owner 2014-10-01 16:51:31.613 [ConnectionRecovery-thread-6] INFO [MultiDestinationMessageListenerContainer]: Connection 6 created 0/10 queue consumers 2014-10-01 16:51:31.614 [ConnectionRecovery-thread-8] INFO [MultiDestinationMessageListenerContainer]: Connection 8 created 0/10 queue consumers 2014-10-01 16:51:31.614 [ConnectionRecovery-thread-10] INFO [MultiDestinationMessageListenerContainer]: Connection 10 created 0/9 queue consumers 2014-10-01 16:51:31.614 [ConnectionRecovery-thread-2] INFO [MultiDestinationMessageListenerContainer]: Connection 2 created 0/10 queue consumers 2014-10-01 16:51:31.614 [ConnectionRecovery-thread-3] INFO [MultiDestinationMessageListenerContainer]: Connection 3 created 0/10 queue consumers 2014-10-01 16:51:31.614 [ConnectionRecovery-thread-5] INFO [MultiDestinationMessageListenerContainer]: Connection 5 created 0/10 queue consumers 2014-10-01 16:51:31.615 [ConnectionRecovery-thread-9] INFO [MultiDestinationMessageListenerContainer]: Connection 9 created 0/10 queue consumers 2014-10-01 16:51:31.615 [ConnectionRecovery-thread-4] INFO [MultiDestinationMessageListenerContainer]: Connection 4 created 0/10 queue consumers 2014-10-01 16:51:31.621 [ConnectionRecovery-thread-7] INFO [MultiDestinationMessageListenerContainer]: Connection 7 created 0/10 queue consumers 2014-10-01 16:51:31.756 [ConnectionRecovery-thread-1] INFO [MultiDestinationMessageListenerContainer]: Connection 1 created 0/10 queue consumers 2014-10-01 16:51:32.613 [ConnectionRecovery-thread-6] INFO [MultiDestinationMessageListenerContainer]: Connection 6 created 9/10 queue consumers 2014-10-01 16:51:32.614 [ConnectionRecovery-thread-8] INFO [MultiDestinationMessageListenerContainer]: Connection 8 created 9/10 queue consumers 2014-10-01 16:51:32.614 [ConnectionRecovery-thread-10] INFO [MultiDestinationMessageListenerContainer]: Connection 10 created 7/9 queue consumers 2014-10-01 16:51:32.614 [ConnectionRecovery-thread-2] INFO [MultiDestinationMessageListenerContainer]: Connection 2 created 10/10 queue consumers 2014-10-01 16:51:32.615 [ConnectionRecovery-thread-3] INFO [MultiDestinationMessageListenerContainer]: Connection 3 created 9/10 queue consumers 2014-10-01 16:51:32.615 [ConnectionRecovery-thread-5] INFO [MultiDestinationMessageListenerContainer]: Connection 5 created 0/10 queue consumers 2014-10-01 16:51:32.615 [ConnectionRecovery-thread-9] INFO [MultiDestinationMessageListenerContainer]: Connection 9 created 7/10 queue consumers 2014-10-01 16:51:32.615 [ConnectionRecovery-thread-4] INFO [MultiDestinationMessageListenerContainer]: Connection 4 created 9/10 queue consumers 2014-10-01 16:51:32.623 [ConnectionRecovery-thread-7] INFO [MultiDestinationMessageListenerContainer]: Connection 7 created 9/10 queue consumers 2014-10-01 16:51:32.756 [ConnectionRecovery-thread-1] INFO [MultiDestinationMessageListenerContainer]: Connection 1 created 10/10 queue consumers 2014-10-01 16:51:32.833 [main] INFO [MessageListenerServiceImpl]: Initialized Connection 1: numQueues=10 initTimeMillis=2631 millis 2014-10-01 16:51:32.833 [main] INFO [MessageListenerServiceImpl]: Initialized Connection 2: numQueues=10 initTimeMillis=2488 millis 2014-10-01 16:51:33.613 [ConnectionRecovery-thread-6] INFO [MultiDestinationMessageListenerContainer]: Connection 6 created 10/10 queue consumers 2014-10-01 16:51:33.614 [ConnectionRecovery-thread-8] INFO [MultiDestinationMessageListenerContainer]: Connection 8 created 10/10 queue consumers 2014-10-01 16:51:33.614 [ConnectionRecovery-thread-10] INFO [MultiDestinationMessageListenerContainer]: Connection 10 created 9/9 queue consumers 2014-10-01 16:51:33.615 [ConnectionRecovery-thread-3] INFO [MultiDestinationMessageListenerContainer]: Connection 3 created 9/10 queue consumers 2014-10-01 16:51:33.615 [ConnectionRecovery-thread-5] INFO [MultiDestinationMessageListenerContainer]: Connection 5 created 0/10 queue consumers 2014-10-01 16:51:33.615 [ConnectionRecovery-thread-9] INFO [MultiDestinationMessageListenerContainer]: Connection 9 created 8/10 queue consumers 2014-10-01 16:51:33.615 [ConnectionRecovery-thread-4] INFO [MultiDestinationMessageListenerContainer]: Connection 4 created 9/10 queue consumers 2014-10-01 16:51:33.623 [ConnectionRecovery-thread-7] INFO [MultiDestinationMessageListenerContainer]: Connection 7 created 10/10 queue consumers 2014-10-01 16:51:34.615 [ConnectionRecovery-thread-5] INFO [MultiDestinationMessageListenerContainer]: Connection 5 created 0/10 queue consumers 2014-10-01 16:51:34.615 [ConnectionRecovery-thread-3] INFO [MultiDestinationMessageListenerContainer]: Connection 3 created 9/10 queue consumers 2014-10-01 16:51:34.615 [ConnectionRecovery-thread-9] INFO [MultiDestinationMessageListenerContainer]: Connection 9 created 8/10 queue consumers 2014-10-01 16:51:34.615 [ConnectionRecovery-thread-4] INFO [MultiDestinationMessageListenerContainer]: Connection 4 created 9/10 queue consumers 2014-10-01 16:51:35.615 [ConnectionRecovery-thread-5] INFO [MultiDestinationMessageListenerContainer]: Connection 5 created 9/10 queue consumers 2014-10-01 16:51:35.615 [ConnectionRecovery-thread-3] INFO [MultiDestinationMessageListenerContainer]: Connection 3 created 9/10 queue consumers 2014-10-01 16:51:35.615 [ConnectionRecovery-thread-9] INFO [MultiDestinationMessageListenerContainer]: Connection 9 created 8/10 queue consumers 2014-10-01 16:51:35.616 [ConnectionRecovery-thread-4] INFO [MultiDestinationMessageListenerContainer]: Connection 4 created 9/10 queue consumers 2014-10-01 16:51:36.616 [ConnectionRecovery-thread-5] INFO [MultiDestinationMessageListenerContainer]: Connection 5 created 9/10 queue consumers 2014-10-01 16:51:36.616 [ConnectionRecovery-thread-3] INFO [MultiDestinationMessageListenerContainer]: Connection 3 created 9/10 queue consumers 2014-10-01 16:51:36.616 [ConnectionRecovery-thread-4] INFO [MultiDestinationMessageListenerContainer]: Connection 4 created 9/10 queue consumers 2014-10-01 16:51:36.616 [ConnectionRecovery-thread-9] INFO [MultiDestinationMessageListenerContainer]: Connection 9 created 8/10 queue consumers 2014-10-01 16:51:37.616 [ConnectionRecovery-thread-5] INFO [MultiDestinationMessageListenerContainer]: Connection 5 created 9/10 queue consumers 2014-10-01 16:51:37.616 [ConnectionRecovery-thread-3] INFO [MultiDestinationMessageListenerContainer]: Connection 3 created 9/10 queue consumers 2014-10-01 16:51:37.616 [ConnectionRecovery-thread-4] INFO [MultiDestinationMessageListenerContainer]: Connection 4 created 9/10 queue consumers 2014-10-01 16:51:37.616 [ConnectionRecovery-thread-9] INFO [MultiDestinationMessageListenerContainer]: Connection 9 created 8/10 queue consumers 2014-10-01 16:51:38.616 [ConnectionRecovery-thread-3] INFO [MultiDestinationMessageListenerContainer]: Connection 3 created 10/10 queue consumers 2014-10-01 16:51:38.617 [ConnectionRecovery-thread-9] INFO [MultiDestinationMessageListenerContainer]: Connection 9 created 10/10 queue consumers 2014-10-01 16:51:38.616 [ConnectionRecovery-thread-4] INFO [MultiDestinationMessageListenerContainer]: Connection 4 created 10/10 queue consumers 2014-10-01 16:51:38.616 [ConnectionRecovery-thread-5] INFO [MultiDestinationMessageListenerContainer]: Connection 5 created 10/10 queue consumers 2014-10-01 16:51:38.643 [main] INFO [MessageListenerServiceImpl]: Initialized Connection 3: numQueues=10 initTimeMillis=8491 millis 2014-10-01 16:51:38.643 [main] INFO [MessageListenerServiceImpl]: Initialized Connection 4: numQueues=10 initTimeMillis=8490 millis 2014-10-01 16:51:38.643 [main] INFO [MessageListenerServiceImpl]: Initialized Connection 5: numQueues=10 initTimeMillis=8490 millis 2014-10-01 16:51:38.643 [main] INFO [MessageListenerServiceImpl]: Initialized Connection 6: numQueues=10 initTimeMillis=3485 millis 2014-10-01 16:51:38.643 [main] INFO [MessageListenerServiceImpl]: Initialized Connection 7: numQueues=10 initTimeMillis=3495 millis 2014-10-01 16:51:38.643 [main] INFO [MessageListenerServiceImpl]: Initialized Connection 8: numQueues=10 initTimeMillis=3485 millis 2014-10-01 16:51:38.643 [main] INFO [MessageListenerServiceImpl]: Initialized Connection 9: numQueues=10 initTimeMillis=8488 millis 2014-10-01 16:51:38.643 [main] INFO [MessageListenerServiceImpl]: Initialized Connection 10: numQueues=9 initTimeMillis=3485 millis 7. To verify that messages are being streamed properly from the Axeda Platform, send DataItems from your connected Assets. You should see messages similar to the following. (Remember that each Asset you are testing must have an associated Machine Stream.) 2014-03-26 10:45:16.309 [pool-1-thread-1] INFO [LogMessageProcessor]: StreamedDataItem: Model,Asset1,799021d6-70a3-7c32-0000-00000000021d,false,Wed Mar 26 14:45:16 EDT 2014,temp,43,analog 2014-03-26 10:45:21.137 [pool-1-thread-2] INFO [LogMessageProcessor]: StreamedDataItem: Model,Asset2,799021d6-70a3-7c32-0000-000000000225,false,Wed Mar 26 14:45:21 EDT 2014,temp,43,analog 2014-03-26 10:45:26.134 [pool-1-thread-3] INFO [LogMessageProcessor]: StreamedDataItem: Model,Asset1,799021d6-70a3-7c32-0000-00000000022b,false,Wed Mar 26 14:45:26 EDT 2014,temp,44,analog 2014-03-26 10:45:31.135 [pool-1-thread-4] INFO [LogMessageProcessor]: StreamedDataItem: Model,Asset2,799021d6-70a3-7c32-0000-000000000231,false,Wed Mar 26 14:45:31 EDT 2014,temp,44,analog 2014-03-26 10:45:36.142 [pool-1-thread-5] INFO [LogMessageProcessor]: StreamedDataItem: Model,Asset1,799021d6-70a3-7c32-0000-000000000237,false,Wed Mar 26 14:45:36 EDT 2014,temp,45,analog 2014-03-26 10:45:41.146 [pool-1-thread-6] INFO [LogMessageProcessor]: StreamedDataItem: Model,Asset2,799021d6-70a3-7c32-0000-00000000023d,false,Wed Mar 26 14:45:41 EDT 2014,temp,45,analog Configuring a CustomMessageProcessor By default, the project is configured to use a LogMessageProcessor that logs each streamed message it receives to standard out. The project takes a StreamedMessage in either an XML or JSON format (as configured in the MachineStream SDKv2 object) and decodes the message into a StreamedMessage Java object. LogMessageProcessor.java implements the MessageProcessor interface. Here is the MessageProcessor.java interface: MessageProcessor.java package com.axeda.tools.streams.processor; import com.axeda.tools.streams.model.StreamedMessage; /** * This class defines the message processor method callback that will called for message processing. * Note that this methods will be called by multiple threads concurrently. */ public interface MessageProcessor { /** * Process a machine stream message. Note that this method will be called by multiple threads concurrently. * The number of concurrent processing threads is defined in MachineStreamsConfig.getNumProcessingThreads(). * If you add code here that significantly slows down message processing, then there is the potential that * MessageListenerService threads will also block. When the MessageListenerService threads block, this means that * messages will start to backup in the ActiveMQ or ASB message queues. If you are processing a large number of messages, * then you may need to adjust your configuration parameters or optimize your processMessage() code. * @param message machine streams message to process */public void processMessage(StreamedMessage message);} An additional class named CustomMessageProcessor.java has been provided so that you can provide your own custom message processing logic: CustomMessageProcessor.java package com.axeda.tools.streams.processor; import org.springframework.stereotype.Component; import com.axeda.tools.streams.model.StreamedAlarm; import com.axeda.tools.streams.model.StreamedDataItemMessage; import com.axeda.tools.streams.model.StreamedMessage; import com.axeda.tools.streams.model.StreamedMobileLocation; import com.axeda.tools.streams.model.StreamedRegistrationMessage; /** * This class was provided for customers to implement their own message processing business * logic. To use this class, change the @Autowired messageProcessor qualifier in * MessageProcessingServiceImpl.java to @Qualifier("customMessageProcessor") */ @Component("customMessageProcessor") public class CustomMessageProcessor implements MessageProcessor { /** * (non-Javadoc) * @see com.axeda.tools.streams.processor.MessageProcessor#processMessage(com.axeda.tools.streams.model.StreamedMessage) * * Process a machine stream message. Note that this method will be called by multiple threads * concurrently. The number of concurrent processing threads is defined in * MachineStreamsConfig.getNumProcessingThreads(). * If you add code here that significantly slows down message processing, then there is the * potential that MessageListenerService threads will also block. When the MessageListenerService * threads block, this means that messages will start to backup in the ActiveMQ or Azure Service Bus message queues. If you * are processing a large number of messages, then you may need to adjust your configuration parameters * or optimize your processMessage() code. */ @SuppressWarnings("unused") @Override public void processMessage(StreamedMessage message) { if (message instanceof StreamedDataItemMessage) { StreamedDataItemMessage dataItem = (StreamedDataItemMessage) message; // add your business logic here } else if (message instanceof StreamedAlarm) { StreamedAlarm alarm = (StreamedAlarm) message; // add your business logic here } else if (message instanceof StreamedMobileLocation) { StreamedMobileLocation mobileLocation = (StreamedMobileLocation) message; // add your business logic here } else if (message instanceof StreamedRegistrationMessage) { StreamedRegistrationMessage registration = (StreamedRegistrationMessage)message; // add your business logic here } } } The Axeda Platform Machine Streams feature currently support 4 different message types: StreamedDataItemMessage StreamedAlarm StreamedMobileLocation StreamedRegistrationMessage For each of the different message types, you should add your message processing business logic. You may want to write each message to your favorite NoSql database or to a flat file. Once you have completed your changes to the CustomObjectMessageProcessor, then you must make one change in the MessageProcessingServiceImpl.java class to use this Spring bean. Uncomment this line // @Qualifier("customMessageProcessor") Comment this line @Qualifier("logMessageProcessor") The following code snppet shows what your changes should look like when you are finished: MessageProcessingServiceImpl.java @Component("messageProcessingService") public class MessageProcessingServiceImpl implements MessageProcessingService private static final Logger LOGGER = LoggerFactory.getLogger(MessageProcessingServiceImpl.class); @Autowired private MessageDecoder messageDecoder; @Autowired // If you want to use the CustomMessageProcessor instead of the default LogMessageProcessor then change this Qualifier to @Qualifier("customMessageProcessor") //@Qualifier("logMessageProcessor") private MessageProcessor messageProcessor; private ExecutorService executorService;

Oct 15, 2015

IoT Tips

Axeda Connected Configuration Items Create/Read/Update

Example: Downsampling timeseries data for mashups using LTTB

Calling a Groovy script as an External Web Service

Axeda: Applying Timeouts when calling Web Services and FTP from Custom Objects

Audit Entry to CSV

How to use ThingWorx REST API with Microsoft Power BI

All Data Items to XML

Advanced Artisan APC-Metadata.xml File

ActiveMQ Java Sample Consumer

Analyzing External and ThingWorx Mashup Service Execution Using Tomcat Access Logs

Accessing the Body of a Post Request in a Groovy Script

Axeda: All about Geofences

Axeda Sample Application: Populating A Web Page with Data Items

Drag and Drop Edge Device Development with ThingWorx, Node.js and Node Red

PostgreSQL Tool Triage

Expert Session: ThingWorx Patch Upgrade

DeliveringArduinoDataToThingworx.pdf

Azure IoT MXChip Demo with ThingWorx Azure IoT Hub Connector (V3)

Creating a Simple Address Book Database in ThingWorx

Cloud Data Transfer Cost Calculator

Axeda Machine Streams: Data Relay Project (Axeda Platform v6.8 and later)

ThingWorx Learning Paths

Getting Started on the ThingWorx Platform Learning Path