IoT Tips

In ThingWorx Analytics, you have the possibility to use an external model for scoring. In this written tutorial, I would like to provide an overview of how you can use a model developed in Python, using the scikit-learn library in ThingWorx Analytics. The provided attachment contains an archive with the following files: iris_data.csv: A dataset for pattern recognition that has a categorical goal. You can click here to read more about this dataset TestRFToPmml.ipynb: A Jupyter notebook file with the source code for the Python model as well as the steps to export it to PMML RF_Iris.pmml: The PMML file with the model that you can directly upload in Analytics without going through the steps of training the model in Python The tutorial assumes you already have some knowledge of ThingWorx and ThingWorx Analytics. Also, if you plan to run the Python code and train the model yourself, you need to have Jupyter notebook installed (I used the one from the Anaconda distribution). For demonstration purposes, I have created a very simple random forest model in Python. To convert the model to PMML, I have used the sklearn2pmml library. Because ThingWorx Analytics supports PMML format 4.3, you need to install sklearn2pmml version 0.56.2 (the highest version that supports PMML 4.3). To read more about this library, please click here Furthermore, to use your model with the older version of the sklearn2pmml, I have installed scikit-learn version 0.23.2.  You will find the commands to install the two libraries in the first two cells of the notebook.   Code Walkthrough The first step is to import the required libraries (please note that pandas library is also required to transform the .csv to a Dataframe object):   import pandas from sklearn.ensemble import RandomForestClassifier from sklearn2pmml import sklearn2pmml from sklearn.model_selection import GridSearchCV from sklearn2pmml.pipeline import PMMLPipeline   After importing the required libraries, we convert the iris_data.csv to a pandas dataframe and then create the features (X) as well as the goal (Y) vectors:   iris_df = pandas.read_csv("iris_data.csv") iris_X = iris_df[iris_df.columns.difference(["class"])] iris_y = iris_df["class"]   To best tune the random forest, we will use the GridSearchCSV and cross-validation. We want to test what parameters have the best validation metrics and for this, we will use a utility function that will print the results:   def print_results(results): print('BEST PARAMS: {}\n'.format(results.best_params_)) means = results.cv_results_['mean_test_score'] stds = results.cv_results_['std_test_score'] for mean, std, params in zip(means, stds, results.cv_results_['params']): print('{} (+/-{}) for {}'.format(round(mean, 3), round(std * 2, 3), params))   We create the random forest model and train it with different numbers of estimators and maximum depth. We will then call the previous function to compare the results for the different parameters:   rf = RandomForestClassifier() parameters = { 'n_estimators': [5, 50, 250], 'max_depth': [2, 4, 8, 16, 32, None] } cv = GridSearchCV(rf, parameters, cv=5) cv.fit(iris_X, iris_y) print_results(cv)   To convert the model to a PMML file, we need to create a PMMLPipeline object, in which we pass the RandomForestClassifier with the tuning parameters we identified in the previous step (please note that in your case, the parameters can be different than in my example). You can check the sklearn2pmml  documentation  to see other examples for creating this PMMLPipeline object :   pipeline = PMMLPipeline([ ("classifier", RandomForestClassifier(max_depth=4,n_estimators=5)) ]) pipeline.fit(iris_X, iris_y)   Then we perform the export:   sklearn2pmml(pipeline, "RF_Iris.pmml", with_repr = True)   The model has now been exported as a PMML file in the same folder as the Jupyter Notebook file and we can upload it to ThingWorx Analytics.   Uploading and Exploring the PMML in Analytics To upload and use the model for scoring, there are two steps that you need to do: First, the PMML file needs to be uploaded to a ThingWorx File Repository Then, go to your Analytics Results thing (the name should be YourAnalyticsGateway_ResultsThing) and execute the service UploadModelFromRepository. Here you will need to specify the repository name and path for your PMML file, as well as a name for your model (and optionally a description)   If everything goes well, the result of the service will be an id. You can save this id to a separate file because you will use it later on. You can verify the status of this model and if it’s ready to use by executing the service GetDetails:   Assuming you want to use the PMML for scoring, but you were not the one to develop the model, maybe you don’t know what the expected inputs and the output of the model are. There are two services that can help you with this: QueryInputFields – to verify the fields expected as input parameters for a scoring job   QueryOutputFields – to verify the expected output of the model The resultType input parameter can be either MODELS or CLUSTERS, depending on the type of model,    Using the PMML for Scoring With all this information at hand, we are now ready to use this PMML for real-time scoring. In a Thing of your choice, define a service to test out the scoring for the PMML we have just uploaded. Create a new service with an infotable as the output (don’t add a datashape). The input data for scoring will be hardcoded in the service, but you can also add it as service input parameters and pass them via a Mashup or from another source. The script will be as follows:   // Values: INFOTABLE dataShape: "" let datasetRef = DataShapes["AnalyticsDatasetRef"].CreateValues(); // Values: INFOTABLE dataShape: "" let data = DataShapes["IrisData"].CreateValues(); data.AddRow({ sepal_length: 2.7, sepal_width: 3.1, petal_length: 2.1, petal_width: 0.4 }); datasetRef.AddRow({ data: data}); // predictiveScores: INFOTABLE dataShape: "" let result = Things["AnalyticsServer_PredictionThing"].RealtimeScore({ modelUri: "results:/models/" + "97471e07-137a-41bb-9f29-f43f107bf9ca", //replace with your own id datasetRef: datasetRef /* INFOTABLE */, });   Once you execute the service, the output should look like this (as we would have expected, according to the output fields in the PMML model):   As you have seen, it is easy to use a model built in Python in ThingWorx Analytics. Please note that you may use it only for scoring, and the model will not appear in Analytics Builder since you have created it on a different platform. If you have any questions about this brief written tutorial, let me know.

Analytics projects typically involve using the Analytics API rather than the Analytics Builder to accomplish different tasks. The attached documentation provides examples of code snippets that can be used to automate the most common analytics tasks on a project such as: Creating a dataset Training a Model Real time scoring predictive and prescriptive Retrieving the validation metrics for a model Appending additional data to a dataset Retraining the model The documentation also provides examples that are specific to time series datasets. The attached .zip file contains both the document as well as some entities that you need to import in ThingWorx to access the services provided in the examples. 

For a recent project, I was needing to find all of the children in a Network Hierarchy of a particular template type... so I put together a little script that I thought I'd share. Maybe this will be useful to others as well.   In my situation, this script lived in the Location template. This was useful so that I could find all the Sensor Things under any particular node, no matter how deep they are.   For example, given a network like this: Location 1 Sensor 1 Location 1A Sensor 2 Sensor 3 Location 1AA Sensor 4 Location 1B Sensor 5 If you run this service in Location 1, you'll get an InfoTable with these Things: Sensor 1 Sensor 2 Sensor 3 Sensor 4 Sensor 5 From Location 1A: Sensor 2 Sensor 3 Sensor 4 From Location 1AA: Sensor 4 From Location 1B: Sensor 5   For this service, these are the inputs/outputs: Inputs: none Output: InfoTable of type NetworkConnection   // CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(AlertSummary) let result = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape({ infoTableName : "InfoTable", dataShapeName : "NetworkConnection" }); // since the hierarchy could contain locations or sensors, need to recursively loop down to get all the sensors function findChildrenSensors(thingName) { let childrenThings = Networks["Hierarchy_NW"].GetChildConnections({ name: thingName /* STRING */ }); for each (var row in childrenThings.rows) { // row.to has the name of the child Thing if (Things[row.to].IsDerivedFromTemplate({thingTemplateName: "Location_TT"})) { findChildrenSensors(row.to); } else if (Things[row.to].IsDerivedFromTemplate({thingTemplateName: "Sensor_TT"})) { result.AddRow(row); } } } findChildrenSensors(me.name);    

Fresh look at getting started with ThingWorx in a relevant context that outlines the DEVOPS needed to kick-start your programming.     For full-sized viewing, click on the YouTube link in the player controls. Visit the Online Success Guide to access our Expert Session videos at any time as well as additional information about ThingWorx training and services.

Data Model Implementation Guide Part 3   Step 7: Unique Components Thing Templates   All of the shared component groups have been created. The next stage is creating the unique component group of ThingTemplates. Each of the below sections will cover one ThingTemplate, how the final property configuration should look, and any other aspects that should be added. The breakdown for the unique component group ThingTemplates is as follows:   Robotic Arm Properties   The properties for the RoboticArm ThingTemplate are as follows: Name Base Type Aspects Data Change Type TimeSincePickup NUMBER, Min Value: 0 Persistent and Logged ALWAYS Axis1 String Persistent and Logged VALUE Axis2 String Persistent and Logged VALUE Axis3 String Persistent and Logged VALUE ClampPressure NUMBER, Min Value: 0 Persistent and Logged ALWAYS ClampStatus String Persistent and Logged ALWAYS   Your properties should match the below configurations.   Pneumatic Gate Properties   The properties for the PneumaticGate ThingTemplate are as follows: Name Base Type Aspects Data Change Type GateStatus String Persistent and Logged ALWAYS   Your properties should match the below configurations.   Conveyor Belt Properties   The properties for the ConveyorBelt ThingTemplate are as follows: Name Base Type Aspects Data Change Type BeltSpeed INTEGER, Min Value: 0 Persistent and Logged ALWAYS BeltTemp INTEGER, Min Value: 0 Persistent and Logged ALWAYS BeltRPM INTEGER, Min Value: 0 Persistent and Logged ALWAYS   Your properties should match the below configurations.   Quality Control Camera   Properties   The properties for the QualityControlCamera ThingTemplate are as follows: Name Base Type Aspects Data Change Type QualityReading INTEGER, Min Value: 0 Persistent and Logged ALWAYS QualitySettings String Persistent and Logged ALWAYS CurrentImage IMAGE Persistent and Logged ALWAYS   Your properties should match the below configurations.   Event   Create a new Event named BadQuality. Select AlertStatus as the Data Shape. Your Event should match the below configurations:     Step 8: Data Tables and Data Shapes   For the Data Model we created, an individual DataTable would be best utilized for each products, production orders, and maintenance requests. Utilizing DataTables will allow us to store and track all of these items within our application. In order to have DataTables, we will need DataShapes to create the schema that each DataTable will follow. This database creation aspect can be considered a part of the Data Model design or a part of the Database Design. Nevertheless, the question of whether to create DataTables is based on the question of needed real time information or needed static information. Products, production orders, and maintenance requests can be considered static data. Tracking the location of a moving truck can be considered a need for real time data. This scenario calls for using DataTables, but a real time application will often have places where Streams and ValueStreams are utilized (DataShapes will also be needed for Streams and ValueStreams). NOTE: The DataShapes (schemas) shown below are for a simplified example. There are different ways you can create your database setup based on your own needs and wants. DataTable Name DataShape Purpose MaintenanceRequestDataTable MaintenanceRequest Store information about all maintenanced requests created ProductDataTable ProductDataShape Store information about the product line ProductionOrderDataTable ProductionOrderDataShape Store all information about production orders that have been placed   Maintenance Requests DataShape   The maintenance requests DataShape needs to be trackable (unique) and contain helpful information to complete the request. The DataShape fields are as follows: Name Base Type Additional Info Primary Key ID String NONE YES Title String NONE NO Request String NONE NO CompletionDate DATETIME NONE NO   Unless you’ve decided to change things around, your DataShape fields should match the below configurations.   Products DataShape   The product DataShape needs to be trackable (unique) and contain information about the product. The DataShape fields are as follows: Name Base Type Additional Info Primary Key ProductId String NONE YES Product String NONE NO Description String NONE NO Cost NUMBER Minimum: 0 NO   Unless you’ve decided to change things around, your DataShape fields should match the below configurations.   Production Order DataShape   The production order DataShape needs to be trackable (unique), contain information that would allow the operator and manager to know where it is in production, and information to help make decisions. The DataShape fields are as follows: Name Base Type Additional Info Primary Key OrderId String NONE YES Product InfoTable: DataShape: ProductDataShape NONE NO ProductionStage String NONE NO OrderDate DATETIME NONE NO DueDate DATETIME NONE NO   Unless you’ve decided to change things around, your DataShape fields should match the below configurations.     Step 9: SystemConnections Implementation   We have created the ThingTemplates and ThingShapes that will be utilized within our Data Model for creating instances (Things). Before we finish the build out of our Data Model, let's create the Services that will be necessary for the MaintenanceSystem and ProductionOrderSystem Things.    This guide will not cover the JavaScript and business logic aspect of creating an application. When complete with the below sections, see the Summary page for how to create that level of definition.       Maintenance System   This is the system managed by the maintenance user and geared towards their needs.   Properties   The properties for the MaintenanceSystem Thing are as follows:     Name Base Type Aspects Data Change Type  MaintEngineerCredentials  PASSWORD  Persistent  VALUE    Your properties should match the below configurations.         Services    The Services for the MaintenanceSystem Thing are as follows:    Service Name  Parameters  Output Base Type Purpose   GetAllMaintenanceRequests  NONE  InfoTable: MaintenanceRequest  Get all of the maintenance requests filed for the maintenance user.  GetFilteredMaintenanceRequests  String: TitleFilter  InfoTable: MaintenanceRequest  Get a filtered version of all maintenance requests filed for the maintenance user.  UpdateMaintenanceRequests  String: RequestTitle  NOTHING  Update a maintenance request already in the system.    Use the same method for creating Services that were provided in earlier sections. Your Services list should match the below configurations.     Production Order System   This is the system utilized by the operator and product manager users and geared towards their needs.   Services   The Services for the ProductionOrderSystem Thing are as follows:      Service Name  Parameters Output Base Type   AssignProductionOrders String: Operator, String: ProductOrder  NOTHING   CreateProductionOrders  String: OrderNumber, String: Product, DATETIME: DueDate  NOTHING  DeleteProductionOrders  String: ProductOrder  NOTHING  GetFilteredProductionOrders  String: ProductOrder  InfoTable: ProductionOrder  GetProductionLineList  NONE  InfoTable: ProductDataShape  GetUnfilteredProductionOrders  NONE  InfoTable: ProductionOrder  MarkSelfOperator  NONE  BOOLEAN  UpdateProductionOrdersOP  String: ProductOrder, String: UpdatedInformation  NOTHING  UpdateProductionOrdersPM  String: ProductOrder, String: UpdatedInformation  NOTHING   Use the same method for creating Services that were provided in earlier sections. Your Services list should match the below configurations.       Challenge Yourself     Complete the implementation of the Data Model shown below by creating the Thing instances of the ThingTemplates we have created. When finish, add more to the Data Model. Some ideas are below.         Ideas for what can be added to this Data Model: #  Idea  1 Add users and permissions   2  Add Mashups to view maintenance requests, products, and production orders  3  Add business logic to the Data Model   Step 10: Next Steps     Congratulations! You've successfully completed the Data Model Implementation Guide. This guide has given you the basic tools to: Create Things, Thing Templates, and Thing Shapes Add Events and Subscriptions   The next guide in the Design and Implement Data Models to Enable Predictive Analytics learning path is Create Custom Business Logic.  

Create Custom Business Logic    Overview   This project will introduce you to creating your first ThingWorx Business Rules Engine.   Following the steps in this guide, you will know how to create your business rules engine and have an idea of how you might want to develop your own. We will teach you how to use your data model with Services, Events, and Subscriptions to establish a rules engine within the ThingWorx platform.   NOTE: This guide's content aligns with ThingWorx 9.3. The estimated time to complete this guide is 60 minutes.    Step 1: Completed Example   Download the attached, completed files for this tutorial: BusinessLogicEntities.xml.   The BusinessLogicEntities.xml file contains a completed example of a Business Rules Engine. Utilize this file to see a finished example and return to it as a reference if you become stuck during this guide and need some extra help or clarification. Keep in mind, this download uses the exact names for entities used in this tutorial. If you would like to import this example and also create entities on your own, change the names of the entities you create.   Step 2: Rules Engine Introduction   Before implementing a business rule engine from scratch, there are a number of questions that should first be answered. There are times in which a business rule engine is necessary, and times when the work can be down all within regular application coding.   When to Create a Rules Engine: When there are logic changes that will often occur within the application. This can be decisions on how to do billing based on the state or how machines in factories should operate based on a release. When business analysts are directly involved in the development or utilization of the application. In general, these roles are often non-technical, but being involved with the application directly will mean the need for a way to make changes. When a problem is highly complex and no obvious algorithm can be created for the solution. This often covered scenarios in which an algorithm might not be the best option, but a set of conditions will suffice.   Advantages of a Rules Engine The key reward is having an outlet to express solutions to difficult problems than can be easily verifiable. A consolidated knowledge base for how a part of a system works and a possible source of documentation. This source of information provides people with varying levels of technical skill to all have insight into a business model.   Business Logic with ThingWorx Core Platform: A centralized location for development, data management, versioning, tagging, and utilization of third party applications. The ability to create the rules engine within the ThingWorx platform and outside of ThingWorx platform. Being that the rules engine can be created outside of the ThingWorx platform, third party rules engines can be used. The ThingWorx platform provides customizable security and provided services that can decrease the time in development.     Step 3: Establish Rules   In order to design a business rules engine and establish rules before starting the development phase, you must capture requirements and designate rule characteristics.   Capture Requirements The first step to building a business rules engine is to understand the needs of the system and capture the rules necessary for success.   Brainstorm and discuss the conditions that will be covered within the rules engine Construct a precise list Identify exact rules and tie them to specific business requirements.   Each business rule and set of conditions within the business rule will need to be independent of other business rules. When there are several scenarios involved, it is best to create multiple rules – one handling each. When business rules are related to similar scenarios, the best methodology is to group the rules into categories.   Category Description Decision Rules Set of conditions regarding business choices Validation Rules Set of conditions regarding data verifications Generation Rules Set of conditions used for data object creation in the system Calculation Rules Set of conditions that handle data input utilized for computing values or assessments   Designate Rule Characteristics Characteristics for the rules include, but are not limited to: Naming conventions/identifiers Rule grouping Rule definition/description Priority Actions that take place in each rule.   After this is completed, you will be ready to tie business requirements to business rules, and those directly to creating your business rules engine within the platform.   Rules Translation to ThingWorx There are different methods for how the one to one connections can be made between rules and ThingWorx. The simplified method below shows one way that all of this can be done within the ThingWorx platform:   Characteristic  ThingWorx Aspect Rule name/identifier Service Name Ruleset  Thing/ThingTemplate Rule definition  Service Implementation Rule conditions Service Implementation Rule actions Service Implementation Data management DataTables/Streams   Much of the rule implementation is handled by ThingWorx Services using JavaScript. This allows for direct access to data, other provided Services, and a central location for all information pertaining to a set of rules. The design provided above also allows for easier testing and security management.   Step 4: Scenario Business Rule Engine    An important aspect to think about before implementing your business rules engine, is how the Service implementation will flow.   Will you have a singular entry path for the entire rules engine? Or will you have several entries based on what is being requested of it? Will you have create only Services to handle each path? Or will you create Events and Subscriptions (Triggers and Listeners) in addition to Services to split the workload?   Based on how you answer those questions, dictates how you will need to break up your implementation. The business rules for the delivery truck scenario are below. Think about how you would break down this implementation.   High Level Flow 1 Customer makes an order with a Company (Merchant). 1.A Customer to Merchant order information is created. 2 The Merchant creates an order with our delivery company, PTCDelivers. 2.A Merchant order information is populated. 2.B Merchant sets delivery speed requested. 2.C Merchant sets customer information for the delivery. 3 The package is added to a vehicle owned by PTCDelivers. 4 The vehicle makes the delivery to the merchant's customer.   Lower Level: Vehicles 1 Package is loaded onto vehicle 1.i Based on the speed selected, add to a truck or plane. 1.ii Ground speed option is a truck. 1.iii Air and Expedited speed options are based on planes usage and trucks when needed. 2 Delivery system handles the deliveries of packages 3 Delivery system finds the best vehicle option for delivery 4 An airplane or truck can be fitted with a limited number of packages.   Lower Level: Delivery 1 Delivery speed is set by the customer and passed on to PTCDelivers. 2 Delivery pricing is set based on a simple formula of (Speed Multiplier * Weight) + $1 (Flat Fee). 2.i Ground arrives in 7 days. The ground speed multiplier is $2. 2.ii Air arrives in 4 days. The air speed multiplier is $8. 2.iii Expedited arrives in 1 day. The expedited speed multiplier is $16. 3 Deliveries can be prioritized based on a number of outside variables. 4 Deliveries can be prioritized based on a number of outside variables. 5 Bulk rate pricing can be implemented.   How would you implement this logic and add in your own business logic for added profits? Logic such as finding the appropriate vehicle to make a delivery can be handled by regular Services. Bulk rates, prioritizing merchants and packages, delivery pricing, and how orders are handled would fall under Business Logic. The MerchantThingTemplate Thing contains a DataChange Subscription for it's list of orders. This Subscription triggers an Event in the PTCDelivers Thing.   The PTCDelivers Thing contains an Event for new orders coming in and a Subscription for adding orders and merchants to their respective DataTables. This Subscription can be seen as the entry point for this scenario. Nevertheless, you can create a follow-up Service to handle your business logic. We have created the PTCDeliversBusinessLogic to house your business rules engine.   Step 5: Scenario Data Model Breakdown   This guide will not go into detail of the data model of the application, but here is a high level view of the roles played within the application.   Thing Shapes ClientThingShape Shape used to represent the various types of clients the business faces (merchants/customers). VehicleThingShape Shape used to represent different forms of transportation throughout the system.   Templates PlaneThingTemplate Template used to construct all representations of a delivery plane. TruckThingTemplate Template used to construct all representations of a delivery truck. MerchantThingTemplate Template used to construct all representations of a merchant where goods are purchased from. CustomerThingTemplate Template used to construct all representations of a customer who purchases goods.   Things/Systems PTCDeliversBusinessLogic This Thing will hold a majority of the business rule implementation and convenience services. PTCDelivers A Thing that will provide helper functions in the application.   DataShapes PackageDeliveryDataShape DataShape used with the package delivery event. Will provide necessary information about deliveries. PackageDataShape DataShape used for processing a package. OrderDataShape DataShape used for processing customer orders. MerchantOrderDataShape DataShape used for processing merchant orders. MerchantDataShape DataShape used for tracking merchants.   DataTables OrdersDatabase DataTable used to store all orders made with customers. MerchantDatabase DataTable used to store all information for merchants.     Step 6: Next Steps   Congratulations! You've successfully completed the Create Custom Business Logic guide, and learned how to: Create business logic for IoT with resources provided in the ThingWorx platform Utilize the ThingWorx Edge SDK platforms with a pre-established business rule engine   We hope you found this guide useful.    The next guide in the Design and Implement Data Models to Enable Predictive Analytics learning path is Implement Services, Events, and Subscriptions.     

Use Case: You’ve published a model from Analytics Builder to Analytics Manager, and then used service CreateOrUpdateThingTemplateForModel on resource TW.AnalysisServices.ModelManagementServicesAPI. A thing created from the resulting template will have an infotable called “data” which needs to be populated in order to trigger an Analysis Event & Job. For example you might have been following the online documentation for Analytics Manager > Working with Thing Predictor > Demo: Using Thing Predictor, link here. This script makes it easy to create a line of test data into field "data" on your thing to trigger the analysis event & job. Also fields causalTechnique, goalName and importantFieldCount are set programmatically, these are needed for the analysis event & job. Also this script might be useful as a general example of how to write to an infotable property on a thing. The JavaScript code is shown here and also attached as a text file to this post: me.causalTechnique = 'FULL_RANGE' me.goalName = 'predict_Compressor_failure' me.importantFieldCount = 3 // ThingPredictor.test_3f1a6a31-e388-4232-9e47-284572658a4a.InputParamsdataDataShape entry object //var newEntry = new Object(); var params = { infoTableName : "InfoTable", dataShapeName : "ThingPredictor.test-integer_afebaef3-b2cf-4347-824c-a39c11ddbb4a.InputParamsdataDataShape" }; // CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(ThingPredictor.test_3f1a6a31-e388-4232-9e47-284572658a4a.InputParamsdataDataShape) var myInfoTable = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params); // 2 - CREATE INFOTABLE ROW USING object var newEntry = new Object(); newEntry._Pressure = 10.5; // NUMBER newEntry._Temperature = 45.1; // NUMBER newEntry._VibrationX = 81; // NUMBER newEntry._VibrationY = 65; // NUMBER //newEntry.key = 4; // STRING - isPrimaryKey = true // 3 - ADD INFOTABLE ROW USING TO INFOTABLE myInfoTable.AddRow(newEntry); // 3 – PERSIST INFOTABLE TO THE THING PROPERTY ‘data’ me.data = myInfoTable;

I have put together a small sample of how to get property values from a Windows Powershell command into Thingworx through an agent using the Java SDK. In order to use this you need to import entities from ExampleExport.xml and then run SteamSensorClient.java passing in the parameters shown in run-configuration.txt (URL, port and AppKey must be adapted for your system). ExampleExport.xml is a sample file distributed with the Java SDK which I’ve also included in the zipfile attached to this post. You need to go in Thingworx Composer to Import/Export … Import from File … Entities … Single File … Choose File … Import. Further instructions / details are given in this short video: Video Link : 2181

      Thingworx extensions are a great place to explore UI ideas and get that special feature you want.   Here is a quick primer on Widgets (Note: there is comprehensive documentation here which explores the complete development process ).The intention is not to explain every detail but just the most important points to get you started. I will explore more in additional posts. I also like images rather than lost of words to read. I have attached the simple Hello Word example as a start point and  I'm using Visual Code as my editor of choice.   The attached zip when unzipped will contain a folder called ui and metadata xml file. Within the ui folder there needs to be a folder that has the same name as the widget name. In this case its helloworld.   Metadata file - The 3 callouts are the most import. Package version: is the current version and each time a change is made the value needs to be updated. name: a unique name used through out the widget definition UIResources: The source locations for the widget definition. The UIResources files are used to define the widget in the ide (Composer) and runtime (Mashup). These 2 environments ide and runtime have matching pairs of css (cascading style sheets)  and a js (javascript) files.   The js files are where most of the work is done. There a number of functions used inside the javascript file but just to get things going we will focus on the renderHtml function. This is the function that will generate the HTML to be inserted in the widget location.   renderHtml(helloWorld.ide.js) In this very simple case the renderHtml in the runtime is the same as in the ide renderHtml (helloWorld.runtime.js)   Hopefully you can see that the HTML is pretty easy just some div and span tags with some code to get the Property called Salutation.   So we have the very basics and we are not worried to much about all the other things not mentioned. So to get the simple extension into Thingworx we use the Import -> Extensions menu option. The UI and metadata.xml file needs to be zipped up (as per attachment).  Below is a animated gif that shows how to import and use the widget   Very Quick Steps to import and use in mashup. Video Link : 2147   The next blog will explore functions and allow a user to click the label and display a random message. This will show how to use events   Widget Extensions Click Event

The following code snippet will retrieve a months worth of data from the system and return it as a CSV document suitable for import into your spreadsheet or reporting tool of choice. import static com.axeda.sdk.v2.dsl.Bridges.* import com.axeda.drm.sdk.Context import com.axeda.common.sdk.id.Identifier import com.axeda.services.v2.* import com.axeda.sdk.v2.exception.* def ac = new AuditCriteria() ac.fromDate = Date.parse('yyyy-MM-dd', '2017-03-01') ac.toDate   = Date.parse('yyyy-MM-dd', '2017-03-31') def retString = '' tcount = 0 while ( (results = auditBridge.find(ac)) != null  && tcount < results .totalCount) {   results.audits.each { res ->     retString += "${res?.user?.id},${res?.asset?.serialNumber},${res?.category},${res.message},${res.date}\n"     tcount++   }   ac.pageNumber = ac.pageNumber + 1 } return retString

Axeda Enterprise has long provided a feature to run custom code on the server side in response to end user requests or events triggered by data sent in by remote agents.  Version 6.6 introduced Axeda Artisan - an Apache Maven based tool to add modern best practices to developing Axeda-based solutions, using modern code editors such as Eclipse and IntelliJ, and allowing for the use of source code control tools like Git or Clearcase.  One downside to Artisan, however, is that it has no export tool - no way to take currently existing entities in the Axeda instance, and save them. The attached Groovy script, GetCustomObjects.groovy, solves that problem for custom objects.  It will iterate an Axeda instance and save any found CustomObjects to disk for backup, or to use to bootstrap an Artisan project from an existing instance. { / }  » groovy GetCustomObjects.groovy usage: getCustomObjects -acceptBadSSL          Ignore any TLS validation issues -h                     help -instance <instance>   instance name - directory to store results -password <password>   password -url <url>             url of Axeda Machine Cloud -username <username>   username An example call might look like: { / } groovy GetCustomObjects.groovy -instance prod-instance -url https://prod-instance.example.com -username <uname> -password <pwd> This will save all custom objects in a directory called prod-instance.

You can control the Tracking Indicator that is used to mark the ThingMark position. The Tracking Indicator is a green hexagon, in the screenshot below the red arrow points to it. You can control the display of this tracking indicator via the Display Tracking Indicator property of the ThingMark widget: But you can also get fancier. Here is an exmaple that shows the tracking indicator for 3 seconds when the tracking has started and then hides it automatically. To achieve such a behavior you'll have to use a bit of Javascript. We'll first create a function hideIn3Sec() in the javascript section of our view and then add it to the javascript handler of the Tracking Acquired event of the ThingMark widget. Step 1: Here is the code for copy/paste convenience: $scope.hideIn3Sec=function(){   // The $timeout function has two arguments: the function to execute (defined inline here)   // and the time in msec after which the function is invoked.   $timeout( function hide(){     // you may have to change 'thingMark-1' by the id of the ThingMark in your own experience     $scope.app.view['Home'].wdg['thingMark-1']['trackingIndicator']=false;   },   3000); } Step 2: That's it. Have fun!

Thingworx actually provides some services for this, but it exports them to an XML file. I'm pretty sure that there are people who will be able to turn this into something easily legible in a mashup. There are two services in CollectionFunctions ExportUserPermissions ImportUserPermissions

The System user is pivotal in securing your application and the simplest approach is to assign the System user to ALL Collections and give it Runtime Service Execute. These Collection Permissions ONLY Export to ThingworxStorage vs. the File Export, it becomes quite painful to manage this and then roll this out to a new machine. Best and fastest solution? Script the Assignment, you can take this script which does it for the System user and extend it to include any other Collection Level permissions you might need to set, like adding Entity Create Design Time for the System user. --------------------------------------------------------- //@ThingworxExtensionApiMethod(since={6,6}) //public void AddCollectionRunTimePermission(java.lang.String collectionName, //       java.lang.String type, //       java.lang.String resource, //       java.lang.String principal, //       java.lang.String principalType, //       java.lang.Boolean allow) //    throws java.lang.Exception // //Service Category: //    Permissions // //Service Description: //    Add a run time permission. // //Parameters: //    collectionName - Collection name (Things, Users, ThingShapes, etc.) - STRING //    type - Permission type (PropertyRead PropertyWrite ServiceInvoke EventInvoke EventSubscribe) - STRING //    resource - Resource name (* = all or enter a specific resource to override) - STRING //    principal - Principal name (name of user or group) - STRING //    principalType - Principal type (User or Group) - STRING //    allow - Permission (true = allow, false = deny) - BOOLEAN //Throws: //    java.lang.Exception - If an error occurs //   var params = {     modelTags: undefined /* TAGS */,     type: undefined /* STRING */ }; // result: INFOTABLE dataShape: EntityCount var EntityTypeList = Subsystems["PlatformSubsystem"].GetEntityCount(params); for each (var row in EntityTypeList.rows) {     try {         var params = {             principal: "System" /* STRING */,             allow: true /* BOOLEAN */,             resource: "*" /* STRING */,             type: "ServiceInvoke" /* STRING */,             principalType: "User" /* STRING */,             collectionName: row.name /* STRING */         };         // no return         Resources["CollectionFunctions"].AddCollectionRunTimePermission(params);     }     catch(err) {     } }

Objective Learn how the Scripto Web Service helps you to present platform information in your HTML with JavaScript and dynamic page updating.  After this tutorial, you will know how to create Axeda Custom Objects that return formatted results to JavaScript using XmlHttpResponse, and how a very simple page can incorporate platform data into your browser-based user interface. Part 1 - Simple Scripto In Using Scripto, you learned how Scripto can be called from very simple clients, even the most basic HTTP tools. This tutorial builds on the examples in that tutorial. The following HelloWorld script accepts a parameter named "foo". This means that the caller of the script may supply a value for this parameter, and the script simple returns a message that includes the value supplied. import static com.axeda.sdk.v2.dsl.Bridges.* import com.axeda.services.v2.* import com.axeda.sdk.v2.exception.* return "Hello world, ${parameters.foo}" In the first part of this tutorial, we'll be creating an HTML page with some JavaScript that simply calls the HelloWorld script and puts the result on the page. Create an HTML File Open up your favorite text editor and create a blank document. Paste in this simple scaffold, which includes a very simple FORM with fields for your developer platform email and password, and the "foo" parameter. <html> <head> <title>Axeda Developer Connection Simple Ajax HelloWorld Example</title> </head> <body> <form name="f1">         Platform email (login): <input name="username" type="text"><br/>         Password: <input name="password" type="password"><br/>         foo: <input name="foo" type="text"><br/> <input value="Go" type="button" onclick='JavaScript: callScripto()'/></p> <div id="result"></div> </form> </body> </html> Pretty basic HTML that you've seen lots of times. Notice the form onclick refers to a JavaScript function. We'll be adding that next. Add the JavaScript Directly under the <title> tag, add the following <script language="Javascript"> var scriptoURL ="http://dev6.axeda.com/services/v1/rest/Scripto/execute/"; var scriptName ="HelloWorld2"; </script> This defines our JavaScript block, and a couple of constants to tell our script where the server's Scripto REST endpoint is, and the name of the script we will be running. Let's add in our callScripto() function. Paste the following directly under the scriptName variable declaration: function callScripto(){ try{                 netscape.security.PrivilegeManager.enablePrivilege("UniversalBrowserRead"); }catch(e){ // must be IE }    var xmlHttpReq =false;    var self =this;    // Mozilla/Safari    if(window.XMLHttpRequest){                 self.xmlHttpReq =new XMLHttpRequest();    }// IE elseif(window.ActiveXObject){                 self.xmlHttpReq =new ActiveXObject("Microsoft.XMLHTTP"); }    var form = document.forms['f1'];    var username = form.username.value;    var password = form.password.value;    var url = scriptoURL + scriptName +"?username="+ username +"&password="+ password;             self.xmlHttpReq.open('POST', url,true);             self.xmlHttpReq.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');             self.xmlHttpReq.onreadystatechange =function() {       if(self.xmlHttpReq.readyState ==4){                     updatepage(self.xmlHttpReq.responseText);       }    }    var foo = form.foo.value;    var qstr ='foo='+escape(foo);    self.xmlHttpReq.send(qstr); } That was a lot to process in one chunk, so let's examine each piece. This piece just tells the browser that we'll be wanting to make some Ajax calls to a remote server. We'll be running the example right off a local file system (at first), so this is necessary to ask for permission. try{                 netscape.security.PrivilegeManager.enablePrivilege("UniversalBrowserRead"); }catch(e){ // must be IE } This part creates an XmlHttpRequest object, which is a standard object available in browsers via JavaScript. Because of slight browser differences, this code creates the correct object based on the browser type. var xmlHttpReq =false; var self =this; // Mozilla/Safari if(window.XMLHttpRequest){                 self.xmlHttpReq =new XMLHttpRequest(); } // IE elseif(window.ActiveXObject){                 self.xmlHttpReq =new ActiveXObject("Microsoft.XMLHTTP"); } Next we create the URL that will be used to make the HTTP call. This simply combines our scriptoURL, scriptName, and platform credentials. var form = document.forms['f1']; var username = form.username.value; var password = form.password.value; var url = scriptoURL + scriptName +"?username="+ username +"&password="+ password; Now let's tell the xmlHttpReq object what we want from it. we'll also reference the name of another JavaScript function which will be invoked when the operation completes. self.xmlHttpReq.open('POST', url,true);     self.xmlHttpReq.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');     self.xmlHttpReq.onreadystatechange =function(){ if(self.xmlHttpReq.readyState ==4){             updatepage(self.xmlHttpReq.responseText); } } Finally, for this function, we'll grab the "foo" parameter from the form and tell the prepped xmlHttpReq object to post it. var qstr ='foo='+escape(foo);     self.xmlHttpReq.send(qstr); almost done. We just need to supply the updatepage function that we referenced above. Add this code directly before the </script> close tag: function updatepage(str){             document.getElementById("result").innerHTML = str; } Try it out Save your file as helloworld.html and open it in a browser by starting your browser and choosing "Open File". You can also download a zip with the file prepared for you at the end of this page. If you are using Internet Explorer, IE will pop a bar asking you if it is OK for the script inside this page to execute a script. Choose "Allow Blocked Content". Type in your platform email address (the address you registered for the developer connection with) and your password. Enter any text that you like for "foo". When you click "Go", the area below the button will display the result of the Scripto call. Note that if you are using Mozilla Firefox, you will be warned about the script wanting to access a remote server. Click "Allow". Congratulations! You have learned how to call a Custom Object-backed Scripto service to display dynamic platform content inside a very simple HTML page. Next Steps Be sure to check out the tutorial on Hosting Custom Applications to learn how you can make this page get directly served from your platform account, with its very own URL. Also explore code samples that show more sophisticated HTML+AJAX examples using Google Charts and other presentation tools.

This script will get all contacts (optionally limited to a particular organization) and check whether there is a DeviceContact associated with it.  If there is no DeviceContact (meaning it is not associated with a device), it deletes the contact. Note - It is worthwhile to test this script by commenting out the contact.delete() line first and reviewing which contacts will be deleted.  Also, this script works by finding all contacts, therefore it is not recommended to run the script repeatedly within a short period of time. Parameter: organizationName  (OPTIONAL) - Str - the name of the organization import net.sf.json.JSONObject import com.axeda.drm.sdk.device.DeviceFinder import com.axeda.drm.sdk.device.Device import com.axeda.drm.sdk.device.ModelFinder import com.axeda.drm.sdk.device.Model import com.axeda.drm.sdk.data.CurrentDataFinder import com.axeda.drm.sdk.data.DataValue import net.sf.json.groovy.JsonSlurper import com.axeda.drm.sdk.contact.Contact import com.axeda.drm.sdk.contact.ContactFinder import com.axeda.drm.sdk.contact.Location import com.axeda.drm.sdk.contact.LocationFinder import com.axeda.drm.sdk.contact.OrganizationFinder import com.axeda.drm.sdk.Context import com.axeda.drm.sdk.contact.Organization import com.axeda.drm.sdk.contact.DeviceContact import com.axeda.drm.sdk.contact.ContactMethodType import com.axeda.drm.sdk.contact.DeviceContactFinder import groovy.json.* import com.axeda.drm.sdk.scripto.Request import com.axeda.common.sdk.id.Identifier /** * ContactDelete.groovy * ----------------------- * * Finds all contacts, then finds the device contact for each contact. If null, deletes the contact. * * @params * organizationName (OPTIONAL) Str - limit the contact deletion to an organization * * * @author Sara Streeter <sstreeter@axeda.com> */ /** * initialize our global variables * json = the contents of our response * infoString = a stringBuilder used to collect debug information during the script * contentType = the content type we will return * scriptname = The name of this Script, used in multiple places */ def json = new groovy.json.JsonBuilder() def infoString = new StringBuilder() def contentType = "application/json" def scriptName = "ContactDelete.groovy" def root = ["result":["deleted":[]]] def timings = [:] timings.contactFinding = 0 timings.contactIterating = 0 wholestart = System.currentTimeMillis() final Context CONTEXT = Context.getSDKContext() try {       def params = Request?.parameters?.size() > 0 ? Request?.parameters : parameters       ContactFinder cfinder = new ContactFinder(CONTEXT)       def start = System.currentTimeMillis()     def organization       if (params.organizationName != null && params.organizationName != ""){         OrganizationFinder oFinder = new OrganizationFinder(CONTEXT)         oFinder.setName(params.organizationName)         organization = oFinder.find()               if (organization){             cfinder.setOrganization(organization)         }     }       List<Contact> contacts = cfinder.findAll()     timings.contactFinding += System.currentTimeMillis()-start       root.result.contactSize = contacts.size()       start = System.currentTimeMillis()     contacts.each{  contact ->           DeviceContactFinder dcfinder = new DeviceContactFinder(CONTEXT)         dcfinder.setContactId(contact.id)         def dc = dcfinder.findAll()         if (dc.size() == 0){             root.result.deleted << [                 id: contact.id.value,                 firstName: contact.firstName,                 lastName: contact.lastName,                 organization: contact.organization?.name             ]             contact.delete()  // comment out this line to check which contacts will be deleted first.         }     }     timings.contactIterating += System.currentTimeMillis()-start     } catch (Exception e) {     processException(scriptName,json,e) } finally {     timings.wholescript = System.currentTimeMillis() - wholestart     root += [timings: timings] } return ['Content-Type': 'application/json', 'Content': JSONObject.fromObject(root).toString(2)] /*     Processes the contents of an Exception and add it to the Errors collection     @param json The markup builder */ private def processException(String scriptName, JsonBuilder json, Exception e) {     // catch the exception output     def logStringWriter = new StringWriter()     e.printStackTrace(new PrintWriter(logStringWriter))     logger.error("Exception occurred in ${scriptName}: ${logStringWriter.toString()}")     /*         Construct the error response         - errorCode Will be an element from an agreed upon enum         - errorMessage The text of the exception      */     json.errors  {         error {             message     "[${scriptName}]: " + e.getMessage()             timestamp   "${System.currentTimeMillis()}"         }     }     return json }

This is an example to show returning an uploaded file as a binary from a Groovy Script. Parameters: model_name serial_number import java.io.StringWriter import java.io.PrintWriter import com.axeda.drm.sdk.Context import com.axeda.drm.sdk.data.* import com.axeda.drm.sdk.device.* import javax.activation.MimetypesFileTypeMap try {     Context ctx = Context.getUserContext()     ModelFinder modelFinder = new ModelFinder(ctx)     modelFinder.setName(parameters.model_name)     Model model = modelFinder.find()     DeviceFinder dfinder = new DeviceFinder(ctx)     dfinder.setModel(model)     dfinder.setSerialNumber(parameters.serial_number)     Device d = dfinder.find()     UploadedFileFinder uff = new UploadedFileFinder(ctx)     uff.device = d     def ufiles = uff.findAll()     UploadedFile ufile     if (ufiles.size() > 0) {         ufile = ufiles[0]         File f = ufile.extractFile()     def type = getType(f)     return ['Content-Type': type, 'Content': new FileInputStream(f)]      } else {     return ['Content-Type': 'text/plain', 'Content': 'No files have been uploaded'] } } catch (Exception e) {     logger.info(e.message)     StringWriter logStringWriter = new StringWriter();     PrintWriter logPrintWriter = new PrintWriter(logStringWriter)     e.printStackTrace(logPrintWriter)     logger.info(logStringWriter.toString()) } static String getType(File f) {   MimetypesFileTypeMap mimeTypesMap = new MimetypesFileTypeMap()   return mimeTypesMap.getContentType(f); }

Update properties on a model. Parameter: modelName (REQUIRED) - String - the name of the model to have its properties updated. import com.axeda.drm.sdk.device.DeviceProperty import com.axeda.drm.sdk.device.ModelFinder import com.axeda.drm.sdk.device.Model import com.axeda.drm.sdk.Context import com.axeda.drm.sdk.device.DevicePropertyFinder import com.axeda.drm.sdk.device.Property import com.axeda.drm.sdk.device.PropertyType import com.axeda.common.sdk.id.Identifier Set<String> REQUIRED_PROPERTIES = [     "TestProperty0","TestProperty1","TestProperty2" ] try {       final def Context CONTEXT = Context.getSDKContext()       ModelFinder modelFinder = new ModelFinder(CONTEXT);     modelFinder.setName(parameters.modelName)    Model model = modelFinder.find();    if (model == null){ throw new Exception("No model found") }    modelProperties = findModelProperties(CONTEXT, model.id)     updateProperties(CONTEXT, model.id, modelProperties, REQUIRED_PROPERTIES)     modelProperties.properties.each{ logger.info("$it.name :$it.value") } } catch (Exception e){     logger.info e.localizedMessage } return true private DeviceProperty findModelProperties(Context context, Identifier modelID) {   def finder = new DevicePropertyFinder(context)   finder.id = modelID   finder.type = PropertyType.MODEL_TYPE   return finder.findOne() as DeviceProperty } private void updateProperties(Context context, Identifier modelID, DeviceProperty modelProperties, Set<String> requiredProperties) {   if (!modelProperties) {     modelProperties = new DeviceProperty(context)     modelProperties.id = modelID     modelProperties.type = PropertyType.MODEL_TYPE     modelProperties.properties = []   }   (requiredProperties - new HashSet<String>(modelProperties.properties.collect { it.name })).inject(modelProperties.properties) { list, propertyName -> list << new Property(0, propertyName, "") }   modelProperties.store() }

Sends an email with an alarm name passed in by an Expression Rule. Parameters (passed in as arguments to ExecuteCustomObject in the Expression Rule): fromaddress toaddress import com.axeda.drm.sdk.contact.Email /* * ExprRuleAlarmToEmail.groovy * * Sends an email with an alarm name passed in by an Expression Rule. * * @param fromaddress - (REQ):Str email address of sender. * @param toaddress - (REQ): Str email address of recipient * * * @note Should be executed from an Expression Rule like the following: * * Type: Alarm * If: Alarm.severity > 490 && Alarm.severity < 700 * Then: ExecuteCustomObject("ExprRuleAlarmToEmail", "fake_sender@axeda.com","fake_recipient@axeda.com") * * @author Sara Streeter <sstreeter@axeda.com> */ try {   String fromaddress = parameters.fromaddress   String toaddress = parameters.toaddress   String subject = "Axeda Alarm - ${alarm.name}"   String body = "You are receiving this alarm ${alarm.name} because you are subscribed to its updates."   sendEmail(fromaddress, toaddress, subject, body) } catch (Exception e) { logger.error(e.message) }     public void sendEmail(String fromAddress,String toAddress,String subject, String body) {         try {             Email.send(fromAddress, toAddress, subject, body);         } catch (AddressException ae) {             logger.error(ae);         }     }

These code snippets illustrate parsing CSV files and populating the Axeda Enterprise with data, locations and organizations.  These files are incoming to the Axeda Platform. Note:  These snippets do NOT handle null values in the CSV due to the lack of a CSV parsing library.  Workaround is to populate empty values with an empty or null signifier (such as a blank space) and test for these on the Groovy side. Code Snippets: CSV file to Data Items CSV file to Location Organization Script Name: CSV file to Data Items Description: Executed from an expression rule with file hint "datainsert", takes a CSV file and adds data based on values. Parameters: OPTIONAL - only needed for debugging modelName - (OPTIONAL) Str - name of the model serialNumber - (OPTIONAL) Str - name of the serial number import com.axeda.drm.sdk.Context import com.axeda.drm.sdk.device.DeviceFinder import com.axeda.drm.sdk.device.ModelFinder import com.axeda.drm.sdk.device.DataItemFinder import com.axeda.drm.sdk.device.DataItem import com.axeda.drm.sdk.data.DataValueEntry import java.util.regex.Pattern import groovy.json.* import com.axeda.drm.services.device.DataItemType import net.sf.json.JSONObject /** * CSVToData.groovy * ----------------------- * * Executed from an expression rule with file hint "datainsert", takes a CSV file and adds data based on values. * * @note  There must be a column with "model" and one with "serial".  The rest of the columns should be data item names with values * in the rows. DOES NOT handle null values in CSV.  Workaround is to insert blank spaces in null values and test for those on the Groovy side. * Solution would be to add a library for CSV parsing such as open csv. * * @params - only needed if NOT executed from expression rule - primarily for debugging * modelName - (OPTIONAL) Str - name of the model * serialNumber - (OPTIONAL) Str - name of the serial number * * */ /** * initialize our global variables * json = the contents of our response * infoString = a stringBuilder used to collect debug information during the script * contentType = the content type we will return * scriptname = The name of this Script, used in multiple places */ def json = new groovy.json.JsonBuilder() def infoString = new StringBuilder() def contentType = "application/json" def scriptName = "CSVToData.groovy" def root = ["result":["items":[]]] def columns = [] try {   Context CONTEXT = Context.getSDKContext()   def modelIndex   def serialIndex   // initialize Model and Device Finders   ModelFinder modelFinder = new ModelFinder(CONTEXT)   DeviceFinder deviceFinder = new DeviceFinder(CONTEXT)   // implicit object compressedFile   File file = compressedFile.getFiles()[0].extractFile() /* //begin non-expression rule code, useful for debugging     File file     modelFinder.setName(Request.parameters.modelname)               def model1 = modelFinder.find()     deviceFinder.setSerialNumber(Request.parameters.serialNumber)     deviceFinder.setModel(model1)     def d = deviceFinder.find()      UploadedFileFinder uff = new UploadedFileFinder(CONTEXT)     uff.device = d     def ufiles = uff.findAll()     UploadedFile ufile     if (ufiles.size() > 0) {         ufile = ufiles[0]         file = ufile.extractFile()     }          */ //end non-expression rule code   file.eachLine {line ->       def row = line.tokenize(',')          // set the column headings       if (columns.size() == 0){         columns = row              // find model and serial index, assumes there's a column that has "model" and "serial", otherwise take columns 0 and 1         def modelpatt = Pattern.compile(/[A-Za-z_\-]{0,}model[A-Za-z_\-]{0,}/, Pattern.CASE_INSENSITIVE)         def serialpatt = Pattern.compile(/[A-Za-z_\-]{0,}serial[A-Za-z_\-]{0,}/, Pattern.CASE_INSENSITIVE)         modelIndex = columns.findIndexOf{ it ==~ modelpatt } > -1 ? columns.findIndexOf{ it ==~ modelpatt } : 0         serialIndex = columns.findIndexOf{ it ==~ serialpatt } > -1 ? columns.findIndexOf{ it ==~ serialpatt } : 1            }       // otherwise populate data       else {                  modelFinder.setName(row.get(modelIndex))           def model = modelFinder.find()                  deviceFinder.setModel(model)           deviceFinder.setSerialNumber(row.get(serialIndex))                  def device = deviceFinder.find()                  def assetInfo = [                     "model": model.name,                     "serial": device.serialNumber,                     "data":[]                     ]                  row.eachWithIndex{ item, index ->               if (index != modelIndex && index != serialIndex){                 def dataItemName = columns[index].replace(" ","")                 DataItemFinder dif = new DataItemFinder(CONTEXT);                 dif.setDataItemName(dataItemName);                 dif.setModel(model);                 DataItem dataItem = dif.find();                              if (dataItem){                     if (item.isNumber()){                        item = Double.valueOf(item)                     }                     DataValueEntry dve = new DataValueEntry(CONTEXT, device, dataItem, item)                     dve.store()                 }                 else {                     DataItem newDataItem                     if (item.isNumber()){                         newDataItem = new DataItem(CONTEXT, model,DataItemType.ANALOG, dataItemName)                         item = Double.valueOf(item)                     }                     else {                        newDataItem = new DataItem(CONTEXT, model,DataItemType.STRING, dataItemName)                     }                    newDataItem.store()                    DataValueEntry dve = new DataValueEntry(CONTEXT, device, newDataItem, item)                     dve.store()                 }                 assetInfo.data << [                         "name": dataItemName,                         "value": item                     ]                            }               root.result.items << assetInfo           }              }   }   logger.info(JSONObject.fromObject(root).toString(2)) } catch (Exception e) {     processException(scriptName,json,e) } //return ['Content-Type': 'application/json', 'Content': JSONObject.fromObject(root).toString(2)] /*     Processes the contents of an Exception and add it to the Errors collection     @param json The markup builder */ private def processException(String scriptName, JsonBuilder json, Exception e) {     // catch the exception output     def logStringWriter = new StringWriter()     e.printStackTrace(new PrintWriter(logStringWriter))     logger.error("Exception occurred in ${scriptName}: ${logStringWriter.toString()}")     /*         Construct the error response         - errorCode Will be an element from an agreed upon enum         - errorMessage The text of the exception      */     json.errors  {         error {             message     "[${scriptName}]: " + e.getMessage()             timestamp   "${System.currentTimeMillis()}"         }     }     return json } Script Name: CSV file to Location Organization Description: Executed from an expression rule with file hint "locorginsert", takes a CSV file and adds orgs and locations based on values. Parameters: OPTIONAL - only needed for debugging modelName - (OPTIONAL) Str - name of the model serialNumber - (OPTIONAL) Str - name of the serial number import com.axeda.drm.sdk.Context import com.axeda.drm.sdk.device.DeviceFinder import com.axeda.drm.sdk.device.ModelFinder import com.axeda.drm.sdk.device.DataItemFinder import com.axeda.drm.sdk.device.DataItem import com.axeda.drm.sdk.data.DataValueEntry import java.util.regex.Pattern import groovy.json.* import com.axeda.drm.services.device.DataItemType import net.sf.json.JSONObject import com.axeda.drm.sdk.contact.Organization import com.axeda.drm.sdk.contact.Location import com.axeda.drm.sdk.contact.OrganizationFinder import com.axeda.drm.sdk.contact.LocationFinder import com.axeda.drm.sdk.data.UploadedFile import com.axeda.drm.sdk.data.UploadedFileFinder /** * CSVToLocOrg.groovy * ----------------------- * * Executed from an expression rule with file hint "locorginsert", takes a CSV file and adds orgs and locations based on values. * * @note  There must be a column with "model" and one with "serial".  The rest of the columns should be either parts of a * location or an organization.  The location parts columns should be prefixed with the org# that they correspond to. * DOES NOT handle null values in CSV.  Workaround is to insert blank spaces in null values and test for those on the Groovy side. * Solution would be to add a library for CSV parsing such as open csv. * * @params - only needed if NOT executed from expression rule - primarily for debugging * modelName - (OPTIONAL) Str - name of the model * serialNumber - (OPTIONAL) Str - name of the serial number * * * */ /** * initialize our global variables * json = the contents of our response * infoString = a stringBuilder used to collect debug information during the script * contentType = the content type we will return * scriptname = The name of this Script, used in multiple places */ def json = new groovy.json.JsonBuilder() def infoString = new StringBuilder() def contentType = "application/json" def scriptName = "CSVToLocOrg.groovy" def root = ["result":["items":[]]] def columns = [] try {   Context CONTEXT = Context.getSDKContext()   def modelIndex   def serialIndex   def locIndices = [:]   def locKeys = ["line1","line2", "address1", "address2", "city","state","zip","country", "org"]   // initialize Finders   ModelFinder modelFinder = new ModelFinder(CONTEXT)   DeviceFinder deviceFinder = new DeviceFinder(CONTEXT)   LocationFinder locationFinder = new LocationFinder(CONTEXT)   OrganizationFinder organizationFinder = new OrganizationFinder(CONTEXT)   // implicit object compressedFile   File file = compressedFile.getFiles()[0].extractFile()   /* //begin non-expression rule code, useful for debugging     File file     modelFinder.setName(Request.parameters.modelname)               def model1 = modelFinder.find()     deviceFinder.setSerialNumber(Request.parameters.serialNumber)     deviceFinder.setModel(model1)     def d = deviceFinder.find()      UploadedFileFinder uff = new UploadedFileFinder(CONTEXT)     uff.device = d     def ufiles = uff.findAll()     UploadedFile ufile     if (ufiles.size() > 0) {         ufile = ufiles[0]         file = ufile.extractFile()     }           */ //end non-expression rule code   file.eachLine {line ->       def row = line.tokenize(',')       // set the column headings       if (columns.size() == 0){         columns = row         // find model and serial index, assumes there's a column that has "model" and "serial", otherwise take columns 0 and 1         def modelpatt = Pattern.compile(/[A-Za-z_\-]{0,}model[A-Za-z_\-]{0,}/, Pattern.CASE_INSENSITIVE)         def serialpatt = Pattern.compile(/[A-Za-z_\-]{0,}serial[A-Za-z_\-]{0,}/, Pattern.CASE_INSENSITIVE)         modelIndex = columns.findIndexOf{ it ==~ modelpatt } > -1 ? columns.findIndexOf{ it ==~ modelpatt } : 0         serialIndex = columns.findIndexOf{ it ==~ serialpatt } > -1 ? columns.findIndexOf{ it ==~ serialpatt } : 1               locKeys.each{ key ->             // construct a regex for each key and create a map for finding/creating             def locPatt = Pattern.compile(/[A-Za-z0-9_\-]{0,}${key}[A-Za-z0-9_\-]{0,}/, Pattern.CASE_INSENSITIVE)             def colIndex = columns.findIndexOf{                     def match = it =~ locPatt                     if (match){                         return match?.getAt(0)                     }                 }                       if (colIndex > -1){                 locIndices[colIndex] = key             }         }       }       // otherwise populate data       else {           modelFinder.setName(row.get(modelIndex))           def model = modelFinder.find()           deviceFinder.setModel(model)           deviceFinder.setSerialNumber(row.get(serialIndex))           def device = deviceFinder.find()           def assetInfo = [                     "model": model.name,                     "serial": device.serialNumber,                     "locs":[]                     ]                   def locMap = [:]           def orgName           def locKey           def locBool = false // make sure we get some criteria           row.eachWithIndex{ item, index ->                            if (index != modelIndex && index != serialIndex && item && item != ""){                   locKey = locIndices[index]                                   if (locKey){                       locBool = true                       if (locKey == "address1"){                         locKey = "line1"                       }                       if (locKey == "address2"){                         locKey = "line2"                       }                       if (locKey == "org"){                             orgName = item                       }                       // don't execute if we've got an organization key                       else {                           // for finding                           locationFinder[locKey] = item                           // for creating (if needed)                           locMap[locKey] = item                       }                   }                               }           }                   assetInfo.org           Organization org                   if (orgName){               organizationFinder.setName(orgName)               org = organizationFinder.find()                           if (!org){                 org = new Organization(CONTEXT, orgName)                 org.store()                  }                       }                  Location loc           if (locBool){               logger.info("with bool")             loc = locationFinder.find()             logger.info(loc?.name)           }                   if (!loc){                          def line1 = locMap["line1"]                           def name = line1?.replace(" ","")?.replace(/\./,"")?.replace("_","") + "_Loc"                           def line2 = locMap["line2"]               def city = locMap["city"]               def state = locMap["state"]               def zip = locMap["zip"]               def country = locMap["country"]                           if (line1 && city){                loc = new Location(CONTEXT,name,line1,line2,city,state,zip,country)                loc.store()                                         }                           if (loc && org){                   org.addLocation(loc)                   org.store()               }                       }                   assetInfo.locs << [                    "name": loc.name,                     "line1": loc.line1,                     "line2": loc.line2,                     "city": loc.city,                     "state": loc.state,                     "zip": loc.zip,                     "country": loc.country                                      ]                    assetInfo.org = [                         "name": org.name                                       ]           root.result.items << assetInfo       }   }   logger.info(JSONObject.fromObject(root).toString(2)) } catch (Exception e) {     processException(scriptName,json,e) } //return ['Content-Type': 'application/json', 'Content': JSONObject.fromObject(root).toString(2)] /*     Processes the contents of an Exception and add it to the Errors collection     @param json The markup builder */ private def processException(String scriptName, JsonBuilder json, Exception e) {     // catch the exception output     def logStringWriter = new StringWriter()     e.printStackTrace(new PrintWriter(logStringWriter))     logger.error("Exception occurred in ${scriptName}: ${logStringWriter.toString()}")     /*         Construct the error response         - errorCode Will be an element from an agreed upon enum         - errorMessage The text of the exception      */     json.errors  {         error {             message     "[${scriptName}]: " + e.getMessage()             timestamp   "${System.currentTimeMillis()}"         }     }     return json }

This is a collection of methods for working with ExtendedObjects based on the functions used in Fleetster. import com.axeda.drm.sdk.device.Device import com.axeda.platform.sdk.v1.services.ServiceFactory import com.axeda.platform.sdk.v1.services.extobject.ExtendedObjectSearchCriteria import com.axeda.platform.sdk.v1.services.extobject.PropertySearchCriteria import com.axeda.platform.sdk.v1.services.extobject.expression.PropertyExpressionFactory import com.axeda.drm.sdk.data.CurrentDataFinder import com.axeda.platform.sdk.v1.services.extobject.ExtendedObject import com.axeda.platform.sdk.v1.services.extobject.Property import java.text.DecimalFormat import com.axeda.drm.sdk.Context import com.axeda.drm.sdk.device.DeviceDataFinder import com.axeda.drm.sdk.user.User import com.axeda.platform.sdk.v1.services.extobject.ExtendedObjectService import com.axeda.platform.sdk.v1.services.extobject.PropertyType import com.axeda.platform.sdk.v1.services.extobject.PropertyDataType import com.axeda.platform.sdk.v1.services.extobject.ExtendedObjectType import com.axeda.common.sdk.id.Identifier import groovy.time.* /* ************************************* */ /* HelperFunctionsExtendedObjects.groovy * Extended Object retrieval/manipulation functions. * * A collection of methods for working with ExtendedObjects. * * author: sara streeter <sstreeter@axeda.com> ************************************* */     def eoSvc = new ServiceFactory().getExtendedObjectService()     def fetchFirstExtendedObject(DataItem dataItem, Map searchcriteria) {         def criteria = new ExtendedObjectSearchCriteria()         criteria.extendedObjectClassName = 'com.axeda.drm.sdk.device.DataItem'         criteria.internalObjectId = dataItem.id.value         criteria.extendedClientKey = "ExtendedDataItem!${dataItem.name}"         def eo       if (searchcriteria != null){             criteria.propertySearchCriteria = advancedPropertySearch(searchcriteria)         }        def queryResult = eoSvc.findExtendedObjects(criteria,  -1, -1, null)         if (queryResult.size() > 0 ){         eo = queryResult.first()         }        return eo     }    def addOrFetchExtendedDataItem(DataItem dataItem, Map searchcriteria) {         def criteria = new ExtendedObjectSearchCriteria()         criteria.extendedObjectClassName = 'com.axeda.drm.sdk.device.DataItem'         criteria.internalObjectId = dataItem.id.value         criteria.extendedClientKey = "ExtendedDataItem!${dataItem.name}"         def eo       if (searchcriteria != null){             criteria.propertySearchCriteria = advancedPropertySearch(searchcriteria)         }        def queryResult = eoSvc.findExtendedObjects(criteria,  -1, -1, null)        if (queryResult.size() == 0 || queryResult == null){               eo = new ExtendedObject()               eo.internalObjectId = dataItem.id.value               eo.extendedObjectType = eoSvc.findExtendedObjectTypeByClassname('com.axeda.drm.sdk.device.DataItem')               eo.externalClientKey = "ExtendedDataItem!${dataItem.name}"               eo = eoSvc.createExtendedObject(eo)             searchcriteria += [eotype: "ExtendedObject"]             createProperties(eoSvc, eo, searchcriteria)          }         else eo = queryResult.first()        return eo     }    def fetchExtendedDataItem(DataItem dataItem) {         def criteria = new ExtendedObjectSearchCriteria()         criteria.extendedObjectClassName = "com.axeda.drm.sdk.device.DataItem"         criteria.extendedClientKey = "ExtendedDataItem!${dataItem.name}"         criteria.internalObjectId = dataItem.id.value        def queryResult = eoSvc.findExtendedObjects(criteria,  -1, -1, null)         return queryResult     }    def fetchExtendedObjectsAdvancedCriteria(DataItem dataItem, Map searchcriteria, String classname, String uniqueKey) {         def criteria = new ExtendedObjectSearchCriteria()         criteria.extendedObjectClassName = "com.axeda.drm.sdk.device.DataItem"         criteria.extendedClientKey = "ExtendedDataItem!${dataItem.name}"         criteria.internalObjectId = dataItem.id.value        if (searchcriteria != null){             criteria.propertySearchCriteria = advancedPropertySearch(searchcriteria)         }        def queryResult = eoSvc.findExtendedObjects(criteria,  -1, -1, null)         return queryResult     }    def fetchExtendedObject(User user, Map searchcriteria) {         def criteria = new ExtendedObjectSearchCriteria()         criteria.extendedObjectClassName = "com.axeda.drm.sdk.user.User"         criteria.extendedClientKey = "ExtendedObject!${user.username}"         criteria.internalObjectId = user.id.value         criteria.propertySearchCriteria = exactdatePropertySearch(searchcriteria)         def queryResult = eoSvc.findExtendedObjects(criteria, -1, -1, null)         return queryResult     }    def addExtendedObject(String eoTypeName, Long referenceId, String referenceName, Map objectProperties) {       def eo = new ExtendedObject()       eo.internalObjectId = referenceId       eo.extendedObjectType = eoSvc.findExtendedObjectTypeByClassname(eoTypeName)       eo.externalClientKey = referenceName       eo = eoSvc.createExtendedObject(eo)       eo = createProperties(eoSvc, eo, objectProperties)        return eo     }    def updateExtendedObject(ExtendedObject ExtendedObject, ExtendedObject ExtendedObject) {                 def ExtendedObjectprop = getProperties(ExtendedObject)                 def ExtendedObjectprop = getProperties(ExtendedObject)                 def newproperties = [:]                 if (ExtendedObjectprop.timestamp != null){                 def ExtendedObjecttype = ExtendedObjectprop.lasttype                 def ExtendedObjecttime = ExtendedObjectprop.lasttime                 if (ExtendedObjecttype == null){                     newproperties["lasttype"] = ExtendedObjectprop.type                     newproperties["lasttime"] = ExtendedObjectprop.timestamp                 }                 else {                     def oldtime = Long.valueOf(ExtendedObjecttime)                     def total = (Long.valueOf(ExtendedObjectprop.timestamp) - oldtime)/1000                     // illustrating getPropertyByName /                     def lasttype = ExtendedObject.getPropertyByName("lasttype")                     lasttype.setValue(ExtendedObjectprop.type)                     def lasttime = ExtendedObject.getPropertyByName("lasttime")                     lasttime.setValue(ExtendedObjectprop.timestamp)                     updateProperty(lasttype)                     updateProperty(lasttime)                     if (ExtendedObjectprop.containsKey(ExtendedObjecttype + "_total")==false)                         {                             newproperties[ExtendedObjecttype + "_total"] = total                         }                     else {                         def totalprop = ExtendedObject.getPropertyByName(ExtendedObjecttype + "_total")                         def lasttotal = Double.valueOf(totalprop.value)                         totalprop.setValue(String.valueOf(Double.valueOf(total) + lasttotal))                         updateProperty(totalprop)                     }                 }                 if (newproperties.size() > 0){                     ExtendedObject = createProperties(eoSvc, ExtendedObject, newproperties)                 }         }         return ExtendedObject    }     def getProperties(ExtendedObject object){         def result = object.properties.inject([:]) { target, property ->               target += [(property.propertyType.name): castPropertyValueToDefinedType(property)]             }         return result     }     def formatProperties(Map properties){         def result = properties.collect { property, value ->                [type: property, time: value]             }         return result     } def updateProperty(Property property){         eoSvc.updateProperty(property)         return property }     //default string version     def createProperties (ExtendedObjectService eoSvc, ExtendedObject object, Map properties) {         return createProperties(eoSvc, object, properties, PropertyDataType.String)     }     //WARNING: PDT may not work if it's a Date; it hasn't been tested.     def createProperties (ExtendedObjectService eoSvc, ExtendedObject object, Map properties, PropertyDataType PDT) {        // http://groovy.codehaus.org/Operators#Operators-ElvisOperator         PDT = PDT?:PropertyDataType.String         properties.each { k , v ->             def property = new com.axeda.platform.sdk.v1.services.extobject.Property()             def propertytype = object.extendedObjectType.getPropertyTypeByName(k.toString())             if (propertytype == null){                 def newPropertyType = new PropertyType()                 newPropertyType.name = k                 newPropertyType.dataType = PDT                 newPropertyType.extendedObjectType = object.extendedObjectType                 eoSvc.createPropertyType(newPropertyType)                 property.propertyType = newPropertyType             } else { property.propertyType = propertytype }             property.value = (PDT == PropertyDataType.Date && v instanceof Date?v.format("yyyy-MM-ddTHH:mm:ssZ"):v.toString())             eoSvc.createPropertyOnExtendedObject(object, property)         }         object = findExtendedObjectById(object.id)         return object     }     def findExtendedObjectById(Long id){         def criteria = new ExtendedObjectSearchCriteria()         criteria.id = id         def queryResult = eoSvc.findExtendedObjects(criteria,  -1, -1, null)         return queryResult.first() }     def findObjectByNextPropertyValue (ExtendedObject object, Map criteria, PropertySearchCriteria psCriteria){        Property prop = object.getPropertyByName(criteria.incrementName)        def incrementedProp = prop.value.toInteger() + criteria.incrementValue.toInteger()            psCriteria.setPropertyExpression(            PropertyExpressionFactory.and(                     psCriteria.propertyExpression,                     PropertyExpressionFactory.eq(criteria.incrementName, incrementedProp.toString())                    )             )        def eoCriteria = new ExtendedObjectSearchCriteria()         eoCriteria.extendedObjectClassName = "com.axeda.drm.sdk.user.User"         eoCriteria.internalObjectId = object.internalObjectId         eoCriteria.propertySearchCriteria = psCriteria        def queryResult = eoSvc.findExtendedObjects(eoCriteria,  -1, -1, null)        return queryResult.first()     }     def advancedPropertySearch (Map searchcriteria){           def propCriteria = new PropertySearchCriteria()        propCriteria.setPropertyExpression(            PropertyExpressionFactory.and(                PropertyExpressionFactory.eq("year", searchcriteria.year),                     PropertyExpressionFactory.and(                         PropertyExpressionFactory.eq("month", searchcriteria.month),                                     PropertyExpressionFactory.eq("date", searchcriteria.date)                             )                    )             )         return propCriteria     }   static def extendedObjectToMap(ExtendedObject object) {     Map result = [:]     result.extendedObjectType = object?.extendedObjectType?.className     result.extendedObjectTypeId = object?.extendedObjectTypeId     result.id = object?.id     result.externalClientKey = object?.externalClientKey     result.internalObjectId = object?.internalObjectId     // build up the properties of the ExtendedObject.     result.properties = object?.properties.inject([:]) { target, property ->       target += [(property?.propertyType?.name): castPropertyValueToDefinedType(property)]     }     return result   }       static def extendedObjectTypeToMap(ExtendedObjectType objectType) {         Map result = [:]         result.className = objectType.className         result.id = objectType.id         result.displayName  = objectType.displayName         result.userDefined = objectType.userDefined         result.description = objectType.description         result.properties = objectType.propertyTypes.inject([]) { List list, PropertyType propertyType ->           list << [                   name: propertyType.name,                   id: propertyType.id,                   description: propertyType.description,                   dataType: propertyType.dataType.toString(),                   extendedObjectType: propertyType.extendedObjectType.className           ]         }         return result       }       private static def castPropertyValueToDefinedType(Property property) {         switch(property.propertyType.dataType) {           case PropertyDataType.Boolean:             return property.value as Boolean           case PropertyDataType.Date:             Calendar calendar = javax.xml.bind.DatatypeConverter.parseDateTime(property.value)             return calendar.getTime()           case PropertyDataType.Double:             return property.value as Double           case PropertyDataType.Integer:             return property.value as Long           case PropertyDataType.String:           default:             return property.value         }       }       static def removeExtendedObject(Long itemId) {         def eoSvc = new ServiceFactory().getExtendedObjectService()         eoSvc.deleteExtendedObject(itemId)       }       def removeUserExtendedObjects(User user) {         def queryresult = fetchUserExtendedObjects(user)         queryresult.each{                 eoSvc.deleteExtendedObject(it.id)         }       }      ExtendedObjectType findExtendedObjectType(String typeName) {     ExtendedObjectType userObjType = eoSvc.findExtendedObjectTypeByClassname(typeName)     return userObjType   }   ExtendedObjectType findOrCreateExtendedObjectType(String typeName) {     ExtendedObjectType type = findExtendedObjectType(typeName)     if (type == null) {       type = new ExtendedObjectType()       type.className = typeName       type.displayName = typeName       type.description = "Autocreated type for $typeName"       type = eoSvc.createExtendedObjectType(type)     }     return type   }