cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

ThingWorx Navigate is now Windchill Navigate Learn More

IoT & Connectivity Tips

Sort by:
Hello everyone,   Following a recent  experience, I felt it was important to share my insights with you. The core of this article is to demonstrate how you can format a Flux request in ThingWorx and post it to InfluxDB, with the aim of reporting the need for performance in calculations to InfluxDB. The following context is renewable energy. This article is not about Kepware neither about connecting to InfluxDB. As a prerequisite, you may like to read this article: Using Influx to store Value Stream properties from... - PTC Community     Introduction   The following InfluxDB usage has been developed for an electricity energy provider.   Technical Context Kepware is used as a source of data. A simulation for Wind assets based on excel file is configured, delivering data in realtime. SQL Database also gather the same data than the simulation in Kepware. It is used to load historical data into InfluxDB, addressing cases of temporary data loss. Once back online, SQL help to records the lost data in InfluxDB and computes the KPIs. InfluxDB is used to store data overtime as well as calculated KPIs. Invoicing third party system is simulated to get electricity price according time of the day.   Orchestration of InfluxDB operations with ThingWorx ThingWorx v9.4.4 Set the numeric property to log Maintain control over execution logic Format Flux request with dynamic inputs to send to Influx DB  InfluxDB Cloud v2 Store logged property Enable quick data read Execute calculation Note: Free InfluxDB version is slower in write and read, and only 30 days data retention max.     ThingWorx model and services   ThingWorx context Due to the fact relevant numeric properties are logged overtime, new KPIs are calculated based on the logged data. In the following example, each Wind asset triggered each minute a calculation to get the monetary gain based on current power produced and current electricity price. The request is formated in ThingWorx, pushed and executed in InfluxDB. Thus, ThingWorx server memory is not used for this calculation.   Services breakdown CalculateMonetaryKPIs Entry point service to calculate monetary KPIs. Use the two following services: Trigger the FormatFlux service then inject it in Post service. Inputs: No input Output: NOTHING FormatFlux _CalculateMonetaryKPI Format the request in Flux format for monetary KPI calculation. Respect the Flux synthax used by InfluxDB. Inputs: bucketName (STRING) thingName (STRING) Output: TEXT PostTextToInflux Generic service to post the request to InfluxDB, whatever the request is Inputs: FluxQuery (TEXT) influxToken (STRING) influxUrl (STRING) influxOrgName (STRING) influxBucket (STRING) thingName (STRING) Output: INFOTABLE   Highlights - CalculateMonetaryKPIs Find in attachments the full script in "CalculateMonetaryKPIs script.docx". Url, token, organization and bucket are configured in the Persitence Provider used by the ValueStream. We dynamically get it from the ValueStream attached to this thing. From here, we can reuse it to set the inputs of two other services using “MyConfig”.   Highlights - FormatFlux_CalculateMonetaryKPI Find in attachments the full script in "FormatFlux_CalculateMonetaryKPI script.docx". The major part of this script is a text, in Flux synthax, where we inject dynamic values. The service get the last values of ElectricityPrice, Power and Capacity to calculate ImmediateMonetaryGain, PotentialMaxMonetaryGain and PotentialMonetaryLoss.   Flux logic might not be easy for beginners, so let's break down the intermediate variables created on the fly in the Flux request. Let’s take the example of the existing data in the bucket (with only two minutes of values): _time _measurement _field _value 2024-07-03T14:00:00Z WindAsset1 ElectricityPrice 0.12 2024-07-03T14:00:00Z WindAsset1 Power 100 2024-07-03T14:00:00Z WindAsset1 Capacity 150 2024-07-03T15:00:00Z WindAsset1 ElectricityPrice 0.15 2024-07-03T15:00:00Z WindAsset1 Power 120 2024-07-03T15:00:00Z WindAsset1 Capacity 160   The request articulates with the following steps: Get source value Get last price, store it in priceData _time ElectricityPrice 2024-07-03T15:00:00Z 0,15 Get last power, store it in powerData _time Power 2024-07-03T15:00:00Z 120 Get last capacity, store it in capacityData _time Capacity 2024-07-03T15:00:00Z 160 Join the three tables *Data on the same time. Last values of price, power and capacity maybe not set at the same time, so final joinedData may be empty. _time ElectricityPrice Power Capacity 2024-07-03T14:00:00Z 0,15 120 160 Perform calculations gainData store the result: ElectricityPrice * Power _time _measurement _field _value 2024-07-03T15:00:00Z WindAsset1 ImmediateMonetaryGain 18 maxGainData store the result: ElectricityPrice * Capacity lossData store the result: ElectricityPrice * (Capacity – Power) Add the result to original bucket   Highlights - PostTextToInflux Find in attachments the full script in "PostTextToInflux script.docx". Pretty straightforward script, the idea is to have a generic script to post a request. The header is quite original with the vnd.flux content type Url needs to be formatted according InfluxDB API     Well done!   Thanks to these steps, calculated values are stored in InfluxDB. Other services can be created to retrieve relevant InfluxDB data and visualize it in a mashup.     Last comment It was the first time I was in touch with Flux script, so I wasn't comfortable, and I am still far to be proficient. After spending more than a week browsing through InfluxDB documentation and running multiple tests, I achieved limited success but nothing substantial for a final outcome. As a last resort, I turned to ChatGPT. Through a few interactions, I quickly obtained convincing results. Within a day, I had a satisfactory outcome, which I fine-tuned for relevant use.   Here is two examples of two consecutive ChatGPT prompts and answers. It might need to be fine-tuned after first answer.   Right after, I asked to convert it to a ThingWorx script format:   In this last picture, the script won’t work. The fluxQuery is not well formatted for TWX. Please, refer to the provided script "FormatFlux_CalculateMonetaryKPI script.docx" to see how to format the Flux query and insert variables inside. Despite mistakes, ChatGPT still mainly provides relevant code structure for beginners in Flux and is an undeniable boost for writing code.  
View full tip
Learn how to use the DBConnection building block to create your own DB tables in ThingWorx.
View full tip
Timers and Schedulers can also be created and configured programmatically via custom services. The following service, which can be created on any Thing, will create a new Timer using the following Inputs:         // create new Thing var params = { name: ThingName /* STRING */, description: undefined /* STRING */, thingTemplateName: "Timer" /* THINGTEMPLATENAME */, tags: undefined /* TAGS */ }; Resources["EntityServices"].CreateThing(params); // read initial configuration // result: INFOTABLE var configtable = Things[ThingName].GetConfigurationTable({tableName: "Settings"}); // update configuration with service parameters configtable.updateRate = updateRate configtable.runAsUser = user // set new configuration table var params = { configurationTable: configtable /* INFOTABLE */, persistent: true /* BOOLEAN */, tableName: "Settings" /* STRING */ }; Things[ThingName].SetConfigurationTable(params);   This code is an example which could also be used to create a new Scheduler. The configuration table for a Timer has the following attributes: updateRate enabled runAsUser The configuration table for a Scheduler has the following attributes: schedule enabled runAsUser  
View full tip
This example provides the ability to generate a simple entity structure and some historical data for each entity. Historical data is run through a ThingWorx service to generate histogram data for display in a bar chart.  The provided ThingWorx entities and PDF document provide the example as well as documentation.
View full tip
Hi,   I have just been playing around with the Edge MicroServer on a couple of Raspberry Pi's.  I obviously want them connected to ThingWorx, however they stop when the SSH session closes which isn't ideal.  I thought about doing something really quick and dirty using 'nohup', but this could have lead to many running processes, and wouldn't still not have started automatically when the Pi booted.  So, I did it right instead using init.d daemon configurations.   There are two files; one for the EMS and one for the Lua Script Resource.  You need to put these in /etc/init.d, then make sure they are owned by root.   sudo chown root /etc/init.d/thingworx* sudo chgrp root /etc/init.d/thingworx*   You'll need to modify the paths in the first lines of these files to match where you have your microserver folder.  Then you need to update the init.d daemon configs and enable the services.   sudo update-rc.d thingworx-ems defaults sudo update-rc.d thingworx-ems enable sudo update-rc.d thingworx-lsr defaults sudo update-rc.d thingworx-lsr enable   You can then start (stop, etc.) like this:   sudo service thingworx-ems start sudo service thingworx-lsr start   They should both also start automatically after a reboot.   Regards,   Greg
View full tip
How to input Database User Credentials at RunTime. This Blog considers that you have already imported the Database Extension and Configured the Thing Template. If you have not done this already please see Steps to connecting to your Relational Database first. Steps: Create a Database Thing template with correct configuration. Example configuration for MySql Database: jDBCDriverClass: com.mysql.jdbc.Driver jDBCConnectionURL: jdbc:mysql://127.0.0.1:3306/<DatabaseNameHere>?allowMultiQueries=true connectionValidationString: SELECT NOW() maxConnections: 100 userName: <DataBaseUserNameHere> password: <DataBasePasswordHere> Create any Generic Thing and add a service to create thing based on the Thing template created in Step 1. Example: // NewDataBaseThingName is the String input for name the database thing to be created. // MySqlServerUpdatedConfiguration is the Thing template with correct configuration var params = {      name: NewDataBaseThingName /* STRING */,      description: NewDataBaseThingName /* STRING */,     thingTemplateName: "MySqlServerUpdatedConfiguration" /* THINGTEMPLATENAME */,     tags: undefined /* TAGS */ }; // no return Resources["EntityServices"].CreateThing(params); Add code to enable and then restart the above thing using EnableThing() and RestartThing() service. Example Things[NewDataBaseThingName].EnableThing(); Things[NewDataBaseThingName].RestartThing(); Test and confirm that the Database Thing services runs as expected. Now Create a DataShape with following Fields: jDBCDriverClass: STRING jDBCConnectionURL: STRING connectionValidationString: STRING maxConnections: NUMBER userName: STRING password: PASSWORD Now in the Generic Thing created in Step 1 add code to update the configuration settings of DataBase Thing. Make sure JDBC Driver Class Name should never be changed. If different database connection is required use different Thing Template. Also, add code to restart the DataBase Thing using RestartThing() service. Example: var datashapeParams = {     infoTableName : "InfoTable",     dataShapeName : "DatabaseConfigurationDS" }; // CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(DatabaseConfigurationDS) var config = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(datashapeParams); var passwordParams = {         data: "DataBasePasswordHere" /* STRING */ }; // DatabaseConfigurationDS entry object var newEntry = new Object(); newEntry.jDBCDriverClass= "com.mysql.jdbc.Driver"; // STRING newEntry.jDBCConnectionURL = "jdbc:mysql://127.0.0.1:3306/<DatabaseNameHere>?allowMultiQueries=true"; // STRING newEntry.connectionValidationString = "SELECT NOW()"; // STRING newEntry.maxConnections = 100; // NUMBER newEntry.userName = "DataBaseUserNameHere"; // STRING newEntry.password = Resources["EncryptionServices"].EncryptPropertyValue(passwordParams); // PASSWORD config.AddRow(newEntry); var configurationTableParams = { configurationTable: config /* INFOTABLE */, persistent: true /* BOOLEAN */, tableName: "ConnectionInfo" /* STRING */ }; // ThingNameForConfigurationUpdate is the input string for Thing Name whose configuration needs to be updated. // no return Things[ThingNameForConfigurationUpdate].SetConfigurationTable(configurationTableParams); Things[ThingNameForConfigurationUpdate].RestartThing(); Test and confirm that the Database Thing services runs as expected.
View full tip
Back in 2018 an interesting capability was added to ThingWorx Foundation allowing you to enable statistical calculation of service and subscription execution.   We typically advise customers to approach this with caution for production systems as the additional overhead can be more than you want to add to the work the platform needs to handle.  This said, these statistics is used consciously can be extremely helpful during development, testing, and troubleshooting to help ascertain which entities are executing what services and where potential system bottlenecks or areas deserving performance optimization may lie.   Although I've used the Utilization Subsystem services for statistics for some time now, I've always found that the Composer table view is not sufficient for a deeper multi-dimensional analysis.  Today I took a first step in remedying this by getting these metrics into Excel and I wanted to share it with the community as it can be quite helpful in giving developers and architects another view into their ThingWorx applications and to take and compare benchmarks to ensure that the operational and scaling is happening as was expected when the application was put into production.   Utilization Subsystem Statistics You can enable and configure statistics calculation from the Subsystem Configuration tab.  The help documentation does a good job of explaining this so I won't mention it here.  Base guidance is not to use Persisted statistics, nor percentile calculation as both have significant performance impacts.  Aggregate statistics are less resource intensive as there are less counters so this would be more appropriate for a production environment.  Specific entity statistics require greater resources and this will scale up as well with the number of provisioned entities that you have (ie: 1,000 machines versus 10,000 machines) whereas aggregate statistics will remain more constant as you scale up your deployment and its load.   Utilization Subsystem Services In the subsystem Services tab, you can select "UtilizationSubsystem" from the filter drop down and you will see all of the relevant services to retrieve and reset the statistics.     Here I'm using the GetEntityStatistics service to get entity statistics for Services and Subscriptions.     Giving us something like this.      Using Postman to Save the Results to File I have used Postman to do the same REST API call and to format the results as HTML and to save these results to file so that they can be imported into Excel.   You need to call '/Thingworx/Subsystems/UtilizationSubsystem/Services/GetEntityStatistics' as a POST request with the Content-Type and Accept headers set to 'application/xml'.  Of course you also need to add an appropriately permissioned and secured AppKey to the headers in order to authenticate and get service execution authorization.     You'll note the Export Results > Save to a file menu over on the right to get your results saved.   Importing the HTML Results into Excel As simple as I would like to hope that getting a standard web formatted file into Excel should be, it didn't turn out to be as easy as I would have hoped and so I have to switch over to Windows to take advantage of Power Query.   From the Data ribbon, select Get Data > From File > From XML.  Then find and select the HTML file saved in the previous step.     Once it has loaded the file and done some preparation, you'll need to select the GetEntityStatistics table in the results on the left.  This should display all of the statistics in a preview table on the right.     Once the query completed, you should have a table showing your statistical data ready for... well... slicing and dicing.     The good news is that I did the hard part for you, so you can just download the attached spreadsheet and update the dataset with your fresh data to have everything parsed out into separate columns for you.     Now you can use the column filters to search for entity or service patterns or to select specific entities or attributes that you want to analyze.  You'll need to later clear the column filters to get your whole dataset back.     Updating the Spreadsheet with Fresh Data In order to make this data and its analysis more relevant, I went back and reset all of the statistics and took a new sample which was exactly one hour long.  This way I would get correct recent min/max execution time values as well as having a better understanding of just how many executions / triggers are happening in a one hour period for my benchmark.   Once I got the new HTML file save, I went into Excel's Data ribbon, selected a cell in the data table area, and clicked "Queries & Connections" which brought up the pane on the right which shows my original query.     Hovering over this query, I'm prompted with some stuff and I chose "Edit".     Then I clicked on the tiny little gear to the right of "Source" over on the pane on the right side.     Finally I was able to select the new file and Power Query opened it up for me.     I just needed to click "Close & Load" to save and refresh the query providing data to the table.     The only thing at this point is that I didn't have my nice little sparklines as my regional decimal character is not a period - so I selected the time columns and did a "Replace All" from '.' to ',' to turn them into numbers instead of text.     Et Voila!   There you have it - ready to sort, filter, search and review to help you better understand which parts of your application may be overly resource hungry, or even to spot faulty equipment that may be communicating and triggering workflows far more often than it should.   Specific vs General Depending on the type of analysis that you're doing you might find that the aggregate statistics are a better option.  As they'll be far, far less that the entity specific statistics they'll do a better job of giving you a holistic view of the types of things that are happening with your ThingWorx applications execution.   The entity specific data set that I'm showing here would be a better choice for troubleshooting and diagnostics to try to understand why certain customers/assets/machines are behaving strangely as we can specifically drill into these stats.  Keep in mind however that you should then compare these findings with the general baseline to see how this particular asset is behaving compared to the whole fleet.   As a size guideline - I did an entity specific version of this file for a customer with 1,000 machines and the Excel spreadsheet was 7Mb compared to the 30kb of the one attached here and just opening it and saving it was tough for Excel (likely due to all of my nested formulas).  Just keep this in mind as you use this feature as there is memory overhead meaning also garbage collection and associated CPU usage for such.
View full tip
Sometimes you need the values from different ThingTemplate members in ONE grid. Therefore it would be great, if you can join 2 "GetImplementedThingsWithData" results into a common one. Here a script that works generally as long as you don't mess with datatypes on same column names. I'm very interested, if someone can find a much easier solution. The Union function was the only one I found suited for the task, but this needs preparation of the infotables upfront. Input: Table1 :Infotable Table2: Infotable Output: Infotable Here the "Snippet": // Define params for an Infotable to hold column names var params = {   infoTableName: "field" /* STRING */ }; // Define column 1 var newField = new Object(); newField.name = "field"; newField.baseType = 'STRING'; // Two 1 columns Infotables to store the field definition; var field1 = Resources["InfoTableFunctions"].CreateInfoTable(params); field1.AddField(newField); var field2 = Resources["InfoTableFunctions"].CreateInfoTable(params); field2.AddField(newField); // Define the cell to add to Infotable var myField = new Object(); myField.field = ""; myField.baseType = "STRING"; // Loop through Table1 var dataShapeFields = Table1.dataShape.fields; for (var fieldName in dataShapeFields) {   logger.debug('field1 name is ' + dataShapeFields[fieldName].name);     myField.field = dataShapeFields[fieldName].name;    field1.AddRow(myField); } // Loop through Table2 var dataShapeFields = Table2.dataShape.fields; for (var fieldName in dataShapeFields) {   logger.debug('field2 name is ' + dataShapeFields[fieldName].name);    myField.field = dataShapeFields[fieldName].name;    field2.AddRow(myField); } // Using inner join functionality to filter only the values that exist in both var params = {   columns2: "field" /* STRING */,   columns1: "field" /* STRING */, joinType: "INNER" /* STRING */,   t1: field1 /* INFOTABLE */, t2: field2 /* INFOTABLE */,   joinColumns1: "field" /* STRING */,   joinColumns2: "field" /* STRING */ }; var commonFields = Resources["InfoTableFunctions"].Intersect(params); // Loop over the result to build a search string var commonColumns = ""; var tableLength = commonFields.rows.length; for (var x = 0; x < tableLength; x++) {   var row = commonFields.rows ;   commonColumns = commonColumns + row.field + ","; } // Reduce Table1 to match only common columns var params = { t: Table1 /* INFOTABLE */, columns: commonColumns /* STRING */ }; var result1 = Resources["InfoTableFunctions"].Distinct(params); // Reduce Table2 to match only common columns var params = {   t: Table2 /* INFOTABLE */,   columns: commonColumns /* STRING */ }; var result2 = Resources["InfoTableFunctions"].Distinct(params); // At the END JOIN the tables together (does not work if colums are different) var params = {   t1: result1 /* INFOTABLE */,   t2: result2 /* INFOTABLE */ }; var result = Resources["InfoTableFunctions"].Union(params);
View full tip
Analytics projects typically involve using the Analytics API rather than the Analytics Builder to accomplish different tasks. The attached documentation provides examples of code snippets that can be used to automate the most common analytics tasks on a project such as: Creating a dataset Training a Model Real time scoring predictive and prescriptive Retrieving the validation metrics for a model Appending additional data to a dataset Retraining the model The documentation also provides examples that are specific to time series datasets. The attached .zip file contains both the document as well as some entities that you need to import in ThingWorx to access the services provided in the examples. 
View full tip
I've had a lot of questions over the years working with Azure IoT, Kepware, and ThingWorx that I really struggled getting answers to. I was always grateful when someone took the time to help me understand, and now it is time to repay the favour.   People ask me many things about Azure (in a ThingWorx context), and one of the common ones has been about MQTT communications from Kepware to ThingWorx using IoT Hub. Recently the topic has come up again as more and more of the ThingWorx expert community start to work with Azure IoT. Today, I took the time to build, test, validate, and share an approach and utilities to do this in cases where the Azure Industrial IoT OPC UA integration is overkill or simply a step later in the project plan. Enjoy!   End to end Integration of Kepware to ThingWorx using MQTT over Azure IoT (YoutTube 45 minute deep-dive)   ThingWorx entities for import (ThingWorx 9.0)   This approach can be quite good for a simple demo if you have a Kepware Integrator or Kepware Enterprise license, but the use of IoT Gateway for many servers and tags can be quite costly.   Those looking to leverage Azure IoT Hub for MQTT integration to ThingWorx would likely also find this recorded session and shared utilities quite helpful.   Cheers, Greg
View full tip
For a recent project, I was needing to find all of the children in a Network Hierarchy of a particular template type... so I put together a little script that I thought I'd share. Maybe this will be useful to others as well.   In my situation, this script lived in the Location template. This was useful so that I could find all the Sensor Things under any particular node, no matter how deep they are.   For example, given a network like this: Location 1 Sensor 1 Location 1A Sensor 2 Sensor 3 Location 1AA Sensor 4 Location 1B Sensor 5 If you run this service in Location 1, you'll get an InfoTable with these Things: Sensor 1 Sensor 2 Sensor 3 Sensor 4 Sensor 5 From Location 1A: Sensor 2 Sensor 3 Sensor 4 From Location 1AA: Sensor 4 From Location 1B: Sensor 5   For this service, these are the inputs/outputs: Inputs: none Output: InfoTable of type NetworkConnection   // CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(AlertSummary) let result = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape({ infoTableName : "InfoTable", dataShapeName : "NetworkConnection" }); // since the hierarchy could contain locations or sensors, need to recursively loop down to get all the sensors function findChildrenSensors(thingName) { let childrenThings = Networks["Hierarchy_NW"].GetChildConnections({ name: thingName /* STRING */ }); for each (var row in childrenThings.rows) { // row.to has the name of the child Thing if (Things[row.to].IsDerivedFromTemplate({thingTemplateName: "Location_TT"})) { findChildrenSensors(row.to); } else if (Things[row.to].IsDerivedFromTemplate({thingTemplateName: "Sensor_TT"})) { result.AddRow(row); } } } findChildrenSensors(me.name);    
View full tip
Please find here an Labview implementation to connect to Thingworx via RestCalls. Have Fun using it. Any Feedback is appreciated. https://github.com/Seppel1985/LabVIEW_TWX_RestAPI
View full tip
Attached (as PDF) are some steps to quickly get started with the Thingworx MQTT Extension so that you can subscribe / publish topics.
View full tip
Getting Started on the ThingWorx Platform Learning Path   Learn hands-on how ThingWorx simplifies the end-to-end process of implementing IoT solutions.   NOTE: Complete the following guides in sequential order. The estimated time to complete this learning path is 210 minutes.   Get Started with ThingWorx for IoT   Part 1 Part 2 Part 3 Part 4 Part 5 Data Model Introduction Configure Permissions Part 1 Part 2 Build a Predictive Analytics Model  Part 1 Part 2
View full tip
Design Your Data Model Guide Part 1   Overview   This project will introduce the process of taking your IoT solution from concept to design. Following the steps in this guide, you will create a solution that doesn’t need to be constantly revamped, by creating a comprehensive Data Model before starting to build and test your solution. We will teach you how to utilize a few proposed best practices for designing the ThingWorx Data Model and provide some prescriptive methods to help you generate a high-quality framework that meets your business needs. NOTE: This guide’s content aligns with ThingWorx 9.3. The estimated time to complete ALL 3 parts of this guide is 60 minutes. All content is relevant but there are additional tools and design patterns you should be aware of. Please go to this link for more details.    Step 1: Data Model Methodology   We will start by outlining the overall process for the proposed Data Model Methodology.       Step Description 1 User Stories Identify who will use the application and what information they need. By approaching the design from a User perspective, you should be able to identify what elements are necessary for your system. 2 Data Sources Identify the real-world objects or systems which you are trying to model. To create a solid design, you need to identify what the “things” are in your system and what data or functionality they expose. 3 Model Breakdown Compose a representative model of modular components to enable uniformity and reuse of functionality wherever possible. Break down user requirements and data, identifying how the system will be modeled in Foundation. 4 Data Strategy Identify the sources of data, then evaluate how many different types of data you will have, what they are, and how your data should be stored. From that, you may determine the data types and data storage requirements. 5 Business Logic Strategy Examine the functional needs, and map them to your design for proper business logic implementation. Determine the business logic as a strategic flow of data, and make sure everything in your design fits together in logical chunks. 6 User Access Strategy Identify each user's access and permission levels for your application. Before you start building anything, it is important to understand the strategy behind user access. Who can see or do what? And why? NOTE: Due to the length of this subject, the ThingWorx Data Model Methodology has been divided into multiple parts. This guide focuses on the first three steps = User Stories, Data Sources, and Model Breakdown. Guides covering the last three steps are linked in the final Next Steps page.    Step 2: User Stories     With a user-based approach to design, you identify requirements for users at the outset of the process. This increases the likelihood of user satisfaction with the result. Utilizing this methodology, you consider each type of user that will be accessing your application and determine their requirements according to each of the following two categories: Category Requirement Details Functionality Determine what the user needs to do. This will define what kind of Services and Subscriptions will need to be in the system and which data elements and Properties must be gathered from the connected Things. Information What information do they need? Examine the functional requirements of the user to identify which pieces of information the users need to know in order to accomplish their responsibilities.   Factory Example   Let’s revisit our Smart Factory example scenario. The first step of the User Story phase of the design process is to identify the potential users of your system. In this example scenario, we have defined three different types of users for our solution: Maintenance Operations Management Each of these users will have a different role in the system. Therefore, they will have different functional and informational needs.   Maintenance   It is the maintenance engineer’s job to keep machines up and running so that the operator can assemble and deliver products. To do this well, they need access to granular data for the machine’s operating status to better understand healthy operation and identify causes of failure. They also need to integrate their maintenance request management system to consolidate their efforts and to create triggers for automatic maintenance requests generated by the connected machines. Required Functionality Get granular data values from all assets Get a list of maintenance requests Update maintenance requests Set triggers for automatic maintenance request generation Automatically create maintenance requests when triggers have been activated Required Information Granular details for each asset to better understand healthy asset behavior Current alert status for each asset When the last maintenance was performed on an asset When the next maintenance is scheduled for an asset Maintenance request for information, including creation date, due date, progress notes   Operations   The operator’s job is to keep the line running and make sure that it’s producing quality products. To do this, operators must keep track of how well their line is running (both in terms of speed and quality). They also need to be able to file maintenance requests when they have issues with the assets on their line. Required Functionality File maintenance request Get quality data from assets on their line Get performance data for the whole line Get a prioritized list of production orders for their line Create maintenance requests Required Information Individual asset performance metrics Full line performance metrics Product quality readings   Management   The production manager oversees the dispatch of production orders and ensures quotas are being met. Managers care about the productivity of all lines and the status of maintenance requests. Required Functional Create production orders Update production orders Cancel production orders Access line productivity data Elevate maintenance request priority Required Information Production line productivity levels (OEE) List of open maintenance requests   Step 3: Data Sources – Thing List     Thing List   Once you have identified the users' requirements, you'll need to determine what parts of your system must be connected. These will be the Things in your solution. Keep in mind that a Thing can represent many different types of connected endpoints. Here are some examples of possible Things in your system: Devices deployed in the field with direct connectivity or gateway-connectivity to Foundation Devices deployed in the field through third-party device clouds Remote databases Connections to external business systems (e.g., Salesforce.com, Weather.com, etc.)   Factory Example   In our Smart Factory example, we have already identified the users of the system and listed requirements for each of those users. The next step is to identify the Things in our solution. In our example, we are running a factory floor with multiple identical production lines. Each of these lines has multiple different devices associated with it. Let’s consider each of those items to be a connected Thing. Things in each line: Conveyor belt x 2 Pneumatic gate Robotic Arm Quality Check Camera Let's also assume we already have both a Maintenance Request System and a Production Order System that are in use today. To add this to our solution, we want to build a connector between Foundation and the existing system. These connectors will be Things as well. Internal system connection Thing for Production Order System Internal system connection Thing for Maintenance Request System NOTE: It is entirely possible to have scenarios in which you want to examine more granular-level details of your assets. For example, the arm and the hand of the assembly robot could be represented separately. There are endless possibilities, but for simplicity's sake, we will keep the list shorter and more high-level. Keep in mind that you can be as detailed as needed for this and future iterations of your solution. However, being too granular could potentially create unnecessary complexity and data overload.    Click here to view Part 2 of this guide.
View full tip
Abstract This article explores an approach on how to optimize ThingWorx thread usage during service execution. It addresses common scenario where the Event Processor executor thread pool can become saturated, which can lead to queuing and slow responsiveness to users. This article is a proposal for configuring and optimizing services, with an example. Please, use the post comments to provide feedback or suggestions. All used capabilities are 100% ThingWorx OOTB. In the following, the acronym SETO (Service execution thread optimization) will be used to refer to entities and their capabilities.   Introduction During the execution of a service, events trigger a subscription that runs in a new thread. In addition to the best practices for timers and schedulers, SETO uses this mechanism to parallelize the execution of a service across subdivisions of a population of Things. When using ThingWorx, clients often encounter issues related to the regular execution of services such as "get data" or "update KPIs".  These services, when executed frequently, can lead to several challenges: Single thread processing: Processing occurs only in one thread, might lead to long execution time. Thread Pool Saturation: When too many services are executed simultaneously, the thread pool can become saturated, slowing down the entire system. Queue Overflows: Thread pool saturation can lead to queue overflows, where new tasks cannot be processed in time. Task Loss: Overflows can result in the loss of important tasks, affecting service reliability. Unresponsive Server: Due to overload, the server can become unresponsive, impacting user experience and operational continuity. Data processing requirements often lead to decreasing execution time without the required consideration towards impact on stability and availability of the other required services.  This post provides explanations and examples of how to think differently about how to execute data processing quickly and reliably while also focusing on the systems stability and reliability.   Why Use Service Execution Thread Optimization? To address these issues, Service Execution Thread Optimization (SETO) is essential. It allows for effective management of resource saturation and its consequences. SETO enables you to control the percent or number of used threads and the safety inter-execution delay.   Selective Execution Management Prioritized processing involves sorting tasks from oldest to newest, focusing on older tasks to prevent them from falling too far behind. However, it is crucial not to limit execution to only old tasks but to find a balance to process all eligible tasks. If all things don’t execute the service before the inter-execution delay, SETO skips them. These remaining things will be prioritized for the next timer event. Selectively orchestrating work execution allows designing your data processing workflow so that it is coherent with your various use cases functional requirements as well as being tuned for performance and your provisioned resources. This example covers functional requirements which require data to be processed within a 1-hour period, and hence sorts the processing by oldest first, and allows for skipping execution windows if the system is busy at that time. Your application of this approach may leverage a more complex approach suited to your needs that could add sensitive assets that should be processed with more urgency depending on the type or location of an asset.  This is where multiple copies of the SETO entity would adapt according to your use cases, selection criteria, work sorting, timing, thread usage configurations, etc.   Applicability and Ease of Use SETO has been designed to avoid touching the existing model as much as possible. The only prerequisite is to have a Datetime property on each Thing to track the last execution of the service, one property per service.   How to use it? Configuration of Allocated Threads Per Service 1. Create an Thing from the PTCSC.SETO.WorkProcessManager_TT ThingTemplate. 2. Configure the AllocatedThreadsPerService Table:   TS or TT: The entity type that carries the service. Accepted values are “TT” for ThingTemplate and “TS” for ThingShape TS or TT name: The name of the entity that carries the service. Service name: The name of the service you want to streamline the execution. Safety timeout: A security time value that defines a hard stop before the next timer trigger, in seconds. Minimum of 5 seconds. Example: If my timer is set to 3 minutes and my safety timeout is set to 10, the service won’t be executed on remaining things at 2 minutes and 50 seconds. The safety timeout must be less than the timer update rate. Last update property name: The name of the datetime property on things used to store the last service execution timestamp. Update older than: The minimum number of minutes since the last update before the service is triggered on an object. If set to 0, all things will be taken in account anytime. Percent available thread: The percentage of available thread to use. Value constrained from 0 to 99. This value is truncated to select a round number. Example: 13 threads available, 60% requested → 13*0,6=7,8, so 7 threads will be used. Use number of threads: Way to the select the requested number of threads. TRUE, use percent of available threads / FALSE, use the number of threads value column. Number of threads: Maximum number of threads allowed to run in parallel to process the list of things. This number should be set to leave at least a few threads free. SETO let at least on thread free, so the number of request threads might be lower or equal to zero. The remaining thread might be used by other execution outside SETO. 3. Creating a Subscription in the new Thing.   Create a Subscription to link the Timer to the Service to execute.   Result example The following example demonstrates how logging is seamlessly integrated into SETO during the execution optimization process. Logging is part of the ServiceExecution_SUB subscription in PTCSC.SETO.WorkProcessManager_TT ThingTemplate.   Important Notes Appropriate entity selection, sorting, scheduling algorithms will depend on the specific use cases and workload. SETO uses QueryImplementingThingsOptimized for better performance, so indexing is encouraged. Processing needs to address resulting status and potentially needed retries as a part of the scheduling algorithm.  You'll want to add use case specific logic which ensures the entities with failing processing are only attempted a couple of times and that their error state is notified and addressed by a system administrator. Ex: just sorting by oldest last execution time will cause anomalous non-working entities to surface to the top and taking priority over all other processing (this is an anti-pattern to avoid, requiring a circuit breaker) SETO can only trigger Services without input parameters.  You could evolve the concept to leverage passing Service parameters through JSON property event data. In this article, a Timer is used to trigger a regular event. As best practice, it is better to use a Scheduler (regular as a Timer) Indeed, when server starts, all timers will start. Scheduler trigger depends on current datetime and not on sever startup.   Conclusion and provided file By optimizing Service execution, businesses can improve the performance and reliability of their applications built on the ThingWorx Platform. Proper configuration helps prevent common issues and ensures smooth service execution. Find attached to this article the SETO entities valid from version 9.3.   I've just put resource here as more generic as it is not only the thread pool that would become saturated in the given context.  It could be many other aspects downstream which are tightly coupled like the database.     NB: thanks to Greg EVA for sharing knowledge and review. 
View full tip
Super simple widget that embeds the HTML5 audio tag, allowing MP3 files to be played and/or triggers by another mashup event.
View full tip
Data Model Introduction    Overview   This project will introduce the ThingWorx Foundation Data Model. Following the steps in this guide, you will consider data interactions based on user needs and requirements, as well as application modularity, reusability, and future updates. We will teach you how to think about a properly constructed foundation that will allow your application to be scalable, flexible, and more secure. NOTE: This guide's content aligns with ThingWorx 9.3. The estimated time to complete this guide is 30 minutes.    Step 1: Benefits   A Data Model creates a uniform representation of all items that interact with one another. There are multiple benefits to such an approach, and the ability to break up items and reuse components is considered a best practice. ThingWorx has adopted this model at a high level to represent individual components of an IoT solution. Feature Benefit Flexibility Once a model has been created, it is simple to update, modify, or remove components without needing to rework the system or retest existing components. Scalability It’s easy to clone and modify devices that are either identical or similar when changing from a Proof of Concept or Pilot Program to a Scaled Business Model. Interoperability Seamlessly plug into other applications. Collaboration A Data Model allows pre-defined links between components, meaning that various parts can be defined when designing the model so that multiple people can work on those individual parts without compromising the interoperability of the components. Seamless platform A Data Model allows for seamless integration with other systems. A properly-formed model will make it easier to create high-value IoT capabilities such as analytics, augmented/virtual reality, industrial connectivity, etc.   Step 2: Entities   Entities   Building an IoT solution in Foundation begins with defining your Data Model, the collection of Entities that represent your connected devices, business processes, and your application. Entities are the highest-level objects created and maintained in Foundation, as explained below.     Thing Shape   Thing Shapes provide a set of characteristics represented as Properties, Services, Events, and Subscriptions that are shared across a group of physical assets. A Thing Shape is best used for composition to describe relationships between objects in your model. They promote reuse of contained Properties and business logic that can be inherited by one or more Thing Templates. In Foundation, the model allows a Thing Template to implement one or more Thing Shapes, which is similar to a class definition in C++ that has multiple inheritance. When you make a change to the Thing Shape, the change is propagated to the Thing Templates and Things that implement that Thing Shape; so, maintaining the model is quick and easy.   Thing Template   Thing Templates provide base functionality with Properties, Services, Events, and Subscriptions that Thing instances use in their execution. Every Thing is created from a Thing Template. A Thing Template can extend another Thing Template. When you release a new version of a product, you simply add the additional characteristics of the version without having to redefine the entire model. This model configuration provides multiple levels of generalization of an asset. A Thing Template can derive one or more additional characteristics by implementing Thing Shapes. When you make a change to the Thing Template, the change is propagated to the Things that implement that Thing Template; so again, maintaining the model is quick and easy. A Thing Template can be used to classify the kind of a Thing or asset class or as a specific product model with unique capabilities. If you have two product models and their interaction with the solution is the same (same Properties, Services, and Events), you could model them as one Thing Template. Classifying Thing Templates is useful for aggregating Things into collections, which are useful in Mashups. You may want separate Thing Templates for indexing, searching, and future evolutions of the products   Thing   Things are representations of physical devices, assets, products, systems, people, or processes that have Properties and business logic. All Things are based on Thing Templates (inheritance) and can implement one or more Thing Shapes (composition). A Thing can have its own Properties, Services, Events, and Subscriptions and can inherit other Properties, Services, Events, and Subscriptions from its Thing Template and Thing Shape(s). How you model the interconnected Things, Thing Templates, and Thing Shapes is key to making your solution easy to develop and maintain in the future as the physical assets change. End users will interface with Things for information in applications and for reading/writing data.   Best Practice: Create a Thing Template to describe a Thing, then create an instance of that Thing Template as a Thing. This practice leverages inheritance in your model and reduces the amount of time you spend maintaining and updating your model.   Step 3: Inheritance Model   Defining Things, Thing Templates, and Thing Shapes in your Data Model allows your application to handle both simple and complex scenarios. Entity Function Thing Shapes Assemble individual components. Thing Templates Combine those components into fully functional objects. Thing Unique representation of a set of identical components defined by the Thing Template.       In this example, there is a Parent/Child model between two related Thing Templates. NOTE: Things and Thing Templates may only inherit ONE Thing Template. Both Things and Thing Templates may inherit any number of Thing Shapes. Thing Templates employ a linear-relationship, while Thing Shapes employ a modular-relationship. Any Thing or Thing Template may have any number of sub-components (i.e. Thing Shapes), but each Thing or Thing Template is just one description of one object as a whole. How you decide to compartmentalize your Data Model into Thing Shapes and Thing Templates to create the actual Things that you’ll be using is a custom design that will be specific to each implementation.   Step 4: Scenario   The ThingWorx Data Model provides a way for you to describe your connected devices and match the complexity of a real-world scenario. Things, Thing Templates, and Thing Shapes are building blocks that define your data model.     You can define the components of Things, Thing Templates, and Thing Shapes, including Properties, Services, Events, and Subscriptions. Component Definition Properties Each Property has a name, description, and a data type (Base Type). Depending on the base type, additional fields may be enabled. A simple scalar type, like a number or string, adds basic fields like default value. More complex base types have more options. Properties can be static (i.e. Model Number) or dynamic (i.e. Temperature). Services A Service is a method/function defined by a block of code that performs logic specifying actions a Thing can take. There are several implementation methods, or handlers (for example: Script, SQLQuery, and SQL command), for services depending on the template you use. The specific implementation of a user-defined Service is done via a server-side script. The Service can then be invoked through a URL, a REST client capable application, or by another Service in ThingWorx. When you create a new service, you can define input properties and an output. You can define individual runtime permissions for each Service. Events Events are triggers that define changes of state (example: device is on, temperature is above/below threshold) of an asset or system and often require an action to correct or respond to a change. Business logic and actions in a ThingWorx application are driven by Events. Subscriptions Action associated with an Event, primary method to set up intelligence in ThingWorx model which enable you to optimize/automate. Subscriptions use Javascript code to define what you want your application to do when the Event occurs.   NOTE: Anything inherited by a Thing Template or Thing will inherit the associated Components.     This diagram shows what a specific Inheritance Model might look like for a connected Tractor. There is one master Template at the top. In this case, it’s a collection of similar types of tractors. The parent Template inherits a few Shapes - an Engine and a Deck that have been used in previous designs. Importing them as Shapes allows us to reuse previous design work and expedite the development process. One of the child Templates incorporates another Shape, this time in the form of a GPS tracking device. Then, at the bottom, there are the specific tractors with individual serial numbers that will report their connected data back to an IoT Application.   Step 5: Next Steps   Congratulations! You've successfully completed the Data Model Introduction, and learned about: The function of a data model for your IoT application Data model components, including Thing, Thing, Shape and Thing Template How ThingWorx components correspond to connected devices Please comment on this post so we can improve this guide in future ThingWorx version iterations.   This guide is part of 2 learning paths: The next guide in the Getting Started on the ThingWorx Platform learning path is Configure Permissions.  The next guide in the Design and Implement Data Models to Enable Predictive Analytics learning path is Design Your Data Model.      
View full tip
Original Post Date:            June 6, 2016   Description: This is a video tutorial on creating a Stream, adding a Data Shape with properties, and writing values to the Stream.    
View full tip
This Best Practices document should offer some guidelines and tips & tricks on how to work with Timers and Schedulers in ThingWorx. After exploring the configuration and creation of Timers and Schedulers via the UI or JavaScript Services, this document will also highlight some of the most common performance issues and troubleshooting techniques.   Timers and Schedulers can be used to run jobs or fire events on a regular basis. Both are implemented as Thing Templates in ThingWorx. New Timer and Scheduler Things can be created based on these Templates to introduce time based actions. Timers can be used to fire events in a certain interval, defined in the Timer's Update Rate (default is 60000 milliseconds = 1 minute). Schedulers can be used to run jobs based on a cron pattern (such as once a day or once an hour). Schedulers will also allow for a more detailed time based setup, e.g. based on seconds, hours, days of week or days months etc. Events fired by both Timers and Schedulers can be subscribed to with Subscriptions which can be utilized to execute custom service scripts, e.g. to generate "fake" or random demo data to update Remote Things in a test environment. In general subscriptions and scripts can be used to e.g. run regular maintenance tasks or periodically required functions (e.g. for data aggregation) For more information about setting up Timers and Schedulers it's recommended to also have a look at the following content:   How to set up and configure Timers How to set up and configure Schedulers How to create and configure Timers and Schedulers via JavaScript Services Events and Subscriptions for Timers and Schedulers   Example   The following example will illustrate on how to create a Timer Thing updating a Remote Thing using random values. To avoid any conflicts with permissions and visibility, use the Administrator user to create Things.   Remote Thing   Create a new Thing based on the Remote Thing Template, called myRemoteThing. Add two properties, numberA and numberB - both Integers and marked as persistent. Save myRemoteThing. Timer Thing   Create a new Thing based on the Timer Template, called myTimerThing. In the Configuration, change the Update Rate to 5000, to fire the Event every 5 seconds. User Context to Administrator. This will run the related services with the Administrator's user visibility and permissions. Save myTimerThing. Subscriptions   To update the myRemoteThing properties when the Timer Event fires, there are two options: Configure a Subscription on myRemoteThing and listen to Timer Events on the myTimerThing. Configure a Subscription on myTimerThing and listen to Timer Events on itself as a source. In this example, let's go with the first option and Edit myRemoteThing. Create a new Subscription pointing to myTimerThing as a Source. Select the Timer Event Note that if no source is selected, the Timer Event is not availabe, as myRemoteThing is based on the Remote Thing Template and not the Timer Template Enable the Subscription. In the Script area use the following code to assign two random numbers to the Thing's custom properties: me.numberA = Math.floor(Math.random() * 100); me.numberB = Math.floor(Math.random() * 100); Save myRemoteThing. Validation   The Subscription will be enabled and active on saving it. Switch to the myRemoteThing Properties Refreshing the Values will show updates with random numbers between 0 and 99 every 5 seconds (Timer Update Rate).   Performance considerations   Timers and Schedulers are handled via the Event Processing Subsystem. Metrics that impact current performance can be seen in Monitoring > Subsystems > Event Processing Implementing Timers and Schedulers on a Thing Template level might flood the system with services executions originating from Subscriptions to Timer / Scheduler triggered Events. Subscribing to another Thing's Events will be handled via the Event Processing Subsystem. Subscribing to an Event on the same Thing will not be handled via the Event Processing Subsystem, but rather execute on the already open in memory Thing. If Timers and Schedulers are not necessarily needed, the Services can be triggered e.g. via Data Change Events, UI Interactions etc. Recursion can be a hidden performance contributer where a Subscription to a certain Event executes a service, triggering another Event with recursive dependencies. Ensure there are no circular dependencies and service calls across Entities. If possible, reads for each and every action from disk should be avoided. Performance can be increased by storing relevant information in memory and using Streams or Datatables or for persistence. If possible, call other Services from within the Subscription instead of handling all code within the Subscription itself. For full details, see also Timers and Schedulers - Best Practice   How to identify and troubleshoot technical issues   Check the Event Processing Subsystem for any spikes in queued Events (tasks submitted) while the total number of tasks completed is not or only slowly increasing. For a historical overview, search the ApplicationLog for "Thingworx System Metrics" to get system metrics since the server has been (re-) started. In the ApplicationLog the message "Subsystem EventProcessingSubsystem is started" indicates that the Subsystem is indeed started and available. Use custom loggers in Services to get more context around errors and execution in the ScriptLog Custom Loggers can be used to identify if Events have fired and Subscriptions are actually triggered Example: logger.debug("myThing: executing subscribed service") For issues with Service execution, see also CS268218 Infinite loops in Services could render the server unresponsive and might flood the system with various Events To change the timing for a Timer, restarting the Thing is not enough. The Timer must be disabled and enabled at the desired start time. Schedulers will allow for a much more flexible timing and setting / changing execution times in advance. For further analysis it's recommended to generate Thread Dumps to get more information about the current state of Threads in the JVM. The ThingWorx Support Tools Extension can help in generating those. See also CS245547 for more information and usage.
View full tip
Events   Timers and Schedulers both come with a specific Event inherited from the Thing Template: Timer ScheduledEvent Both have a Data Shape allowing to capture the timestamp of when the Event was actually fired. Events in ThingWorx are triggered when a specific condition is met. In this context the condition is met and the Event is fired when a Timer has expired or a Scheduler's time is reached. Once an Event is triggered, Subscriptions will take care of executing custom Services to react to the Event. Subscriptions   Subscriptions listen to Events and can be used to react to certain Events with running custom Service scripts. To follow-up on Timers and Schedulers, a new Subscription must be created, listening to any related Event fired. Add a new Subscription to the Thing with       As the Subscription is usually listening to the Thing that it is configured on, the Source has to be left empty. When listening to other Entities' Subscriptions the corresponding Entity can be picked in the Source Entity picker. Ensure to check the Enabled checkbox to actually enable the Subscription and allow it for executing code in the Script area. The following Script will log into the ScriptLog once the Timer Event is fired     The following Script will log into the ScriptLog once the ScheduledEvent Event is fired  
View full tip
Announcements