IoT Tips

Timers and Schedulers can also be created and configured programmatically via custom services. The following service, which can be created on any Thing, will create a new Timer using the following Inputs:         // create new Thing var params = { name: ThingName /* STRING */, description: undefined /* STRING */, thingTemplateName: "Timer" /* THINGTEMPLATENAME */, tags: undefined /* TAGS */ }; Resources["EntityServices"].CreateThing(params); // read initial configuration // result: INFOTABLE var configtable = Things[ThingName].GetConfigurationTable({tableName: "Settings"}); // update configuration with service parameters configtable.updateRate = updateRate configtable.runAsUser = user // set new configuration table var params = { configurationTable: configtable /* INFOTABLE */, persistent: true /* BOOLEAN */, tableName: "Settings" /* STRING */ }; Things[ThingName].SetConfigurationTable(params);   This code is an example which could also be used to create a new Scheduler. The configuration table for a Timer has the following attributes: updateRate enabled runAsUser The configuration table for a Scheduler has the following attributes: schedule enabled runAsUser  

A Feature - a piece of information that is potentially useful for prediction. Any attribute could be a feature, as long as it is useful to the model. Feature engineering – Feature engineering is the process of transforming raw data into features that better represent the underlying problem to the predictive models, resulting in improved model accuracy on unseen data. It’s a vaguely agreed space of tasks related to designing feature sets for Machine Learning applications. Components: First, understanding the properties of the task you’re trying to solve and how they might interact with the strengths and limitations of the model you are going to use. Second, experimental work were you test your expectations and find out what actually works and what doesn’t. Feature engineering as a technique, has three sub categories of techniques: Feature selection, Dimension reduction and Feature generation. Feature Selection: Sometimes called feature ranking or feature importance, this is the process of ranking the attributes by their value to predictive ability of a model. Algorithms such as decision trees automatically rank the attributes in the data set. The top few nodes in a decision tree are considered the most important features from a predictive stand point. As a part of a process, feature selection using entropy based methods like decision trees can be employed to filter out less valuable attributes before feeding the reduced dataset to another modeling algorithm. Regression type models usually employ methods such as forward selection or backward elimination to select the final set of attributes for a model. For example: Project Development decision-tree:                                                  Dimension Reduction: This is sometimes called feature extraction. The most classic example of dimension reduction is principle component analysis or PCA. PCA allows us to combine existing attributes into a new data frame consisting of a much reduced number of attributes by utilizing the variance in the data. The attributes which "explain" the highest amount of variance in the data form the first few principal components and we can ignore the rest of the attributes if data dimensionality is a problem from a computational standpoint. Feature Generation or Feature Construction: Quite simply, this is the process of manually constructing new attributes from raw data. It involves intelligently combining or splitting existing raw attributes into new one which have a higher predictive power. For example a date stamp may be used to generate 2 new attributes such as AM and PM which may be useful in discriminating whether day or night has a higher propensity to influence the response variable. Feature construction is essentially a data transformation process. Tips for Better Feature Engineering Tip 1: Think about inputs you can create by rolling up existing data fields to a higher/broader level or category. As an example, a person’s title can be categorized into strategic or tactical. Those with titles of “VP” and above can be coded as strategic. Those with titles “Director” and below become tactical. Strategic contacts are those that make high-level budgeting and strategic decisions for a company. Tactical are those in the trenches doing day-to-day work.  Other roll-up examples include: Collating several industries into a higher-level industry: Collate oil and gas companies with utility companies, for instance, and call it the energy industry, or fold high tech and telecommunications industries into a single area called “technology.” Defining “large” companies as those that make $1 billion or more and “small” companies as those that make less than $1 billion.   Tip 2: Think about ways to drill down into more detail in a single field. As an example, a contact within a company may respond to marketing campaigns, and you may have information about his or her number of responses. Drilling down, we can ask how many of these responses occurred in the past two weeks, one to three months, or more than six months in the past. This creates three additional binary (yes=1/no=0) data fields for a model. Other drill-down examples include: Cadence: Number of days between consecutive marketing responses by a contact: 1–7, 8–14, 15–21, 21+ Multiple responses on same day flag (multiple responses = 1, otherwise =0) Tip 3: Split data into separate categories also called bins. For example, annual revenue for companies in your database may range from $50 million (M) to over $1 billion (B). Split the revenue into sequential bins: $50–$200M, $201–$500M, $501M–$1B, and $1B+. Whenever a company falls with the revenue bin it receives a one; otherwise the value is zero. There are now four new data fields created from the annual revenue field. Other examples are: Number of marketing responses by contact: 1–5, 6–10, 10+ Number of employees in company: 1–100, 101–500, 502–1,000, 1,001–5,000, 5,000+ Tip 4: Think about ways to combine existing data fields into new ones. As an example, you may want to create a flag (0/1) that identifies whether someone is a VP or higher and has more than 10 years of experience. Other examples of combining fields include: Title of director or below and in a company with less than 500 employees Public company and located in the Midwestern United States You can even multiply, divide, add, or subtract one data field by another to create a new input. Tip 5: Don’t reinvent the wheel – use variables that others have already fashioned. Tip 6: Think about the problem at hand and be creative. Don’t worry about creating too many variables at first, just let the brainstorming flow.

1. Add an Json parameter Example: { ​    "rows":[         {             "email":"example1@ptc.com"         },         {             "name":"Qaqa",             "email":"example2@ptc.com"         }     ] } 2. Create an Infotable with a DataShape usingCreateInfoTableFromDataShape(params) 3. Using a for loop, iterate through each Json object and add it to the Infotable usingInfoTableName.AddRow(YourRowObjectHere) Example: var params = {     infoTableName: "InfoTable",     dataShapeName : "jsontest" }; var infotabletest = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params); for(var i=0; i<json.rows.length; i++) {     infotabletest.AddRow({name:json.rows.name,email:json.rows.email}); }

There are multiples approaches to improve the performance Increase the NetWork bandwidth between client PC and ThingWorx Server Reduce the unnecessary handover when client submit requests to ThingWorx server through NetWork Here are suggestions to reduce the unnecessary handover between client and server Eliminate the use of proxy servers between client and ThingWorx It is compulsory to download Combined.version.date.xxx.xxx.js file when the first time to load mashup page (TTFB and Content Download time). Loading performance is affected by using of proxy servers between client and ThingWorx Server. This is testing result with proxy server set up This is the test result after eliminiating proxy server from the same environment Cut off extensions that not used in ThingWorx server After installed extensions, the size of Combined.version.date.xxx.xxx.js increased Avoid Http/Https request errors There is a https request error when calling Google map. It takes more than 20 seconds

Timers and schedulers can be useful tool in a Thingworx application.  Their only purpose, of course, is to create events that can be used by the platform to perform and number of tasks.  These can range from, requesting data from an edge device, to doing calculations for alerts, to running archive functions for data.  Sounds like a simple enough process.  Then why do most platform performance issues seem to come from these two simple templates? It all has to do with how the event is subscribed to and how the platform needs to process events and subscriptions.  The tasks of handling MOST events and their related subscription logic is in the EventProcessingSubsystem.  You can see the metrics of this via the Monitoring -> Subsystems menu in Composer.  This will show you how many events have been processed and how many events are waiting in queue to be processed, along with some other settings.  You can often identify issues with Timers and Schedulers here, you will see the number of queued events climb and the number of processed events stagnate. But why!?  Shouldn't this multi-threaded processing take care of all of that.  Most times it can easily do this but when you suddenly flood it with transaction all trying to access the same resources and the same time it can grind to a halt. This typically occurs when you create a timer/scheduler and subscribe to it's event at a template level.  To illustrate this lets look at an example of what might occur.  In this scenario let's imagine we have 1,000 edge devices that we must pull data from.  We only need to get this information every 5 minutes.  When we retrieve it we must lookup some data mapping from a DataTable and store the data in a Stream.  At the 5 minute interval the timer fires it's event.  Suddenly all at once the EventProcessingSubsystem get 1000 events.  This by itself is not a problem, but it will concurrently try to process as many as it can to be efficient.  So we now have multiple transactions all trying to query a single DataTable all at once.  In order to read this table the database (no matter which back end persistence provider) will lock parts or all of the table (depending on the query).  As you can probably guess things begin to slow down because each transaction has the lock while many others are trying to acquire one.  This happens over and over until all 1,000 transactions are complete.  In the mean time we are also doing other commands in the subscription and writing Stream entries to the same database inside the same transactions.  Additionally remember all of these transactions and data they access must be held in memory while they are running.  You also will see a memory spike and depending on resource can run into a problem here as well. Regular events can easily be part of any use case, so how would that work!  The trick to know here comes in two parts.  First, any event a Thing raises can be subscribed to on that same Thing.  When you do this the subscription transaction does not go into the EventProcessingSubsystem.  It will execute on the threads already open in memory for that Thing.  So subscribing to a timer event on the Timer Thing that raised the event will not flood the subsystem. In the previous example, how would you go about polling all of these Things.  Simple, you take the exact logic you would have executed on the template subscription and move it to the timer subscription.  To keep the context of the Thing, use the GetImplimentingThings service for the template to retrieve the list of all 1,000 Things created based on it.  Then loop through these things and execute the logic.  This also means that all of the DataTable queries and logic will be executed sequentially so the database locking issue goes away as well.  Memory issues decrease also because the allocated memory for the quries is either reused or can be clean during garbage collection since the use of the variable that held the result is reallocated on each loop. Overall it is best not to use Timers and Schedulers whenever possible.  Use data triggered events, UI interactions or Rest API calls to initiate transactions whenever possible.  It lowers the overall risk of flooding the system with recourse demands, from processor, to memory, to threads, to database.  Sometimes, though, they are needed.  Follow the basic guides in logic here and things should run smoothly!

Distributed Timer and Scheduler Execution in a ThingWorx High Availability (HA) Cluster Written by Desheng Xu and edited by Mike Jasperson    Overview Starting with the 9.0 release, ThingWorx supports an “active-active” high availability (or HA) configuration, with multiple nodes providing redundancy in the event of hardware failures as well as horizontal scalability for workloads that can be distributed across the cluster.   In this architecture, one of the ThingWorx nodes is elected as the “singleton” (or lead) node of the cluster.  This node is responsible for managing the execution of all events triggered by timers or schedulers – they are not distributed across the cluster.   This design has proved challenging for some implementations as it presents a potential for a ThingWorx application to generate imbalanced workload if complex timers and schedulers are needed.   However, your ThingWorx applications can overcome this limitation, and still use timers and schedulers to trigger workloads that will distribute across the cluster.  This article will demonstrate both how to reproduce this imbalanced workload scenario, and the approach you can take to overcome it.   Demonstration Setup   For purposes of this demonstration, a two-node ThingWorx cluster was used, similar to the deployment diagram below:   Demonstrating Event Workload on the Singleton Node   Imagine this simple scenario: You have a list of vendors, and you need to process some logic for one of them at random every few seconds.   First, we will create a timer in ThingWorx to trigger an event – in this example, every 5 seconds.     Next, we will create a helper utility that has a task that will randomly select one of the vendors and process some logic for it – in this case, we will simply log the selected vendor in the ThingWorx ScriptLog.     Finally, we will subscribe to the timer event, and call the helper utility:     Now with that code in place, let's check where these services are being executed in the ScriptLog.     Look at the PlatformID column in the log… notice that that the Timer and the helper utility are always running on the same node – in this case Platform2, which is the current singleton node in the cluster.   As the complexity of your helper utility increases, you can imagine how workload will become unbalanced, with the singleton node handling the bulk of this timer-driven workload in addition to the other workloads being spread across the cluster.   This workload can be distributed across multiple cluster nodes, but a little more effort is needed to make it happen.   Timers that Distribute Tasks Across Multiple ThingWorx HA Cluster Nodes   This time let’s update our subscription code – using the PostJSON service from the ContentLoader entity to send the service requests to the cluster entry point instead of running them locally.       const headers = { "Content-Type": "application/json", "Accept": "application/json", "appKey": "INSERT-YOUR-APPKEY-HERE" }; const url = "https://testcluster.edc.ptc.io/Thingworx/Things/DistributeTaskDemo_HelperThing/services/TimerBackend_Service"; let result = Resources["ContentLoaderFunctions"].PostJSON({ proxyScheme: undefined /* STRING */, headers: headers /* JSON */, ignoreSSLErrors: undefined /* BOOLEAN */, useNTLM: undefined /* BOOLEAN */, workstation: undefined /* STRING */, useProxy: undefined /* BOOLEAN */, withCookies: undefined /* BOOLEAN */, proxyHost: undefined /* STRING */, url: url /* STRING */, content: {} /* JSON */, timeout: undefined /* NUMBER */, proxyPort: undefined /* INTEGER */, password: undefined /* STRING */, domain: undefined /* STRING */, username: undefined /* STRING */ });   Note that the URL used in this example - https://testcluster.edc.ptc.io/Thingworx - is the entry point of the ThingWorx cluster.  Replace this value to match with your cluster’s entry point if you want to duplicate this in your own cluster.   Now, let's check the result again.   Notice that the helper utility TimerBackend_Service is now running on both cluster nodes, Platform1 and Platform2.   Is this Magic?  No!  What is Happening Here?   The timer or scheduler itself is still being executed on the singleton node, but now instead of the triggering the helper utility locally, the PostJSON service call from the subscription is being routed back to the cluster entry point – the load balancer.  As a result, the request is routed (usually round-robin) to any available cluster nodes that are behind the load balancer and reporting as healthy.   Usually, the load balancer will be configured to have a cookie-based affinity - the load balancer will route the request to the node that has the same cookie value as the request.  Since this PostJSON service call is a RESTful call, any cookie value associated with the response will not be attached to the next request.  As a result, the cookie-based affinity will not impact the round-robin routing in this case.   Considerations to Use this Approach   Authentication: As illustrated in the demo, make sure to use an Application Key with an appropriate user assigned in the header. You could alternatively use username/password or a token to authenticate the request, but this could be less ideal from a security perspective.   App Deployment: The hostname in the URL must match the hostname of the cluster entry point.  As the URL of your implementation is now part of your code, if deploy this code from one ThingWorx instance to another, you would need to modify the hostname/port/protocol in the URL.   Consider creating a variable in the helper utility which holds the hostname/port/protocol value, making it easier to modify during deployment.   Firewall Rules: If your load balancer has firewall rules which limit the traffic to specific known IP addresses, you will need to determine which IP addresses will be used when a service is invoked from each of the ThingWorx cluster nodes, and then configure the load balancer to allow the traffic from each of these public IP address.   Alternatively, you could configure an internal IP address endpoint for the load balancer and use the local /etc/hosts name resolution of each ThingWorx node to point to the internal load balancer IP, or register this internal IP in an internal DNS as the cluster entry point.

Those who have been working with ThingWorx for many years will have noticed the work done around ingress stress testing and performance optimization.  Adding InfluxDB as a time-series data persistence provider really helped level up these capabilities while simultaneously decreasing the overall resources required by the infrastructure.  However with this ease comes a hidden challenge: query and data processing performance to work it into something useful.   Often It's Too Much Data In general most customers that I work with want to collect far too much data -- without knowing what it will be used for, or what processing will be required in order to make it usable and useful.  This is a trap in general with how many people envision IoT projects, being told by infrastructure providers that cloud storage and compute resources are abundant and cheap and that they should get as much data as possible.  This buildup of data means that more effort needs to be spent working it into something useful (data engineering/feature extraction) and addressing common data issues (quality, gaps, precision, etc.).  This might be fine for mature companies with large data analytics teams; however this is a makeup that I've only seen in the largest of our customers.  Some advice - figure out what you need and how you'll use it, and then collect that.  Work on extracting value today rather than hoping that extra data collected  now will provide some insights years from now.   Example - Problem Statement You got your Thing Model designed, and edge devices connected.  Now you've got data flowing in and being stored every 5 seconds in InfluxDB.  Great progress!  Now on to building the applications which cover the various use cases. The raw data is most likely going to need to be processed and potentially even significantly transformed into other information in order to make it useful.  Turning a "powered on and running" BOOLEAN to an "hour meter" INTEGER is a simple example.  Then you may need to provide a report showing equipment run time hours by day over a month.  The maintenance team may also have asked to look for usage patterns which lead to breakdowns, requiring extracting other data points from the initial one like number of daily starts, average daily run time, average time between restarts. The problem here is that unless you have prepared these new data points and stored them as well (say in a Stream), you are going to have to build these data sets on the fly, and that can be time and resource intensive and not give you the response time expected.  As you can imagine, repeatedly querying and processing large volumes of unchanging raw data is going to have resource and time implications - so this is why data collection and data use need to be thought about separately.   Data Engineering In the above examples, the key is actually creating new data points which are calculated progressively throughout normal operation.  This not only makes the information that you want available when you need it - in the right format - but it also significantly reduces resource requirements by constantly reprocessing raw data.  It also helps managing data purging, because as you create and store usable insights, you can eventually just archive away your old raw data streams.   Direct Database Queries vs. Thingworx Data Services Despite the above being a rule of thumb, sometimes a simple well structured database query can get you exactly what you need and do so quite quickly.  This is especially true for InfluxDB when working with extremely large time-series datasets.  The challenge here is that ThingWorx persistence providers abstract away the complexity of writing ones own database queries, so we can't easily get at the databases raw power and are forced to query back more data than needed and work it into a usable format in memory (which is not fast).   Leveraging the InfluxDB API using the ContentLoader Technique As InfluxDBs API is 100% REST, we can access it using in-built ThingWorx Content Loader services.  Check out this demonstration and explanation video where I talk about how to interact directly with InfluxDB in order to crush massive time-series data and get back much more usable and manageable data sets.  It is important to note here that you should use a read-only database user here, as you should never modify the ThingWorx databases to avoid untested scenarios which may lead to data corruption.   Optimizing ThingWorx query performance with the InfluxDB REST API - YouTube InfluxToolBox ThingWorx demo project (by T. Wobben)      

I've had a lot of questions over the years working with Azure IoT, Kepware, and ThingWorx that I really struggled getting answers to. I was always grateful when someone took the time to help me understand, and now it is time to repay the favour.   People ask me many things about Azure (in a ThingWorx context), and one of the common ones has been about MQTT communications from Kepware to ThingWorx using IoT Hub. Recently the topic has come up again as more and more of the ThingWorx expert community start to work with Azure IoT. Today, I took the time to build, test, validate, and share an approach and utilities to do this in cases where the Azure Industrial IoT OPC UA integration is overkill or simply a step later in the project plan. Enjoy!   End to end Integration of Kepware to ThingWorx using MQTT over Azure IoT (YoutTube 45 minute deep-dive)   ThingWorx entities for import (ThingWorx 9.0)   This approach can be quite good for a simple demo if you have a Kepware Integrator or Kepware Enterprise license, but the use of IoT Gateway for many servers and tags can be quite costly.   Those looking to leverage Azure IoT Hub for MQTT integration to ThingWorx would likely also find this recorded session and shared utilities quite helpful.   Cheers, Greg

Everywhere in the Thingworx Platform (even the edge and extensions) you see the data structure called InfoTables.  What are they?  They are used to return data from services, map values in mashup and move information around the platform.  What they are is very simple, how they are setup and used is also simple but there are a lot of ways to manipulate them.  Simply put InfoTables are JSON data, that is all.  However they use a standard structure that the platform can recognize and use. There are two peices to an InfoTable, the DataShape definition and the rows array.  The DataShape is the definition of each row value in the rows array.  This is not accessible directly in service code but there are function and structures to manipulate it in services if needed. Example InfoTable Definitions and Values: { dataShape: {     fieldDefinitions : {           name: "ColOneName", baseType: "STRING"     },     {           name: "ColTwoName", baseType: "NUMBER"     }, rows: [     {ColOneName: "FirstValue", ColTwoName: 13},     {ColOneName: "SecondValue, ColTwoName: 14}     ] } So you can see that the dataShape value is made up of a group of JSON objects that are under the fieldDefinitions element.  Each field is a "name" element, which of course defined the field name, and the "baseType" element which is the Thingworx primitive type of the named field.  Typically this structure is automatically created by using a DataShape object that is defined in the platform.  This is also the reason DataShapes need to be defined, so that fields can be defined not only for InfoTables, but also for DataTables and Streams.  This is how Mashups know what the structure of the data is when creating bindings to widgets and other parts of the platform can display data in a structured format. The other part is the "rows" element which contains an array of of JSON objects which contain the actual data in the InfoTable. Accessing the values in the rows is as simple as using standard JavaScript syntax for JSON.  To access the number in the first row of the InfoTable referenced above (if the name of the InfoTable variable is "MyInfoTable") is done using MyInfoTable.rows[0].ColTowName.  This would return a value of 13.  As you can not the JSON array index starts at zero. Looping through an InfoTable in service script is also very simple.  You can use the index in a standard "for loop" structure, but a little cleaner way is to use a "for each loop" like this... for each (row in MyInfoTable.rows) {     var colOneVal = row.ColOneName;     ... } It is important to note that outputs of many base services in the platform have an output of the InfoTable type and that most of these have system defined datashapes built into the platform (such as QueryDataTableEntries, GetImplimentingThings, QueryNumberPropertyHistory and many, many more).  Also all service results from query services accessing external databases are returned in the structure of an InfoTable. Manipulating an InfoTable in script is easy using various functions built into the platform.  Many of these can be found in the "Snippets" tab of the service editor in Composer in both the InfoTableFunctions Resource and InfoTable Code Snippets. Some of my favorites and most commonly used... Create a blank InfoTable: var params = {   infoTableName: "MyTable" }; var MyInfoTable= Resources["InfoTableFunctions"].CreateInfoTable(params); Add a new field to any InfoTable: MyInfoTable.AddField({name: "ColNameThree", baseType: "BOOLEAN"}); Delete a field: MyInfoTable.RemoveField("ColNameThree"); Add a data row: MyInfoTable.AddRow({ColOneName: "NewRowValue", ColTwoName: 15}); Delete one or more data row matching the values defined (Note you can define multiple field in this statement): //delete all rows that have a value of 13 in ColNameOne MyInfoTable.Delete({ColNameOne: 13}); Create an InfoTable using a predefined DataShape: var params = {   infoTableName: "MyInfoTable",   dataShapeName: "dataShapeName" }; var MyInfoTable = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params); There are many more functions built into the platform, including ones to filter, sort and query rows.  These can be extremely useful when tying to return limited or more strictly structured InfoTable data.  Hopefully this gives you a better understanding and use of this critical part of the Thingworx Platform.

Hello everyone, This post is meant to fill the gap that Basic Rules of ThingWorx Development is having. You can follow these rules even before starting the development process and keep them in mind to have an organized and easy to maintain application. I will update this post in the future with more best practices and advice. Best Practices and suggestions: In order to have a clean and quick progress in any project the approach should be modular. If the modular approach is implemented also the development process should be thought of in a modular way. This will give much needed independence to each individual developer especially if the team concurrently works on the same instance. Some rules need to be in place in order for the project to be as smooth as possible: Every developer must have its own user. This is more important when developing on the same Thingworx instance but it’s a good practice when developing on individual instances as well. Every developer will be responsible for complete modules, from the respective screens of the GUI to the functionality services and business logic. If concurrent work on the same Entity needs to happen then communication between the developers and time sharing on that entity is needed without developers overwriting each other’s code. Don't decide to go into edit mode if there is someone else already editing. That will get you to a dead end. For the point no. 3 to work, after editing an Entity each user must press the Cancel Edit button and leave that Entity in View mode. When searching for services or properties developers should avoid pressing on the name of the Entity which is a link that directly opens the Entity in Edit mode they should rather use the button with the magnifying glass to the left of the name that will then take them in View mode. As a result of the modular approach each module will have its own Utility Thing that will contain services, properties, events and subscriptions that help develop the functionality for that module. Each module will have its own tags and the format could be: <Client_Name><GUI/Business><Module_Name>   8. The integration of the modules will be done in the Master by a single person in charge with that master or by each developer at a time.   9. Depending on the case the Data Model could be treated as a module in its own right or can be integrated in each module if the project permits. How to manage multiple users working on the same code in Composer: (Thanks to Pai Chung) Currently Thingworx within the development environment allows you to heavily document all your works, that includes ‘Save with Comment’. We encourage the use of the Documentation field and the ‘Save with Comment’ option. However generally development is not isolated to one environment. Thingworx provides several ways to back up the information. Backup – this is a true Database backup that creates an additional database in ThingworxBackupStorage and basically can be used as a restore, by copying it back into ThingworxStorage Export to ThingworxStorage – this is a full model export (with or without data) that can be triggered at any time. It can use Date filters to export according to Modified date. This is server side. Export to File – this allows you to export a single or group of entities/data according to a variety of filters. This is client side. Export to Source Controlled Entities – this allows you to export to a file folder structure or Zip that can be easily checked into a Source Control system. How to approach Source Control: After some initial modeling, Export to Source Control Entities and check this into your Source Control system From this point forward all developers have to follow a Check in/check out process Every time an Entity Group security setting is made, Export to ThingworxStorage and also check that into Source Control overwrite the previous. All in use Extensions should be in one zip and also reside in Source Control To do a restore or deploy Install the Platform Install extensions Import from ThingworxStorage the last Export checked in Import each single Entity file, in the proper order. Import each single Data file   6.  Clean up dead entities (if there is a reference list) Additional steps to take to help safeguard the development. Make sure the Automatic backup is running Export the Entity to a subfolder with the Date of the Edit     3.  Full Export to ThingworxStorage to run every day after development stars - This can be scripted and triggered by a timer or scheduler subscription (<Server>/Thingworx/ExportDatabase/?WithData=true). In this way you have a backup with everything that was on before you started working each day so you can roll back if an error occurs. CONTINUED 7 Sep 2015 How to organize wiring needs when developing the GUI: Starting from the idea that we can divide the GUI elements in Display Elements and Action Elements I have created a common form in order to be filled with information necessary for the wiring of that Element. UI Element Type Display Element / User Action Element Thing Name Name of the thing where data / service is found Service Name Service inside the Thing that returns the data / is the subject of the action Property(ies) Name Thing property / column name (when service returns an infotable) for Data Elements / Input parameters for the service to be run if User Action Element Additional Logic Additional information regarding the way the information sources change when preconditions are met. Usually means new services or mashup logic is needed.  I suggest that an additional companion document to the GUI description document to be created. This document will contain the previous form (table) for each screen/slide so that the work on specific screen/slide could be done independently. To be continued...

NOTE: Even though I have tried on ODataConnector and SwaggerConnector, these steps below should be working for all the Thingworx Integration Connectors viz. GenericConnector, HTTPConnector, ODataConnector, SAPODataConnector, SwaggerConnector, WindchillSwaggerConnector.   This document guides you to add a custom header in any Thingworx Integration connector. Step 1. Create a Datashape say "CustomHeadersDataShape" and add a string field with Name the same as the header name you want to add. In this case, I want to add a header called "Prefer" so it will look something like              Step 2: Go to the Integration Connector which you want to add this custom header. Navigate to "Services". Under the "Inherited Services", edit/overwrite the "GetCustomHeaderParameters" service by clicking on the edit (pencil) icon. Step 3: In the JavaScript Code sniped section add below code snipped   var params = { infoTableName : "InfoTable", dataShapeName : "CustomHeadersDataShape" }; var result = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params); var preferValue = "odata.maxpagesize=50"; var newRow = {"Prefer" : preferValue }; result.AddRow(newRow);   Step 4: Save the service and execute "GetCustomHeaderParameters". You should see something like         Now your custom header "Prefer: odata.maxpagesize=50" is set. further execution of your connector services will consider this header until it is reset.

Background: The frequency with which an Agent checks its connection to the Axeda Cloud Server is called the Agent “ping rate” (also known as heartbeat). (For Axeda IDM Agents, ping rate is referred to as “poll rate”; the meaning is the same.) Pings are a very important aspect of Firewall-Friendly communication. All communication between the Agent and the Cloud Server is initiated by the Agent. In addition to indicating the Agent is still active, the Ping also gives the Cloud Server an opportunity to send commands back to the Agent on the Ping acknowledgement. The ping rate effectively defines how long users must wait before they can deliver a command or request to an Agent. Typical commands may include setting a data item, starting an Access Session, or running a script. The place where Ping rate is most noticeable to system users is when requesting a remote session. When a session request has been submitted by the user, the Cloud Server waits for the next Agent ping in order to send down the command to begin the session process. A longer ping rate means the remote session takes longer to get started. (Note that the same is true of any command initiated from the Axeda Cloud Server.) Ping traffic comprises the majority of inbound traffic to the Cloud Server. The higher the ping rate, the more resources are consumed on the Server and the greater the requirements for network bandwidth for the customer. Unnecessarily high ping rates will result in an increase in network traffic on your customer's network. By default, the ping rate for Firewall-Friendly Agents is 60 seconds, or every 1 minute. The Agent ping rate is set using Axeda Builder when configuring the project. The ping rate can also be set via an action from the Axeda Cloud Server. When set via an action, the new ping rate is in effect until the next Agent restart (at which time the Agent will go back to the default ping rate set in the project). The Axeda Cloud Server also uses Agent ping rate to determine when assets are missing. One of the model settings is to define how many missed pings (or missed pings and time) will cause a device to be marked as missing. The default setting for new models is to mark assets as missing after they’ve missed 3 consecutive pings. Recommendations: Make sure that your Agents’ ping rates are set to reasonable frequencies. The ping rate should be set based on use case and not necessarily volume. The recommended practice is to make sure the ping rate is never set less than 60 seconds. Where possible the ping rate should be set to 2 minutes or higher. In the end, it is often user expectations around starting Access sessions that drives the ping rate value. If only occasional user access is required, one recommendation is to dynamically adjust the ping rate when conditions require expedited communication with the Cloud Server. One use case is to expedite a remote session when a device is in alarm condition or when an end user needs assistance. In this case you would temporarily increase the ping rate. This can be done using an action from the Cloud Server, by downloading a software package ping rate update, or by Agent extension using the SDK. (For information about using the Agent SDK, see the Axeda® Platform Extending Axeda® Agents PDF.) You can configure alerts to indicate if an asset is missing. Axeda recommends that you configure the alert to a reasonable time given your resources and the expense of tracking every missing asset. A reasonable missing alert for your organization may be 1-2 days, meaning the Server generates the missing asset alert only after the asset has been missing for one or two days, based on its ping rate, and an asset should be marked as missing only after 15 missed pings or 30 minutes (whichever is less). The most common cause of a missing asset is not an issue with the device but rather the loss of Internet connectivity. Note: Any communication from an Agent also serves the function of a Ping. E.g., if the ping rate is set to 30 minutes and the device is sending a data value every 5 minutes, the effective Ping rate is 5 minutes. Need more information? For information about specifying Agent ping rate, see the online help in Axeda® Builder (Enterprise Server Settings). If setting the ping rate from Platform actions or verifying Agent ping rates, see the online help of the Axeda® Connected Management Applications.

  The IoT Enterprise Deployment Center’s goal is to create and share knowledge around the best practices for architecting, designing, and deploying successful, enterprise-scale Thingworx IoT Solutions.    To accomplish this goal, the EDC team takes a “real world” approach, using simulated IoT assets and users to benchmark the capabilities of different Thingworx deployment configurations. First, each implementation is pushed to its limit in an effort to establish real-world baselines, metrics which can be used to help customers determine which architecture choices will work for their custom needs. Then, each implementation is pushed beyond its limits, providing useful insight into where and why things fail, and illuminating potential implementation changes which could push the boundaries further.   Through the simulations testing to come, the EDC will be publishing the resulting benchmarks for all to see! These benchmarks will include details on implementation goals and performance metrics for different stages of deployment. Additionally, best-practice articles which illustrate how to deploy the different architectural components (those referenced within the benchmarks) will also be posted, highlighting the optimal approach to integrating everything into the Thingworx platform.   Stay tuned to see more about just how versatile the ThingWorx Platform can be! We look forward to discussing these findings as they are published right here on the PTC Community. 

Fresh look at getting started with ThingWorx in a relevant context that outlines the DEVOPS needed to kick-start your programming.     For full-sized viewing, click on the YouTube link in the player controls. Visit the Online Success Guide to access our Expert Session videos at any time as well as additional information about ThingWorx training and services.

Get Started with ThingWorx for IoT Guide Part 5   Step 13: Extend Your Model   Modify the application model, enhance your UI, and add features to the house monitoring application to simulate a request as it might come from an end user. For this step, we do not provide explicit instructions, so you can use critical thinking to apply your skills. After completing the previous steps, your Mashup should look like this:   In this part of the lesson, you'll have an opportunity to: Complete an application enhancement in Mashup Builder Compare your work with that of a ThingWorx engineer Import and examine ThingWorx entities provided for download that satisfy the requirements Understand the implications of ThingWorx modeling options   Task Analysis   Add a garage to the previously-created house monitoring web application and include a way to display information about the garage in the UI. You will need to model the garage using Composer and add to the web application UI using Mashup Builder. What useful information could a web application for a garage provide? How could information about a garage be represented in ThingWorx? What is the clearest way to display information about a garage?   Tips and Hints   See below for some tips and hints that will help you think about what to consider when modifying the application in ThingWorx. Modify your current house monitoring application by adding a garage: Extend your model to include information about a garage using Composer. Add a display of the garage information to your web application UI using Mashup Builder.   Best Practices   Keep application development manageable by using ThingWorx features that help organize entities you create.   Modeling   The most important feature of a garage is the status of the door. In addition to its current status, a user might be interested in knowing when the garage door went up or down. Most garages are not heated, so a user may or may not be interested in the garage temperature.   Display   The current status of the garage door should be easily visible. Complete the task in your Composer before moving forward. The Answer Key below reveals how we accomplished this challenge so you can compare your results to that of a ThingWorx engineer.   Answer Key   Confirm you configured your Mashup to meet the enhancement requirements when extending your web application. Use the information below to check your work.   Create New Thing   Creating a new Thing is one way to model the garage door. We explain other methods, including their pros and cons, in the Solution discussion below. Did you create a new Thing using the Building Template? Did you apply a Tag to the new Thing you created?   Review detailed steps for how to Create a Thing in Part 2 of this guide.   Add Property   Any modeling strategy requires the addition of a new Property to your model. We explore options for selecting an appropriate base type for the garage Property in the Solution discussion on the next step. Did you add a Property to represent the garage door? Did you use the Boolean type? Did you check the Logged? check-box to save history of changes?   Review detailed steps for how to Add a Property. in Part 1, Step 3 of this guide.   Add Widget   In order to display the garage door status, you must add a Widget to your Mashup. We used a check-box in our implementation. We introduce alternative display options in the Solution discussion on the next step. Did you add a Widget to your Mashup representing the garage door status? Review detailed steps for how to Create an Application UI in Part 3, Step 8 of this guide.   Add Data Source   If you created a new Thing, you must add a new data source. This step is not required if you added a Property to the existing Thing representing a house. Did you add a data source from the garage door Property of your new Thing? Did you check the Execute on Load check-box? Review detailed steps for how to Add a Data Source to a Mashup in Part 4, Step 10 of this guide.   Bind Data Source to Widget   You must bind the new garage door Property to a Widget in order to affect the visualization. Did you bind the data source to the Widget you added to your Mashup? Review detailed steps for how to Bind a Data Source to a Widget in Part 4, Step 10 of this guide.   Solution   If you want to inspect the entities as configured by a ThingWorx engineer, import this file into your Composer. Download the attached example solution:   FoundationChallengeSolution.xml Import the xml file into, then open MyHouseAndGarage Thing. See below for some options to consider in your application development.   Modeling   There are several ways the garage door property could be added to your existing model. The table below discusses different implementations we considered. We chose to model the status of the garage door as a Property of a new Thing created using the building Template. Modeling Method Pros Cons Add Property to BuildingTemplate The Garage property will be added to existing house Thing All future Things using Building Template will have a garage door property Add Property to existing house Thing House and garage are linked No separate temperature and watts Property for garage Add Property to new Thing created with BuildingTemplate All Building features available No logical link between house and garage   Property Base Type   We chose to represent the status of the door with a simple Boolean Property named 'garageDoorOpen' Thoughtful property naming ensures that the values, true and false, have a clear meaning. Using a Boolean type also makes it easy to bind the value directly to a Widget. The table below explains a few Base Type options. Modeling Method Pros Cons Boolean Easy to bind to Widget Information between open and closed is lost Number Precise door status Direction information is lost String Any number of states can be represented An unexpected String could break UI   Visualization   We chose a simple Check-box Widget to show the garage door status, but there are many other Widgets you could choose depending on how you want to display the data. For example, a more professional implementation might display a custom image for each state.   Logging   We recommended that you check the Logged option, so you can record the history of the garage door status.   Step 14: Next Steps   Congratulations! You've successfully completed the Get Started with ThingWorx for IoT tutorial, and learned how to: Use Composer to create a Thing based on Thing Shapes and Thing Templates Store Property change history in a Value Stream Define application logic using custom defined Services and Subscriptions Create an applicaton UI with Mashup Builder Display data from connected devices Test a sample application The next guide in the Getting Started on the ThingWorx Platform learning path is Data Model Introduction.

Get Started with ThingWorx for IoT Guide Part 4   Step 10: Display Data   Now that you have configured the visual part of your application, you need to bind the Widgets in your Mashup to a data source, and enable your application to display data from your connected devices.   Add Services to Mashup   Click the Data tab in the top-right section of the Mashup Builder. Click on the green + symbol in the Data tab.   Type MyHouse in the Entity textbox. Click MyHouse. In the Filter textbox below Services, type GetPropertyValues. Click the arrow to the right of the GetPropertyValues service to add it.   Select the checkbox under Execute on Load. NOTE: If you check the Execute on Load option, the service will execute when the Mashup starts. 8. In the Filter textbox under Services, type QueryProperty. 9. Add the QueryPropertyHistory service by clicking the arrow to the right of the service name. 10. Click the checkbox under Execute on Load. 11. Click Done. 12. Click Save.   Bind Data to Widgets   We will now connect the Services we added to the Widgets in the Mashup that will display their data.   Gauge   Configure the Gauge to display the current power value. Expand the GetPropertyValues Service as well as the Returned Data and All Data sections. Drag and drop the watts property onto the Gauge Widget.   When the Select Binding Target dialogue box appears, select # Data.   Map   Configure Google Maps to display the location of the home. Expand the GetPropertyValues service as well as the Returned Data section. Drag and drop All Data onto the map widget.   When the Select Binding Target dialogue box appears, select Data. Click on the Google Map Widget on the canvas to display properties that can configured in the lower left panel. Set the LocationField property in the lower left panel by selecting building_lat_lng from the drop-down menu.   Chart   Configure the Chart to display property values changing over time. Expand the QueryPropertyHistory Service as well as the Returned Data section. Drag and drop All Data onto the Line Chart Widget. When the Select Binding Target dialogue box appears, select Data. In the Property panel in the lower left, select All from the Category drop-down. Enter series in Filter Properties text box then enter 1 in NumberOfSeries . Enter field in Filter Properties text box then click XAxisField. Select the timestamp property value from the XAxisField drop-down. Select temperature from the DataField1 drop-down.   Verify Data Bindings   You can see the configuration of data sources bound to widgets displayed in the Connections pane. Click on GetPropertyValues in the data source panel then check the diagram on the bottom of the screen to confirm a data source is bound to the Gauge and Map widget.   Click on the QueryPropertyHistory data source and confirm that the diagram shows the Chart is bound to it. Click Save.   Step 11: Simulate a Data Source   At this point, you have created a Value Stream to store changing property value data and applied it to the BuildingTemplate. This guide does not include connecting edge devices and another guide covers choosing a connectivity method. We will import a pre-made Thing that creates simulated data to represent types of information from a connected home. The imported Thing uses Javascript code saved in a Subscription that updates the power and temperature properties of the MyHouse Thing every time it is triggered by its timer Event.    Import Data Simulation Entities   Download the attached sample:  Things_House_data_simulator.xml. In Composer, click the Import/Export icon at the lower-left of the page. Click Import. Leave all default values and click Browse to select the Things_House_data_simulator.xml file that you just downloaded. Click Open, then Import, and once you see the success message, click Close.   Explore Imported Entities   Navigate to the House_data_simulator Thing by using the search bar at the top of the screen. Click the Subscriptions tab. Click Event.Timer under Name. Select the Subscription Info tab. NOTE: Every 30 seconds, the timer event will trigger this subscription and the Javascript code in the Script panel will run. The running script updates the temperature and watts properties of the MyHouse Thing using logic based on both the temperature property from MyHouse and the isACrunning property of the simulator itself. 5. Expand the Subscription Info menu. The simulator will send data when the Enabled checkbox is checked. 6. Click Done then Save to save any changes.   Verify Data Simulation   Open the MyHouse Thing and click Properties an Alerts tab Click the Refresh button above where the current property values are shown   Notice that the temperature property value changes every 30 seconds when the triggered service runs. The watts property value is 100 when the temperature exceeds 72 to simulate an air conditioner turning on.   Step 12: Test Application   Browse to your Mashup and click View Mashup to launch the application.   NOTE: You may need to enable pop-ups in your browser to see the Mashup.       2. Confirm that data is being displayed in each of the sections.        Test Alert   Open MyHouse Thing Click the Properties and Subscriptions Tab. Find the temperature Property and click on pencil icon in the Value column. Enter the temperature property of 29 in the upper right panel. Click Check mark icon to save value. This will trigger the freezeWarning alert.   Click Refresh to see the value of the message property automatically set.   7. Click the the Monitoring icon on the left, then click ScriptLog to see your message written to the script log.   Click here to view Part 5 of this guide. 

Get Started with ThingWorx for IoT Guide Part 3   Step 7: Create Alerts and Subscriptions   An Event is a custom-defined message published by a Thing, usually when the value of a Property changes. A Subscription listens for a specific Event, then executes Javascript code. In this step, you will create an Alert which is quick way to define both an Event and the logic for when the Event is published.   Create Alert   Create an Alert that will be sent when the temperature property falls below 32 degrees. Click Thing Shapes under the Modeling tab in Composer, then open the ThermostatShape Thing Shape from the list.   Click Properties and Alerts tab.   Click the temperature property. Click the green Edit button if not already in edit mode, then click the + in the Alerts column.   Choose Below from the Alert Type drop down list. Type freezeWarning in the Name field.   Enter 32 in the Limit field. Keep all other default settings in place. NOTE: This will cause the Alert to be sent when the temperature property is at or below 32.        8. Click ✓ button above the new alert panel.       9. Click Save.     Create Subscription   Create a Subscription to this event that uses Javascript to record an entry in the error log and update a status message. Open the MyHouse Thing, then click Subscriptions tab.   Click Edit if not already in edit mode, then click + Add.   Type freezeWarningSubscription in the Name field. After clicking the Inputs tab, click the the Event drop down list, then choose Alert. In the Property field drop down, choose temperature.   Click the Subscription Info tab, then check the Enabled checkbox   Create Subscription Code   Follow the steps below to create code that sets the message property value and writes a Warning message to the ThingWorx log. Enter the following JavaScript in the Script text box to the right to set the message property.                       me.message = "Warning: Below Freezing";                       2. Click the Snippets tab. NOTE: Snippets provide many built-in code samples and functions you can use. 3. Click inside the Script text box and hit the Enter key to place the cursor on a new line. 4. Type warn into the snippets filter text box or scroll down to locate the warn Snippet. 5. Click All, then click the arrow next to warn, and Javascript code will be added to the script window. 6. Add an error message in between the quotation marks.                       logger.warn("The freezeWarning subscription was triggered");                       7. Click Done. 8. Click Save.   Step 8: Create Application UI ThingWorx you can create customized web applications that display and interact with data from multiple sources. These web applications are called Mashups and are created using the Mashup Builder. The Mashup Builder is where you create your web application by dragging and dropping Widgets such as grids, charts, maps, buttons onto a canvas. All of the user interface elements in your application are Widgets. We will build a web application with three Widgets: a map showing your house's location on an interactive map, a gauge displaying the current value of the watts property, and a graph showing the temperature property value trend over time. Build Mashup Start on the Browse, folder icon tab of ThingWorx Composer. Select Mashups in the left-hand navigation, then click + New to create a new Mashup.   For Mashup Type select Responsive.   Click OK. Enter widgetMashup in the Name text field, If Project is not already set, click the + in the Project text box and select the PTCDefaultProject, Click Save. Select the Design tab to display Mashup Builder.   Organize UI On the upper left side of the design workspace, in the Widget panel, be sure the Layout tab is selected, then click Add Bottom to split your UI into two halves.   Click in the bottom half to be sure it is selected before clicking Add Left Click anywhere inside the lower left container, then scroll down in the Layout panel to select Fixed Size Enter 200 in the Width text box that appears, then press Tab key of your computer to record your entry.   Click Save   Step 9: Add Widgets Click the Widgets tab on the top left of the Widget panel, then scroll down until you see the Gauge Widget Drag the Gauge widget onto the lower left area of the canvas on the right. This Widget will show the simulated watts in use.   Select the Gauge object on the canvas, and the bottom left side of the screen will show the Widget properties. Select Bindable from the Catagory dropdown and enter Watts for the Legend property value, and then press tab..   Click and drag the Google Map Widget onto the top area of the canvas. NOTE: The Google Map Widget has been provisioned on PTC CLoud hosted trial servers. If it is not available, download and install the Google Map Extension using the step-by-step guide for using Google Maps with ThingWorx . Click and drag the Line Chart Widget onto the lower right area of the canvas. Click Save

Design and Implement Data Models to Enable Predictive Analytics Learning Path   Design and implement your data model, create logic, and operationalize an analytics model.   NOTE: Complete the following guides in sequential order. The estimated time to complete this learning path is 390 minutes.    Data Model Introduction  Design Your Data Model Part 1 Part 2 Part 3  Data Model Implementation Part 1 Part 2 Part 3  Create Custom Business Logic  Implement Services, Events, and Subscriptions Part 1 Part 2  Build a Predictive Analytics Model  Part 1 Part 2 Operationalize an Analytics Model  Part 1 Part 2  

Getting Started on the ThingWorx Platform Learning Path   Learn hands-on how ThingWorx simplifies the end-to-end process of implementing IoT solutions.   NOTE: Complete the following guides in sequential order. The estimated time to complete this learning path is 210 minutes.   Get Started with ThingWorx for IoT   Part 1 Part 2 Part 3 Part 4 Part 5 Data Model Introduction Configure Permissions Part 1 Part 2 Build a Predictive Analytics Model  Part 1 Part 2