IoT & Connectivity Tips

Getting Started on the ThingWorx Platform Learning Path   Learn hands-on how ThingWorx simplifies the end-to-end process of implementing IoT solutions.   NOTE: Complete the following guides in sequential order. The estimated time to complete this learning path is 210 minutes.   Get Started with ThingWorx for IoT   Part 1 Part 2 Part 3 Part 4 Part 5 Data Model Introduction Configure Permissions Part 1 Part 2 Build a Predictive Analytics Model  Part 1 Part 2

Design Your Data Model Guide Part 1   Overview   This project will introduce the process of taking your IoT solution from concept to design. Following the steps in this guide, you will create a solution that doesn’t need to be constantly revamped, by creating a comprehensive Data Model before starting to build and test your solution. We will teach you how to utilize a few proposed best practices for designing the ThingWorx Data Model and provide some prescriptive methods to help you generate a high-quality framework that meets your business needs. NOTE: This guide’s content aligns with ThingWorx 9.3. The estimated time to complete ALL 3 parts of this guide is 60 minutes. All content is relevant but there are additional tools and design patterns you should be aware of. Please go to this link for more details.    Step 1: Data Model Methodology   We will start by outlining the overall process for the proposed Data Model Methodology.       Step Description 1 User Stories Identify who will use the application and what information they need. By approaching the design from a User perspective, you should be able to identify what elements are necessary for your system. 2 Data Sources Identify the real-world objects or systems which you are trying to model. To create a solid design, you need to identify what the “things” are in your system and what data or functionality they expose. 3 Model Breakdown Compose a representative model of modular components to enable uniformity and reuse of functionality wherever possible. Break down user requirements and data, identifying how the system will be modeled in Foundation. 4 Data Strategy Identify the sources of data, then evaluate how many different types of data you will have, what they are, and how your data should be stored. From that, you may determine the data types and data storage requirements. 5 Business Logic Strategy Examine the functional needs, and map them to your design for proper business logic implementation. Determine the business logic as a strategic flow of data, and make sure everything in your design fits together in logical chunks. 6 User Access Strategy Identify each user's access and permission levels for your application. Before you start building anything, it is important to understand the strategy behind user access. Who can see or do what? And why? NOTE: Due to the length of this subject, the ThingWorx Data Model Methodology has been divided into multiple parts. This guide focuses on the first three steps = User Stories, Data Sources, and Model Breakdown. Guides covering the last three steps are linked in the final Next Steps page.    Step 2: User Stories     With a user-based approach to design, you identify requirements for users at the outset of the process. This increases the likelihood of user satisfaction with the result. Utilizing this methodology, you consider each type of user that will be accessing your application and determine their requirements according to each of the following two categories: Category Requirement Details Functionality Determine what the user needs to do. This will define what kind of Services and Subscriptions will need to be in the system and which data elements and Properties must be gathered from the connected Things. Information What information do they need? Examine the functional requirements of the user to identify which pieces of information the users need to know in order to accomplish their responsibilities.   Factory Example   Let’s revisit our Smart Factory example scenario. The first step of the User Story phase of the design process is to identify the potential users of your system. In this example scenario, we have defined three different types of users for our solution: Maintenance Operations Management Each of these users will have a different role in the system. Therefore, they will have different functional and informational needs.   Maintenance   It is the maintenance engineer’s job to keep machines up and running so that the operator can assemble and deliver products. To do this well, they need access to granular data for the machine’s operating status to better understand healthy operation and identify causes of failure. They also need to integrate their maintenance request management system to consolidate their efforts and to create triggers for automatic maintenance requests generated by the connected machines. Required Functionality Get granular data values from all assets Get a list of maintenance requests Update maintenance requests Set triggers for automatic maintenance request generation Automatically create maintenance requests when triggers have been activated Required Information Granular details for each asset to better understand healthy asset behavior Current alert status for each asset When the last maintenance was performed on an asset When the next maintenance is scheduled for an asset Maintenance request for information, including creation date, due date, progress notes   Operations   The operator’s job is to keep the line running and make sure that it’s producing quality products. To do this, operators must keep track of how well their line is running (both in terms of speed and quality). They also need to be able to file maintenance requests when they have issues with the assets on their line. Required Functionality File maintenance request Get quality data from assets on their line Get performance data for the whole line Get a prioritized list of production orders for their line Create maintenance requests Required Information Individual asset performance metrics Full line performance metrics Product quality readings   Management   The production manager oversees the dispatch of production orders and ensures quotas are being met. Managers care about the productivity of all lines and the status of maintenance requests. Required Functional Create production orders Update production orders Cancel production orders Access line productivity data Elevate maintenance request priority Required Information Production line productivity levels (OEE) List of open maintenance requests   Step 3: Data Sources – Thing List     Thing List   Once you have identified the users' requirements, you'll need to determine what parts of your system must be connected. These will be the Things in your solution. Keep in mind that a Thing can represent many different types of connected endpoints. Here are some examples of possible Things in your system: Devices deployed in the field with direct connectivity or gateway-connectivity to Foundation Devices deployed in the field through third-party device clouds Remote databases Connections to external business systems (e.g., Salesforce.com, Weather.com, etc.)   Factory Example   In our Smart Factory example, we have already identified the users of the system and listed requirements for each of those users. The next step is to identify the Things in our solution. In our example, we are running a factory floor with multiple identical production lines. Each of these lines has multiple different devices associated with it. Let’s consider each of those items to be a connected Thing. Things in each line: Conveyor belt x 2 Pneumatic gate Robotic Arm Quality Check Camera Let's also assume we already have both a Maintenance Request System and a Production Order System that are in use today. To add this to our solution, we want to build a connector between Foundation and the existing system. These connectors will be Things as well. Internal system connection Thing for Production Order System Internal system connection Thing for Maintenance Request System NOTE: It is entirely possible to have scenarios in which you want to examine more granular-level details of your assets. For example, the arm and the hand of the assembly robot could be represented separately. There are endless possibilities, but for simplicity's sake, we will keep the list shorter and more high-level. Keep in mind that you can be as detailed as needed for this and future iterations of your solution. However, being too granular could potentially create unnecessary complexity and data overload.    Click here to view Part 2 of this guide.

Design Your Data Model Guide Part 2   Step 4: Data Sources – Component Breakout   Component Breakout     Once you have a full list of Things in your system (as well as requirements for each user), the next step is to identify the information needed from each Thing (based on the user's requirements). This involves evaluating the available data and functionality for each Thing. You then align the data and functionality with the user's requirements to determine exactly what you need, while eliminating that which you do not. This is important, as there can be cost and security benefits to only collecting data you need, and leaving what you don't. NOTE: Remember from the Data Model Introduction that a Thing's Components include Properties, Services, Events, and Subscriptions.   Factory Example   Using the Smart Factory example, let’s go through the different Things and break down each Thing's components that are needed for each of our users.   Conveyor Belts   The conveyor belt is simple in operation but could potentially have a lot of available data. Maintenance Engineer - needs to know granular data for the belt and if it has any alerts emergency shutdown (service) machine state (on/off) (property) serial number (property) last maintenance date (property) next scheduled maintenance date(property) power consumption (property) belt speed (property) belt motor temp (property) belt motor rpm (property) error notification (event) auto-generated maintenance requests (subscription) Operator - needs to know if the belt is working as intended belt speed (property) alert status (event) Production Manager - wants access to the data the Operator can see but otherwise has no new requirements   Robotic Arm   The robotic arm has 3 axes of rotation as well as a clamp hand. Maintenance Engineer - needs to know granular data for the arm and if it has any alerts time since last pickup (property): how long it has been since the last part was picked up by this hand? product count (property): how many products the hand has completed emergency shutdown (service) machine state (on/off) (property) serial number (property) last maintenance date (property) next scheduled maintenance date (property) power consumption (property) arm rotation axis 1 (property) arm rotation axis 2 (property) arm rotation axis 3 (property) clamp pressure (property) clamp status (open/closed) (property) error notification (event) 15.auto-generated maintenance requests (subscription) Operator - needs to know if the robotic arm is working as intended clamp status (open/closed) (property) error notification (event) product count (property): How many products has the hand completed? Production Manager - wants access to the data the Operator can see but otherwise has no new requirements   Pneumatic Gate   The pneumatic gate has two states, open and closed. Maintenance Engineer - needs to know granular data for the gate and if it has any alerts emergency shutdown (service) machine state (on/off) (property) serial number (property) last maintenance date (property) next scheduled maintenance date (property) power consumption (property) gate status (open/closed) (property) error notification (event) auto-generated maintenance requests (subscription) Operator - needs to know if the pneumatic gate is working as intended. gate status (open/closed) (property) error notification (event) The Production Manager wants access to the data the Operator can see but otherwise has no new requirements   Quality Control Camera   The QC camera uses visual checks to make sure a product has been constructed properly. Maintenance Engineer - needs to know granular data for the camera and if it has any alerts machine state (property): on/off serial number (property) last maintenance date (property) next scheduled maintenance date (property) power consumption (property) current product quality reading (property) images being read (property) settings for production quality assessment (property) error notification (event) auto-generated maintenance requests (subscription) product count (property): how many products the camera has seen Operator - needs to keep track of the quality check results and if there are any problems with the camera setup settings for production quality assessment (property) error notification (event) bad quality flag (event) product count (property): how many products the camera has seen Production Manager - wants access to the data the Operator can see but otherwise has no new requirements   Maintenance Request System Connector   Determining the data needed from the Maintenance Request System is more complex than from the physical components, as it will be much more actively used by all of our users. It is important to note that the required functionality already exists in our system as is, but it needs bridges created to connect it to a centralized system. Maintenance Engineer - needs to receive and update maintenance requests maintenance engineer credentials (property): authentication with the maintenance system endpoint configuration for connecting to the system (property) get unfiltered list of maintenance requests (service) update description of maintenance request (service) close maintenance request (service) Operator - needs to create and track maintenance requests operator credentials (property): authentication with the maintenance system endpoint configuration for connecting to the system (property) create maintenance request (service) get filtered list of maintenance requests for this operator (service) Production Manager - needs to monitor the entire system - both the creation and tracking of maintenance requests; needs to prioritize maintenance requests to keep operations flowing smoothly production manager credentials (property): authentication with the maintenance system endpoint configuration for connecting to the system (property) create maintenance request (service) get unfiltered list of maintenance requests (service) update priority of maintenance request (service)   Production Order System Connector   Working with the Production Order System is also more complex than the physical components of the lines, as it will be more actively used by two of the three users. It is important to note that the required functionality already exists in our existing production order system as is, but it needs bridges created to connect to a centralized system. Maintenance Engineer - will not need to know anything about production orders, as it is outside the scope of their job needs Operator - needs to know which production orders have been set up for the line, and needs to mark orders as started or completed operator credentials (property): authentication with the production order system endpoint configuration for connecting to the system (property) mark themselves as working a specific production line (service) get a list of filtered production orders for their line (service) update production orders as started/completed (service) Production Manager - needs to view the status of all production orders and who is working on which line production manager credentials (property): authentication with the production order system endpoint configuration for connecting to the system (property) get a list of production lines with who is working them (service) get the list of production orders with filtering options (service) create new production orders (service) update existing production orders for quantity, and priority (service) assign a production order to a production line (service) delete production orders (service)   Step 5: Data Sources – Thing-Component Matrix     Now that you have identified the Components necessary to build your solution (as well as the Things involved in enabling said Components), you are almost ready to create your Data Model design. Before moving onto the design, however, it is very helpful to get a good picture of how these Components interact with different parts of your solution. To do that, we recommend using a Thing-Component Matrix. A Thing-Component Matrix is a grid in which you will list Things in rows and Components in columns. This allows you to identify where there are overlaps between Components. From there, you can break those Components down into reusable Groups. Really, all you're doing in this step is taking the list of individual Things and their corresponding Components and organizing them. Instead of thinking of each item's individually-required functionality, you are now thinking of how those Components might interact and/or be reused across multiple Things.   Sample Thing-Component Matrix   As a generic example, look at the chart presented here.   You have a series of Things down the rows, while there are a series of Components (i.e. Properties, Services, Events, and Subscriptions) in the columns. This allows you to logically visually identify how some of those Components are common across multiple Things (which is very important in determining our recommendations for when to use Thing Templates vs. Thing Shapes vs. directly-instantiated Things). If we were to apply this idea to our Smart Factory example, we would create two sections of our Thing-Component Matrix, i.e. the Overlapping versus Unique Components. NOTE: It is not necessary to divide your Thing Component Matrix between Overlapping vs Unique if you don't wish to do so. It is done here largely for the sake of readability.   Overlapping Matrix   This matrix represents all the overlapping Components that are shared by multiple types of Things in our system:   Unique Matrix   This matrix represents the Components unique to each type of Thing:     Step 6: Model Breakdown         Breaking down your use case into a Data Model is the most important part of the design process for ThingWorx. It creates the basis for which every other aspect of your solution is overlaid. To do it effectively, we will use a multi-step approach. This will allow us to identify parts we can group and separate, leading to a more modular design.   Entity Relationship Diagram   To standardize the represention of Data Models, it is important to have a unified view of what a representation might look like. For this example, we have developed an Entity Relationship Diagram schematic used for Data Model representation. We will use this representation to examine how to build a Data Model.   Breakdown Process   ThingWorx recommends following an orderly system when building the specifics of your Data Model. You've examined your users and their needs. You've determined the real-world objects and systems you want to model. You've broken down those real-world items by their Component functionality. Now, you will follow these steps to build a specific Data Model for your application. Step Description 1 Prioritize the Groups of Components from your Thing-Component Matrix by each Group's Component quantity. 2 Create a base Thing Template for the largest group. 3 Iterate over each Group deciding which entity type to create. 4 Validate the design through instantiation. In the next several pages, we'll examine each of these steps in-depth.   Click here to view Part 3 of this guide.   

Get Started with ThingWorx for IoT Guide Part 1   Overview   This project will introduce you to the principles of ThingWorx Foundation by creating a working web application. Following the steps in this guide, you will create the building blocks of your first application for the Internet of Things (IoT). You will use ThingWorx Composer to create Thing Templates, which are then used to create Things that model the application domain. A simulator is imported to generate time-series data that is saved to a Value Stream. After modeling the application in ThingWorx Composer, you'll use Mashup Builder to create the web application Graphical User Interface (GUI). No coding is required in modeling the application, or in composing the web GUI that displays dynamically-generated data. NOTE: This guide’s content aligns with ThingWorx 9.3. The estimated time to complete ALL 5 parts of this guide is 30 minutes.      Step 1: Data Model   Model-based design with reusable building blocks makes your applications scalable and flexible. A ThingWorx application is built from Things, each based on a Thing Template that defines the common Properties (characteristics) and Services (behaviors) for a set of entities. Once a Thing Template is created, you can easily instantiate multiple Things without duplicating effort. In this tutorial, we will develop an application for a house including a thermostat, an electrical meter, and a sensor data simulator. We will demonstrate how to capture, store, and visualize data using the ThingWorx Foundation Server.   You will create Thing Shapes that model both a thermostat and an electric meter. You will then create a Thing Template that represents a house based on these shapes and other Properties.   Step 2: Create Thing Shapes Thing Shapes are components that contain Properties and Services. In Java programming terms, they are similar to an interface. In this section, you will build Thing Shapes for an electric meter and a thermostat. Meter Start on the Browse, folder icon tab of ThingWorx Composer. Under the Modeling section of the left-hand navigation panel hover over Thing Shapes, then click the + button.   Type MeterShape in the Name field. NOTE: Thing Shape names are case sensitive   If Project is not already set, choose PTCDefaultProject. Click Save. Add Properties Click Properties and Alerts tab at the top of your shape.   Click + Add. Enter the property name from the first row of the table below into the Name field of the Thing Shape Name Base Type Persistent? Logged? meterID STRING X   currentPower NUMBER   X costPerKWh NUMBER X X currentCost NUMBER     Select the Base Type from the drop-down menu that is listed in the table next to the Property name.   Check Persistent and/or Logged if there is an X in the table row of the Property. NOTE: When Persistent is selected, the property value will be retained when a Thing is restarted. Properties that are not persisted will be reset to the default during a restart. When Logged is selected, every property value change will be automatically logged to a specified Value Stream. Click ✓+ button. TIP: When adding multiple properties at once, click Done and Add after each, once you've entered a Name, selected a Base Type and any other criteria. If adding a single property, click Done. Repeat steps 2 through 6 for each of the properties in the rows of the table. Click the done ✓ Button. You'll see that these Properties have been created for the Meter Thing Shape.   Click Save. Thermostat This time we will use a shortcut to create a Thing Shape. In the top, left of the screen you will find +, click the new entity icon, then select Thing Shape from the list.   TIP: This is a shortcut you can use to create anything you can access from the Home tab in Composer. Type ThermostatShape in the Name field. If Project is not already set, choose PTCDefaultProject. Select the Properties and Alerts tab at the top. Click + Add and create the following properties following the same steps as before: Name Base Type Persistent? Logged? thermostatID STRING X   temperature NUMBER X X setTemperature NUMBER X X message STRING   X Click Save. You'll see that these Properties have been created for the Thermostat Thing Shape.       Step 3: Create Thing Template You can create reusable building blocks called Thing Templates in ThingWorx to maintain scalability and flexibility of your application development. With Thing Templates you define a set of similar objects by specifying the Properties (characteristics) and Services (behaviors) that are common for all the objects. In Java programming terms, a Thing Template is like an abstract class and can be created by extending other Thing Templates. Once a Thing Template is defined and saved in ThingWorx Foundation Server, you can replicate multiple Things to model a complete set without duplicating effort. In this step, you will create a Thing Template that defines properties for a building. This building Template could be used to create multiple Things that each represent a specific home, business, or other building structure. Start on the Browse, folder icon tab on the far left of ThingWorx Composer.   Under the Modeling section of the left-hand navigation panel, hover over Thing Templates and click the + button Type BuildingTemplate in the Name field. NOTE: Thing Template names are case sensitive If Project is not already set, click the + in the Project text box and select the PTCDefaultProject. In the Base Thing Template box, click + to choose GenericThing as the Template.   In the Implemented Shapes field, click the + to select the MeterShape Thing Shape.   Click Save. Add Properties In this step, you will specify the Properties that represent the characteristics of a building. Some Properties like the building location may never change (static), while other properties like power and temperature information may change every few seconds (dynamic). Select the Properties and Alerts tab under Thing Template: BuildingTemplate.   Click the Edit button if the Template is not already open for editing, then click + Add next to My Properties. Enter the property name in the Name field copied from a row of the table below, Select the Base Type of the property from the drop down menu. Check Persistent and/or Logged if there is an X in the table row of the Property. NOTE: When Persistent is selected, the property value will be retained during a system restart. Properties that are not persisted will be reset to the default during a system restart. When Logged is selected, every property value change will be automatically logged to a specified Value Stream. Click the ✓+ button. TIP: When adding multiple properties at once, click Check+ after each, once you've entered a Name, selected a Base Type and any other criteria. If adding a single property, click Check button. Repeat steps 3 through 6 for each of the properties in the rows of the table. Name Base Type Persistent Logged buildingID STRING x   building_lat_long LOCATION x   watts NUMBER x x After entering the final property, click the ✓ button. Click Save. You should see the following properties in your Composer.   In the next part of this introductory exercise, we will create a single Thing based on this Template to represent a house.

Distributed Timer and Scheduler Execution in a ThingWorx High Availability (HA) Cluster Written by Desheng Xu and edited by Mike Jasperson    Overview Starting with the 9.0 release, ThingWorx supports an “active-active” high availability (or HA) configuration, with multiple nodes providing redundancy in the event of hardware failures as well as horizontal scalability for workloads that can be distributed across the cluster.   In this architecture, one of the ThingWorx nodes is elected as the “singleton” (or lead) node of the cluster.  This node is responsible for managing the execution of all events triggered by timers or schedulers – they are not distributed across the cluster.   This design has proved challenging for some implementations as it presents a potential for a ThingWorx application to generate imbalanced workload if complex timers and schedulers are needed.   However, your ThingWorx applications can overcome this limitation, and still use timers and schedulers to trigger workloads that will distribute across the cluster.  This article will demonstrate both how to reproduce this imbalanced workload scenario, and the approach you can take to overcome it.   Demonstration Setup   For purposes of this demonstration, a two-node ThingWorx cluster was used, similar to the deployment diagram below:   Demonstrating Event Workload on the Singleton Node   Imagine this simple scenario: You have a list of vendors, and you need to process some logic for one of them at random every few seconds.   First, we will create a timer in ThingWorx to trigger an event – in this example, every 5 seconds.     Next, we will create a helper utility that has a task that will randomly select one of the vendors and process some logic for it – in this case, we will simply log the selected vendor in the ThingWorx ScriptLog.     Finally, we will subscribe to the timer event, and call the helper utility:     Now with that code in place, let's check where these services are being executed in the ScriptLog.     Look at the PlatformID column in the log… notice that that the Timer and the helper utility are always running on the same node – in this case Platform2, which is the current singleton node in the cluster.   As the complexity of your helper utility increases, you can imagine how workload will become unbalanced, with the singleton node handling the bulk of this timer-driven workload in addition to the other workloads being spread across the cluster.   This workload can be distributed across multiple cluster nodes, but a little more effort is needed to make it happen.   Timers that Distribute Tasks Across Multiple ThingWorx HA Cluster Nodes   This time let’s update our subscription code – using the PostJSON service from the ContentLoader entity to send the service requests to the cluster entry point instead of running them locally.       const headers = { "Content-Type": "application/json", "Accept": "application/json", "appKey": "INSERT-YOUR-APPKEY-HERE" }; const url = "https://testcluster.edc.ptc.io/Thingworx/Things/DistributeTaskDemo_HelperThing/services/TimerBackend_Service"; let result = Resources["ContentLoaderFunctions"].PostJSON({ proxyScheme: undefined /* STRING */, headers: headers /* JSON */, ignoreSSLErrors: undefined /* BOOLEAN */, useNTLM: undefined /* BOOLEAN */, workstation: undefined /* STRING */, useProxy: undefined /* BOOLEAN */, withCookies: undefined /* BOOLEAN */, proxyHost: undefined /* STRING */, url: url /* STRING */, content: {} /* JSON */, timeout: undefined /* NUMBER */, proxyPort: undefined /* INTEGER */, password: undefined /* STRING */, domain: undefined /* STRING */, username: undefined /* STRING */ });   Note that the URL used in this example - https://testcluster.edc.ptc.io/Thingworx - is the entry point of the ThingWorx cluster.  Replace this value to match with your cluster’s entry point if you want to duplicate this in your own cluster.   Now, let's check the result again.   Notice that the helper utility TimerBackend_Service is now running on both cluster nodes, Platform1 and Platform2.   Is this Magic?  No!  What is Happening Here?   The timer or scheduler itself is still being executed on the singleton node, but now instead of the triggering the helper utility locally, the PostJSON service call from the subscription is being routed back to the cluster entry point – the load balancer.  As a result, the request is routed (usually round-robin) to any available cluster nodes that are behind the load balancer and reporting as healthy.   Usually, the load balancer will be configured to have a cookie-based affinity - the load balancer will route the request to the node that has the same cookie value as the request.  Since this PostJSON service call is a RESTful call, any cookie value associated with the response will not be attached to the next request.  As a result, the cookie-based affinity will not impact the round-robin routing in this case.   Considerations to Use this Approach   Authentication: As illustrated in the demo, make sure to use an Application Key with an appropriate user assigned in the header. You could alternatively use username/password or a token to authenticate the request, but this could be less ideal from a security perspective.   App Deployment: The hostname in the URL must match the hostname of the cluster entry point.  As the URL of your implementation is now part of your code, if deploy this code from one ThingWorx instance to another, you would need to modify the hostname/port/protocol in the URL.   Consider creating a variable in the helper utility which holds the hostname/port/protocol value, making it easier to modify during deployment.   Firewall Rules: If your load balancer has firewall rules which limit the traffic to specific known IP addresses, you will need to determine which IP addresses will be used when a service is invoked from each of the ThingWorx cluster nodes, and then configure the load balancer to allow the traffic from each of these public IP address.   Alternatively, you could configure an internal IP address endpoint for the load balancer and use the local /etc/hosts name resolution of each ThingWorx node to point to the internal load balancer IP, or register this internal IP in an internal DNS as the cluster entry point.

Is your team operating an effective DevOps pipeline? DevOps is an important part of a mature, enterprise ready application, but the process isn’t simple.   This expert session will focus on how a full DevOps pipeline looks like and how PTC can help to build a seamless pipeline. Join us for our upcoming Expert Session to learn how to create a Docker image, integrate Azure with Docker and Git, and set up a seamless DevOps pipeline.   When? Thursday, September 30th 2021 | 11 AM EST Host: Tori FIrewind, Senior Engineer in PTC IOT Enterprise Deployment Center Registration link: https://www.ptc.com/en/resources/iiot/webcast/devops-pipeline-thingworx 

With ThingWorx, we can already use univariate anomaly alerts (on a single sensor value). However, in many situations, the readings from an individual sensor may not tell you much about the overall issue and a multivariate anomaly detector can be more useful. This post is intended to provide an overview of the Azure Anomaly Detector and how it can be integrated with ThingWorx. The attachment contains: A document with detailed instructions about the setup; A .csv file with the multivariate timeseries dataset; A .twx file with some entities that need to be imported in ThingWorx as well as the CSVParser extension that needs to be installed; A .zip file that will need to uploaded in an Azure Blob Container at some point in the setup

Those who have been working with ThingWorx for many years will have noticed the work done around ingress stress testing and performance optimization.  Adding InfluxDB as a time-series data persistence provider really helped level up these capabilities while simultaneously decreasing the overall resources required by the infrastructure.  However with this ease comes a hidden challenge: query and data processing performance to work it into something useful.   Often It's Too Much Data In general most customers that I work with want to collect far too much data -- without knowing what it will be used for, or what processing will be required in order to make it usable and useful.  This is a trap in general with how many people envision IoT projects, being told by infrastructure providers that cloud storage and compute resources are abundant and cheap and that they should get as much data as possible.  This buildup of data means that more effort needs to be spent working it into something useful (data engineering/feature extraction) and addressing common data issues (quality, gaps, precision, etc.).  This might be fine for mature companies with large data analytics teams; however this is a makeup that I've only seen in the largest of our customers.  Some advice - figure out what you need and how you'll use it, and then collect that.  Work on extracting value today rather than hoping that extra data collected  now will provide some insights years from now.   Example - Problem Statement You got your Thing Model designed, and edge devices connected.  Now you've got data flowing in and being stored every 5 seconds in InfluxDB.  Great progress!  Now on to building the applications which cover the various use cases. The raw data is most likely going to need to be processed and potentially even significantly transformed into other information in order to make it useful.  Turning a "powered on and running" BOOLEAN to an "hour meter" INTEGER is a simple example.  Then you may need to provide a report showing equipment run time hours by day over a month.  The maintenance team may also have asked to look for usage patterns which lead to breakdowns, requiring extracting other data points from the initial one like number of daily starts, average daily run time, average time between restarts. The problem here is that unless you have prepared these new data points and stored them as well (say in a Stream), you are going to have to build these data sets on the fly, and that can be time and resource intensive and not give you the response time expected.  As you can imagine, repeatedly querying and processing large volumes of unchanging raw data is going to have resource and time implications - so this is why data collection and data use need to be thought about separately.   Data Engineering In the above examples, the key is actually creating new data points which are calculated progressively throughout normal operation.  This not only makes the information that you want available when you need it - in the right format - but it also significantly reduces resource requirements by constantly reprocessing raw data.  It also helps managing data purging, because as you create and store usable insights, you can eventually just archive away your old raw data streams.   Direct Database Queries vs. Thingworx Data Services Despite the above being a rule of thumb, sometimes a simple well structured database query can get you exactly what you need and do so quite quickly.  This is especially true for InfluxDB when working with extremely large time-series datasets.  The challenge here is that ThingWorx persistence providers abstract away the complexity of writing ones own database queries, so we can't easily get at the databases raw power and are forced to query back more data than needed and work it into a usable format in memory (which is not fast).   Leveraging the InfluxDB API using the ContentLoader Technique As InfluxDBs API is 100% REST, we can access it using in-built ThingWorx Content Loader services.  Check out this demonstration and explanation video where I talk about how to interact directly with InfluxDB in order to crush massive time-series data and get back much more usable and manageable data sets.  It is important to note here that you should use a read-only database user here, as you should never modify the ThingWorx databases to avoid untested scenarios which may lead to data corruption.   Optimizing ThingWorx query performance with the InfluxDB REST API - YouTube InfluxToolBox ThingWorx demo project (by T. Wobben)      

We will host a live Expert Session: "5 Common Mistakes for Developing Scalable IoT Applications" on June 22nd, 11h00 EST.   Please find below the description of the expert session and the registration link.   Expert Session: 5 Common Mistakes for Developing Scalable IoT Applications Date and Time: June 22nd, 11h00 EST Duration: 1 hour Host: Tori Firewind, Mike Jasperson and Prachi Rath - Enterprise Deployment Center Registration Here: https://www.ptc.com/en/resources/iiot/webcast/5-common-dev-mistakes-for-scalable-iot-applications    Description: To build scalable applications, it’s necessary to identify the common mistakes made and ensure to avoid them at the early stages of development.   In this expert session, the PTC Enterprise Deployment Team will elaborate on why scalability is important and how one can avoid the common development pitfalls in IoT.    Existing Recorded sessions can be found on support portal using the keyword ‘Expert Sessions’. You can also suggest topics for upcoming sessions using this small form.   Here are some recorded sessions that might be of your interest. You can find recordings for the full library of webinars using the keyword ‘Expert Sessions’ in PTC support portal search Thingworx Active Active Clustering This session will cover the main aspects of the High Availability Clustering feature launched with the ThingWorx 9.0 release.   Recoding Link Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts This session highlights the key points you should evaluate to properly plan your upgrade to Thingworx 9. Recording Link Top 5 items to check for Thingworx Performance Troubleshooting How to troubleshoot performance issues in a Thingworx Environment? Here we cover the top 5 investigation steps that will help you understand the source of your environment issues and allow better communication with PTC Technical Support     Recording Link

Recently I have been accompanying an integration partner and end customer around an issue experienced with ThingWorx resource exhaustion.  Early on it seemed like this was an issue with the ThingWorx Azure IoT Hub Connector as it would freeze up and become unresponsive.  Following a root cause analysis it became clear that it was actually caused by a lack of a number of standard cloud design patterns, which if used would have automatically adapted operation of the overall solution to be far more resilient as well as resource optimized.   The way that the logic was structured, it prioritized job execution on entities with the oldest last success time and would continue to retry these executions (IoT Direct Methods) every few seconds until successful.  There were a number of problems here, but I'll unpack a few in order to tie the problem to the solution via design patterns.   1) No exception handling When the direct method execution failed/timed out or the system reported being unable to execute the remote service, this response was not used to adapt the solutions behavior. 2) No backoff retry mechanism As exceptions were not caught, an adaptive retry mechanism with incremental or exponential backoff could not be leveraged to limit the impact of the build up of the failing retries. 3) No exception tracking Tracking that exceptions were occurring and counting them would allow powering an exponential backoff retry algorithm (with jitter), a Cancel or Circuit Breaker pattern (stop doing something which is just broken), as well as provided alerting to address specific areas of the distributed solution experiencing issues. 4) Conflicting priorities It was interesting to see the manifestation of the conflicting interests of wanting to ensure checks and balances (had all needed data) and system resiliency.  Retries and resource usage built up exponentially due to the transient error instead of backing them off.  Trying so hard to get the needed data from failing sensors meant that operational sensors were deprioritized and their data was not received either - spreading the localized issue to the whole system.   Around the time that I shared my recommendations and some examples of how to make the solution more resilient, one of my technical colleagues at Microsoft shared some extremely interesting and relevant design patterns documented by Microsoft as a part of the "Microsoft Azure Well-Architected Framework".  This framework with included Design Patterns for specific cloud application goals allows applying well-known industry standard approaches to dealing with the challenges of large scale distributed enterprise systems (reliability, performance, cost optimization).   She later then shared this blog post describing exactly the exponential backoff retry with jitter pattern which we had together recommended to the systems integrator.   What's interesting for us ThingWorx people is that this framework from Microsoft is about well-architected cloud solutions and does not specifically reference the Azure stack, and as such many of these approaches and design practices can be employed in your ThingWorx applications.  What are you waiting for?  Go check them out!

In our interactions with PTC customers we often learn they have previously performed Analytics modeling in Python, Matlab, R, or even built home grown analyses in languages such as Java or C++. As expected, when adopting an Industrial Innovation Platform such as ThingWorx that also has its own ThingWorx Analytics module, customers do not want to reimplement everything from scratch and would rather integrate their previous work in the Smart Applications built in ThingWorx, leveraging a combination of their existing toolset together with ThingWorx Analytics modeling. That is certainly possible and there are multiple ways to do that. In this article we will focus on several general ways to make that happen, but it is important to keep in mind that language specific approaches are also possible and we are happy to discuss those in the specific context of the customer.   Here are five different ways to bring existing Analytics into ThingWorx: If the task is to reuse an existing predictive model developed in a language such as Python/R/Matlab, typically one can export that model in PMML (Predictive Model Markup Language), an xml format, and import it in ThingWorx Analytics using the AnalyticsServer_ResultsThing -> UploadModel service. Libraries such as sklearn2pmml & r2pmml can be utilized towards that goal. The imported model can then be used in the same fashion as a ThingWorx Analytics developed model to power smart applications built in ThingWorx. If the Analysis involves more complex tasks than Predictive Modeling, such as custom data normalizations or non-standard Machine Learning models or home grown algorithms, one can use the options below. Call the ThingWorx exposed REST Web API from Python/Matlab/R/Java/Javascript. Every service from ThingWorx can be called that way, and the API can also be used to push analyses results into ThingWorx for further consumption, perhaps together with other sources of data such as sensor readings, in the smart applications built there. The documentation for the ThingWorx REST API can be found here.  Expose the existing Analytics via using a thin layer of REST Web Services. For example, in Python, this can be done using Flask, with few lines of code. Then, the orchestration can happen from ThingWorx by calling the exposed Web Service and weaving the results back into smart applications. Often our customers' current architecture involves a relational database (e.g. SQL Server, Oracle, etc) that is powering the existing Analytics, and stores the end results (predictions, correlations, etc). In this scenario, we can connect ThingWorx directly to that database to read these results.  Finally, in the case of complex Analytics, where a tighter integration with ThingWorx is desired, existing Analytics / algorithms can be wrapped into a ThingWorx Extension or an Analytics Provider using the corresponding PTC SDKs.  When choosing an integration option, customers need to carefully balance complexity of integration, constraints of their architecture, Analytics modeling complexity, as well as end user consumption requirements.

Check our expert session recorded library! The recordings will also be published in our Customer events library, posted on each event. Stay tunned!   Your feedback is very important to us! After watching the recordings, please take 2 min to complete this survey   Thingworx Foundation Session Name Link Duration Thingworx Mashup 101 - Do's and Don'ts Recording link 00:33:41 Thingworx Active Active Clustering (High Availability Recording link 00:26:24 Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts Recording link 00:27:02 Thingworx Flow Overview Recording link 00:43:40 Top 5 items to check for Thingworx Performance Troubleshooting Recording link 00:26:55 ThingWorx DEVOPS QuickStart Guide Recording link 00:45:05 ThingWorx Backup And Recovery Recording Link 00:20:14 Expert Session - Designing your Data Model in Thingworx Recording link 00:26:45 ThingWorx Installation Recording link 00:15:07 Expert Session - Introduction To Edge Connectivity Recording link 00:15:56 Expert Session - Basic Mashup Design in Thingworx Recording link 00:36:31 Expert Session - Extensions101 Recording Link 00:30:08 Expert Session – Developing your Data Model in Thingworx Recording link 00:39:19 Thingworx Scalability Recording link 00:09:18 Expert Sessions - ThingWorx Patch Upgrade Recording link 00:03:19   Thingworx Navigate Session Name Link Duration Understanding license requirements for Thingworx Navigate Recording link 00:32:40 Navigate SSL and Authentication Recording Link 00:34:30 Navigate 3D Viewer Recording Link 00:43:25 Component Based App Development Recording Link 00:24:07 Navigate 9.0 – What’s new Recording link 00:27:07 Overview of SSO Implementation for ThingWorx Navigate and Windchill with PingFederate Recording link 00:18:36 Identifying the right SSO mix for Navigate 1 6 Recording link 00:57:56 Navigate Configuration - PingFederate Automation Script Recording link 00:51:07 Expert Session - Navigate Configuration/Windchill Authentication Recording link 00:23:07 What’s new with Navigate 1.8 and the new Navigate 1.8 installer Recording link 01:05:26 Creating an I*E task for use in Navigate Recording link 00:05:36   Vuforia Expert Capture Session Name Link Duration VEC In a Nutshell Video Link 00:31:39

We will host a live Expert Session: "Top 5 Thingworx environment monitoring best practices" on March 25th, 10h00 EST.   Please find below the description of the expert session and the registration link.   Expert Session: Top 5 Thingworx environment monitoring best practices Date and Time: March 25th, 10h00 EST Duration: 1 hour Host: Tori Firewind, Tim Atwood and Dave Bernbeck from Enterprise Deployment Center Registration Here: https://www.ptc.com/en/resources/iiot/webcast/top-5-thingworx-monitoring-best-practices    In this session, we will be reviewing the main monitoring practices to keep a heathy environment and discuss the main issues from the audience. Bring your questions!.   Existing Recorded sessions can be found on support portal using the keyword ‘Expert Sessions’. You can also suggest topics for upcoming sessions using this small form.   Here are some recorded sessions that might be of your interest. You can find recordings for the full library of webinars using the keyword ‘Expert Sessions’ in PTC support portal search   Thingworx Active Active Clustering This session will cover the main aspects of the High Availability Clustering feature launched with the ThingWorx 9.0 release.   Recoding Link Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts This session highlights the key points you should evaluate to properly plan your upgrade to Thingworx 9. Recording Link Top 5 items to check for Thingworx Performance Troubleshooting How to troubleshoot performance issues in a Thingworx Environment? Here we cover the top 5 investigation steps that will help you understand the source of your environment issues and allow better communication with PTC Technical Support     Recording Link

We will host a live Expert Session: "Top 5 Thingworx environment monitoring best practices" on March 25th, 10h00 EST.   Please find below the description of the expert session and the registration link.   Expert Session: Top 5 Thingworx environment monitoring best practices Date and Time: March 25th, 10h00 EST Duration: 1 hour Host: Tori Firewind, Tim Atwood and Dave Bernbeck from Enterprise Deployment Center - Enterprise Deployment Center Registration Here: https://www.ptc.com/en/resources/iiot/webcast/top-5-thingworx-monitoring-best-practices    In this session, we will be reviewing the main monitoring practices to keep a heathy environment and discuss the main issues from the audience. Bring your questions!.   Existing Recorded sessions can be found on support portal using the keyword ‘Expert Sessions’. You can also suggest topics for upcoming sessions using this small form.   Here are some recorded sessions that might be of your interest. You can find recordings for the full library of webinars using the keyword ‘Expert Sessions’ in PTC support portal search   Thingworx Active Active Clustering This session will cover the main aspects of the High Availability Clustering feature launched with the ThingWorx 9.0 release.   Recoding Link Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts This session highlights the key points you should evaluate to properly plan your upgrade to Thingworx 9. Recording Link Top 5 items to check for Thingworx Performance Troubleshooting How to troubleshoot performance issues in a Thingworx Environment? Here we cover the top 5 investigation steps that will help you understand the source of your environment issues and allow better communication with PTC Technical Support     Recording Link

We will host a live Expert Session: "Thingworx Mashup 101 - Do's and Don'ts" on February 24th, 13h30 EST.   Please find below the description of the expert session and the registration link.   Expert Session: Thingworx Mashup 101 - Do's and Don'ts Date and Time: February 24th, 13h30 EST Duration: 1 hour Host: Aanjan Ravi - Technical Product Manager Registration Here: https://www.ptc.com/en/events/thingworx-mashup-101   Description: This session covers the most common and useful tips about how to correctly use Mashup builder, Widgets and Layouts – and what to avoid -  to create applications with good principles of UI/UX and easier to maintain.   Existing Recorded sessions can be found on support portal using the keyword ‘Expert Sessions’. You can also suggest topics for upcoming sessions using this small form.   Here are some recorded sessions that might be of your interest. You can find recordings for the full library of webinars using the keyword ‘Expert Sessions’ in PTC support portal search Thingworx Active Active Clustering This session will cover the main aspects of the High Availability Clustering feature launched with the ThingWorx 9.0 release.   Recoding Link Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts This session highlights the key points you should evaluate to properly plan your upgrade to Thingworx 9. Recording Link Top 5 items to check for Thingworx Performance Troubleshooting How to troubleshoot performance issues in a Thingworx Environment? Here we cover the top 5 investigation steps that will help you understand the source of your environment issues and allow better communication with PTC Technical Support     Recording Link

Hi All,   We will host a live Expert Session: "Thignworx Active Active Clustering" on January 21th 8h00 EST. Please find below the description of the expert session and the registration link.   Expert Session: Thignworx Active Active Clustering Date and Time: January 21th 8h00 EST Duration: 1 hour Host: Ayush Tiwari - IoT Product Manager Registration Here: https://www.ptc.com/en/customer-success/expert-sessions-for-thingworx-foundation-webcasts (scroll down, the session is in the bottom of the page)   Description: This session will cover the main aspects of the High Availability Clustering feature for High Availability configuration launched with the ThingWorx 9.0 release. Join us and bring your questions with you!    Existing Recorded sessions can be found on support portal using the keyword ‘Expert Sessions’. You can also suggest topics for upcoming sessions using this small form.   Here are some recorded sessions that might be of your interest. You can find recordings for the full library of webinars using the keyword ‘Expert Sessions’ in PTC support portal search Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts This session highlights the key points you should evaluate to properly plan your upgrade to Thingworx 9   Recording Link Thingworx Flow Overview Flow is a powerful component of the ThingWorx platform.  This session will take the Flow discussion beyond basic applications and into more customized and complex solutions.​ This will focus on use cases, main features such as triggers, connector options, main enhancements for Thingworx 9.0 and a short demonstration   Recoding Link

Hi All,   We will host a live Expert Session: "Thignworx Active Active Clustering" on January 21th 8h00 EST. Please find below the description of the expert session and the registration link.   Expert Session: Thignworx Active Active Clustering Date and Time: January 21th 8h00 EST Duration: 1 hour Host: Ayush Tiwari - IoT Product Manager Registration Here: https://www.ptc.com/en/customer-success/expert-sessions-for-thingworx-foundation-webcasts (scroll down, the session is in the bottom of the page)   Description: This session will cover the main aspects of the High Availability Clustering feature for High Availability configuration launched with the ThingWorx 9.0 release.   Existing Recorded sessions can be found on support portal using the keyword ‘Expert Sessions’. You can also suggest topics for upcoming sessions using this small form.   Here are some recorded sessions that might be of your interest. You can find recordings for the full library of webinars using the keyword ‘Expert Sessions’ in PTC support portal search Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts This session highlights the key points you should evaluate to properly plan your upgrade to Thingworx 9   Recording Link Thingworx Flow Overview Flow is a powerful component of the ThingWorx platform.  This session will take the Flow discussion beyond basic applications and into more customized and complex solutions.​ This will focus on use cases, main features such as triggers, connector options, main enhancements for Thingworx 9.0 and a short demonstration   Recoding Link

We will host a live Expert Session: "Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts" on January 12th 8h00 EST.   Please find below the description of the expert session and the registration link: Expert Session: Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts Date and Time: January 12th 8h00 EST Duration: 1 hour Host: Ayush Tiwari - IoT Product Manager Registration Here: https://www.ptc.com/en/customer-success/expert-sessions-for-thingworx-foundation-webcasts    Description: This session will highlight the key points you should evaluate to properly plan your upgrade to Thingworx 9.   Existing Recorded sessions can be found on support portal using the keyword ‘Expert Sessions’. You can also suggest topics for upcoming sessions using this small form.   Here are some recorded sessions that might be of your interest. You can find recordings for the full library of webinars using the keyword ‘Expert Sessions’ in PTC support portal search.   Thingworx Flow Overview Flow is a powerful component of the ThingWorx platform.  This session will take the Flow discussion beyond basic applications and into more customized and complex solutions.​ This will focus on use cases, main features such as triggers, connector options, main enhancements for Thingworx 9.0 and a short demonstration   Recoding Link Top 5 items to check for Thingworx Performance Troubleshooting How to troubleshoot performance issues in a Thingworx Environment? Here we cover the top 5 investigation steps that will help you understand the source of your environment issues and allow better communication with PTC Technical Support     Recording Link

We will host a live Expert Session: "Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts" on January 12th 8h00 EST.   Please find below the description of the expert session and the registration link: Expert Session: Upgrade to Thingworx 9 – How to Plan / Evaluate Impacts Date and Time: January 12th 8h00 EST Duration: 1 hour Host: Ayush Tiwari - IoT Product Manager Registration Here: https://www.ptc.com/en/customer-success/expert-sessions-for-thingworx-foundation-webcasts    Description: This session will highlight the key points you should evaluate to properly plan your upgrade to Thingworx 9.   Existing Recorded sessions can be found on support portal using the keyword ‘Expert Sessions’. You can also suggest topics for upcoming sessions using this small form.   Here are some recorded sessions that might be of your interest. You can find recordings for the full library of webinars using the keyword ‘Expert Sessions’ in PTC support portal search.   Thingworx Flow Overview Flow is a powerful component of the ThingWorx platform.  This session will take the Flow discussion beyond basic applications and into more customized and complex solutions.​ This will focus on use cases, main features such as triggers, connector options, main enhancements for Thingworx 9.0 and a short demonstration   Recoding Link Top 5 items to check for Thingworx Performance Troubleshooting How to troubleshoot performance issues in a Thingworx Environment? Here we cover the top 5 investigation steps that will help you understand the source of your environment issues and allow better communication with PTC Technical Support     Recording Link