IoT Tips

Anomaly Detection (also known as Outlier Detection) is a set of techniques that identify unusual occurrences in data. The premise is that such occurrences may be early indicators of future negative events (e.g. failure of assets or production lines).  Data Science algorithms for Anomaly Detection include both Supervised and Unsupervised methods. In Unsupervised Anomaly Detection, the algorithms make the assumption that most of the data points are "normal" (e.g. normal operation of the asset) and are looking for data points that are most dissimilar to the remainder of the dataset. Supervised Anomaly Detection requires a labeled set of Anomalies, in which case predictive algorithms can be applied directly on this data.   Thingworx employs a number of algorithms in support of Anomaly Detection: Simple threshold alerts. These are easy to setup on Thing properties but require a domain expert to provide such thresholds. Then, the alert will automatically fire when the value of the monitored property goes outside the predefined range of values, often seen as the "bad" side of the threshold. Statistical Process Control (SPC). This can be implemented using Thingworx Analytics Property Transforms. Most companies use a subset of SPC charts and rules to monitor production processes. Examples include the X Bar and R charts, as well as the Western Electric rules (e.g. one point outside the average +/- three sigma range). Explainable and widely accepted, SPC can also provide an earlier warning system compared to simple threshold alerts, in that it captures more complex patterns. Clustering. Using Thingworx Analytics, one can build an optimal clustering for the available data points. Under the assumption that data is representative of mostly normal operation and that there is not a significant pre-defined pattern of anomalies that form their own cluster, one can identify outliers by looking at the distance between points and their corresponding cluster centers. Points that are very far from their corresponding cluster center can be labeled as anomalies. Semi-supervised Anomaly Alerting (formerly known as ThingWatcher). This functionality identifies single property time series behavior that is statistically different than what was seen in a finite window of “known normal operation”. As such, it does not identify a “bad” event, or even a precursor to a “bad” event.  Rather, it points the end user to further investigate a situation which may lead to a “bad” event. Anomaly Alerts can be easily setup like any other Alert on a Thing property. Multiple Anomaly Alerts can be setup on the same or different properties of a Thing. Behind the scenes, the platform builds a time series neural network model for the known normal operation data, which is then applied to incoming data, and, if the errors are significantly different than those on known normal operation over a period of time, then an Anomaly Alert is produced. The techniques mentioned above are either unsupervised or semi-supervised. If the dataset contains labeled anomalies (e.g. asset faults, or suspicious patterns) then supervised predictive techniques (such as regression, decision trees, neural nets, or ensemble methods) are available to model the relationships between such anomalies (dependent variables) and various variables of interest (independent variables). These models can then be employed to monitor assets or production lines for upcoming anomalies. In many real-world use cases, anomalies are relatively rare; care needs to be taken when building such predictive models. Techniques such as up-sampling can prove beneficial in such situations.   What constitutes an Anomaly depends on the observed data and the current context. If only few data points are initially available, then it is possible that a lot of future data is predicted as an Anomaly, despite being normal operation. Also, in terms of context, if an Anomaly Detection is trained on a connected product in the Winter, it is likely to say all Summer operation is anomalous. This can be tackled by having multiple anomaly detection alerts implemented, one for each different context of operation (e.g. season, recipe being manufactured, operation done by a robot).   Another consideration is lead time vs explainability. For example, when a threshold alert fires, it is obvious why, but it may not be early enough to take action. As more advanced methods are employed, more complex patterns can be captured, hence more lead time, but typically at the expense of explainability. For example, semi supervised Anomaly Alerting (formerly known as ThingWatcher) uses time windows, aggregations, and derivatives of up to the third order, resulting in significantly less explainability when an Anomaly is presented to the end user.   Choosing the appropriate Anomaly Detection technique is use case dependent, balancing the desired lead time and explainability. If historical data on failures/anomalies is not available, a good place to start is Statistical Process Control, as it provides a balanced approach between the two dimensions, in addition to being already in use across many manufacturing companies.

    Raise your hand if you’re ready for seamless, rapid deployment of your ThingWorx applications with visibility into your various environments! It’s time to say goodbye to error-prone deployments with manual dependency tracking and hello to Solution Central!   Releasing this fall as part of 8.5, Solution Central is a brand-new cloud service coming to the ThingWorx platform to enable you to efficiently manage your ThingWorx applications across the enterprise.   With Solution Central, you’ll no longer be caught chasing missing dependencies (like ThingShapes, Mashups, templates or library extensions). Solution Central automatically identifies and packages up the dependencies required for your application. No more manual dependency madness!   Whether you’re managing many apps deployed to a few environments or a single app deployed to hundreds of environments, Solution Central allows you to accelerate your deployment through an intuitive UI or powerful APIs for automation.   Here’s how it works: Begin by creating your application in Composer with a project. Let Solution Central automatically package up all the artifacts and dependencies required for your application. Allow Solution Central to publish your solution package to the cloud. Deploy your application to your various environments (local servers, data centers, cloud systems) directly from Solution Central. It’s like your company has its own private app store. Here’s a sneak peek of the Solution Central UI! Keep an eye out for the release of ThingWorx 8.5 at the end of Sept 2019 and begin accelerating your app deployment! Check out the presentation and demo my fellow PM Chris Baldwin and I delivered at LiveWorx19—and be sure to attend LiveWorx20! To navigate to our session recording, search for “Introducing Solution Central: Your Gateway to Accelerated IIoT Value Across the Enterprise” here.   Sound interesting? Message me directly to discover how you can become part of the Solution Central Private Preview Program!   -Kaya  

Smoothing Large Data Sets Purpose In this post, learn how to smooth large data sources down into what can be rendered and processed more easily on Mashups. Note that the Time Series Chart  widget is limited to load 8,000 points (hard-coded). This is because rendering more points than this is almost never necessary or beneficial, given that the human eye can only discern so many points and the average monitor can only render so many pixels. Reducing large data sources through smoothing is a recommended best practice for ThingWorx, and for data analysis in general.   To show how this is done, there are sample entities provided which can be downloaded and imported into ThingWorx. These demonstrate the capacity of ThingWorx to reduce tens of thousands of data points based on a "smooth factor" live on Mashups, without much added load time required. The tutorial below steps through setting these entities up, including the code used to generate the dummy data.   Smoothing the Data on Mashups Create a Value Stream for storing the historical data. Create a Data Shape for use in the queries. The fields should be: TestProperty - NUMBER timestamp - DATETIME Create a Thing (TestChartCapacityThing) for simulating property updates and therefore Value Stream updates. There is one property: TestProperty - NUMBER - not persistent - logged The custom query service on this Thing (QueryNamedPropertyHistory) will have the logic for smoothing the data. Essentially, many points are averaged into one point, reducing the overall size, before the data is returned to the mashup. Unfortunately, there is no service built-in to do this (nothing OOTB service). The code is here (input parameters are to - DATETIME; from - DATETIME; SmoothFactor - INTEGER): // This is just for passing the property name into the query var infotable = Resources["InfoTableFunctions"].CreateInfoTable({infotableName: "NamedProperties"}); infotable.AddField({name: "name", baseType: "STRING"}); infotable.AddRow({name: "TestProperty"}); var queryResults = me.QueryNamedPropertyHistory({ maxItems: 9999999, endDate: to, propertyNames: infotable, startDate: from }); // This will be filled in below, based on the smoothing calculation var result = Resources["InfoTableFunctions"].CreateInfoTable({infotableName: "SmoothedQueryResults"}); result.AddField({name: "TestProperty", baseType: "NUMBER"}); result.AddField({name: "timestamp", baseType: "DATETIME"}); // If there is no smooth factor, then just return everything if(SmoothFactor === 0 || SmoothFactor === undefined || SmoothFactor === "") result = queryResults; else { // Increment by smooth factor for(var i = 0; i < queryResults.rows.length; i += SmoothFactor) { var sum = 0; var count = 0; // Increment by one to average all points in this interval for(var j = i; j < (i+SmoothFactor); j++) { if(j < queryResults.rows.length) if(j === i) { // First time set sum equal to first property value sum = queryResults.getRow(j).TestProperty; count++; } else { // All other times, add property values to first value sum += queryResults.getRow(j).TestProperty; count++; } } var average = sum / count; // Use count because the last interval may not equal smooth factor result.AddRow({TestProperty: average, timestamp: queryResults.getRow(i).timestamp}); } } Create a Timer for updating the property values on the Thing. The Timer should subscribe to itself, containing this code (ensure it is enabled as well): var now = new Date(); if(now.getMilliseconds() % 3 === 0) // Randomly reset the number to simulate outliers Things["TestChartCapacityThing"].TestProperty = Math.random()*100; else if(Things["TestChartCapacityThing"].TestProperty > 100) Things["TestChartCapacityThing"].TestProperty -= Math.random()*10; else Things["TestChartCapacityThing"].TestProperty += Math.random()*10; Don't forget to set the runAsUser in the Timer configuration. To generate many properties, set the updateRate to a small value, like 10 milliseconds. Disable the Timer after many thousands of properties are logged in the Value Stream. Create a Mashup for displaying the property data and capacity of the query to smooth the data. The Mashup should run the service created in step 4 on load. The service input comes from widgets on the mashup: Bindings: Place a Time Series Chart widget in the bottom of the Mashup layout. Bind the data from the query to the chart. View the Mashup. Note the difference in the data... All points in one minute: And a smooth factor of 10 in one minute: Note that the outliers still appear, and the peaks are much easier to see. With fewer points, trends become easier to spot and data is easier to understand. For monitoring the specific nature of the outliers, utilize alerts and other types of displays. Alternative forms of data reduction could involve using the mean of each interval (given by the smoothing factor) or the min or max, as needed for the specific use case. Display multiple types of these options for an even more detailed view. Remember, though, the more data needs to be processed, the slower the Mashup will load. As usual, ensure all mashups are load tested and that the number of end users per Mashup is considered during application design.

  When it’s time to make pizza, most of us head to the fridge for our bag of dough; we don’t head for the flour and yeast to start from scratch. So, why would your ThingWorx apps be any different? Start with pre-built solutions like Asset Advisor to rapidly create health monitoring apps and dramatically reduce your development time.   We previously introduced Asset Advisor on Episode 04 of “ThingWorx on Air.” Today, we dive deeper into Asset Advisor with Greg Huet, Asset Advisor’s technical product manager (aka product owner). Listen to Ep. 06: Rapidly Build IIoT Apps for Service & Monitoring with Asset Advisor to hear Greg share our strategy of studying existing use cases and finding similarities that we can pre-build into solutions so that you don’t have to build them from scratch. Hear how you can use Asset Advisor out-of-the-box with tweaks for your company’s configurations or as an accelerated starting point where you can add as much customization as your use case desires—it’s like building a custom pizza, but starting with pre-made dough, rather than yeast and flour.   Greg also mentions the ThingWorx Application Development Guide. Be sure to check out my previous post, where Ward, one of the document creators, shares four of his top tips from the guide.   Now, sit back, relax and go enjoy some pizza while you listen to Episode 06.   As always, stay connected! Kaya

Thingworx Analytics is offered through the User interface called Analytics Builder with some pre-configured functionality. However, should you want to create your own jobs and mashups, all features from Analytics Builder and some more are available through the Thingworx Services.  Running most functionality requires that you provide some data to run the Analytics Services. This is where the datasetRef parameter is required.        Data uploaded through Analytics Builder Any dataset uploaded through builder will require have a datasetUri as shown in the image above and format will be parquet (all small letters) datasetUri can be obtained from the list of datasets in builder Passing data as an in-body Dataset If data isn't uploaded through Analytics Builder, data can be supplied as an Infotable in the data parameter of the datasetRef. Metadata will also need to be supplied if a new dataset is being created (create Job of the AnalyticsServer_DataThing) If this data is being supplied for a scoring job, as long as the column names match up to what the model is expecting, TWX Analytics will inference them appropriately. The filter parameter is for parquet datasets already uploaded into TWXA and will take an ANSI SQL statement format to add conditions to reduce number of rows. Exclusions is an single column infotable list of the columns you wish to remove from the job you are trying to submit Example: If you want Profiles to only run on 5 out of 10 columns, you would give a list of 5 columns that you don't want to include in this exclusions infotable. Data may also be supplied as a csv file in the file repo in some cases, in which case you would give the dataseturi parameter the location of the file on the TWX File repo (of the format thingworx://UseCaseFileRepo/tempdata.csv) and the format which would be csv

Developing Great IoT Solutions Brought to you once again by your EDC team, find attached here a brand-new, comprehensive overview of ThingWorx best practices! This guide was crafted by combining all available feedback, from support cases to PTC Community threads, and tapping all internal resources. Let this guide serve to bridge the knowledge gaps ThingWorx developers most commonly see.    The Developing Great IoT Solutions (DGIS) Guide is a great way to inform both business and technically minded folks about the capabilities of the ThingWorx Platform. Learn how to design good solutions from a high-level, an overview designed specifically with the business audience in mind. Or, learn how to implement good IoT designs through a series of technical examples. Start from very little knowledge of the Platform and end up understanding data structures and aggregation, how to use the collection widget, and how to build a fully functional rules engine for sending and acknowledging alerts in ThingWorx.   For the more advanced among us, check out the Appendix. Find here a handy list of do's and don'ts surrounding ThingWorx best practice in development, with links to KCS, Help Center, and Community content.   Reinforce your understanding of the capabilities of the ThingWorx Platform with this guide, today!   A big thanks to all who were involved on this project! Happy developing!

OpenDJ is a directory server which is also the base for WindchillDS. It can be used for centralized user management and ThingWorx can be configured to login with users from this Directory Service.   Before we start Pre-requisiste Docker on Ubuntu JKS keystore with a valid certificate JKS keystore is stored in /docker/certificates - on the machine that runs the Docker environments Certificate is generated with a Subject Alternative Name (SAN) extension for hostname, fully qualified hostname and IP address of the opendj (Docker) server Change the blue phrases to the correct passwords, machine names, etc. when following the instructions If possible, use a more secure password than "Password123456"... the one I use is really bad   Related Links https://hub.docker.com/r/openidentityplatform/opendj/ https://backstage.forgerock.com/docs/opendj/2.6/admin-guide/#chap-change-certs https://backstage.forgerock.com/knowledge/kb/article/a43576900   Configuration Generate the PKCS12 certificate Assume this is our working directory on the Docker machine (with the JKS certificate in it)   cd /docker/certificates   Create .pin file containing the keystore password   echo "Password123456" > keystore.pin   Convert existing JKS keystore into a new PKCS12 keystore   keytool -importkeystore -srcalias muc-twx-docker -destalias server-cert -srckeystore muc-twx-docker.jks -srcstoretype JKS -srcstorepass `cat keystore.pin` -destkeystore keystore -deststoretype PKCS12 -deststorepass `cat keystore.pin` -destkeypass `cat keystore.pin`   Export keystore and Import into truststore   keytool -export -alias server-cert -keystore keystore -storepass `cat keystore.pin` -file server-cert.crt keytool -import -alias server-cert -keystore truststore -storepass `cat keystore.pin` -file server-cert.crt     Docker Image & Container Download and run   sudo docker pull openidentityplatform/opendj sudo docker run -d --name opendj --restart=always -p 389:1389 -p 636:1636 -p 4444:4444 -e BASE_DN=o=opendj -e ROOT_USER_DN=cn=Manager -e ROOT_PASSWORD=Password123456 -e SECRET_VOLUME=/var/secrets/opendj -v /docker/certificates:/var/secrets/opendj:ro openidentityplatform/opendj   After building the container, it MUST be restarted immediately in order for recognizing the new certificates   sudo docker restart opendj   Verify that the certificate is the correct one, execute on the machine that runs the Docker environments: openssl s_client -connect localhost:636 -showcerts   Load the .ldif Use e.g. JXplorer and connect   Select the opendj node LDIF > Import File (my demo breakingbad.ldif is attached to this post) Skip any warnings and messages and continue to import the file   ThingWorx Tomcat If ThingWorx runs in Docker as well, a pre-defined keystore could be copied during image creation. Otherwise connect to the container via commandline: sudo docker exec -it <ThingworxImageName> /bin/sh Tomcat configuration cd /usr/local/openjdk-8/jre/lib/security openssl s_client -connect 10.164.132.9:636 -showcerts Copy the certifcate between BEGIN CERTIFACTE and END CERTIFICATE of above output into opendj.pem, e.g. echo "<cert_goes_here>" > opendj.pem Import the certificate keytool -keystore cacerts -import -alias opendj -file opendj.pem -storepass changeit   ThingWorx Composer As the IP address is used (the hostname is not mapped in Docker container) the certificate must have a SAN containing the IP address     Only works with the TWLDAPExample Directory Service not the ADDS1, because ADDS1 uses hard coded Active Directory queries and structures and therefore does not work with OpenDJ. User ID (cn) must be pre-created in ThingWorx, so the user can login. There is no automatic user creation by the Directory Service. Make sure the Thing is Enabled under General Information   Appendix LDAP Structure for breakingbad.ldif cn=Manager / Password123456 All users with password Password123456    

  Hi everyone,   In case you’re looking for more reasons to appreciate the power of Azure, today we’re answering 10 frequently asked questions around how and why to use Azure SQL with ThingWorx.   You likely already know that we support multiple persistence providers, like Azure SQL, InfluxDB, H2, MSSQL and PostgreSQL, for you to store and persist your ThingWorx data. Here’s an up-close-and-personal look into why we recommend Azure SQL.   1. What is Azure SQL?         Azure SQL is a relational database hosted in the Azure cloud and is a fully managed Platform as a Service (PaaS) Database Engine. Azure SQL Database engine is based on the Enterprise Edition of SQL Server. The Azure platform fully manages every Azure SQL Database with a high percentage of data availability and guarantees no data loss. Azure SQL Database comes with built-in high availability, disaster recovery, and upgrade for the database. Refer to Microsoft's Azure SQL Database - Platform as a Service documentation for more information on Azure SQL Database and its features.   2. What are the top 3 reasons to use Azure SQL with ThingWorx?   Ease of Use and Management: Azure SQL greatly reduces the need to manage database resources for ThingWorx. It helps to reduce your total cost of ownership for managing database resources for ThingWorx by managing virtual machines, operating system, database software, upgrades, high availability, and backups for you, so you can focus on building your IoT solution. It provides unmatched scale and high availability for compute and storage without sacrificing performance. With Azure SQL, you can scale your application on demand with up to 99.95% availability.   Hybrid Deployments: ThingWorx supports multiple persistence providers to store IoT data for different use cases. Please refer to the ThingWorx Model and Data Best Practices Guide to learn more. If you’re already using Microsoft SQL Server with ThingWorx on premise, then you can use Azure SQL for your cloud deployments of ThingWorx-based IIoT solutions in hybrid scenarios. This allows you to reduce development time—develop once and deploy anywhere through a common programming surface area across Azure SQL (on cloud) and SQL Server (on premise). You can leverage ThingWorx federation to run ThingWorx in different deployment topologies.   If needed, you can also accelerate your on-premise SQL Server migrations without changing the application code by leveraging Managed Instance. Use the Azure Hybrid Benefit Savings Calculator to calculate your TCO. Enjoy additional deployment flexibility with Single Database for SQL applications created in the cloud or Elastic Pool for multi-tenant applications.   Security and Compliance: Azure SQL Database meets the most stringent compliance standards with built-in auditing and information protection technology. With its availability in different regions, its best suited for Government cloud and sovereign cloud. Please see this link to check for the latest update on Azure product availability by region. You can also get multi-layered security provided by Microsoft across physical datacenters, infrastructure, and operations and will always have the latest SQL Server capabilities in the cloud, with no patching or upgrading. It also offers protection to your databases from malicious acts with fine-grained access controls, Always Encrypted technology, and advanced threat protection capabilities.   3. How do I configure ThingWorx for Azure SQL? From ThingWorx Foundation platform version 8.4 release onwards, ThingWorx provides you an option to choose Azure SQL as a persistence provider to store your value stream, stream, and data table data. This Help Center provides all the details and steps to help you set up Azure SQL with ThingWorx.    You can run ThingWorx with Azure SQL either by downloading the ThingWorx Azure SQL .WAR file or by running it as containerized ThingWorx Docker images by downloading ThingWorx Dockerfiles. For reference, see the below image to help you download ThingWorx 8.4 artifacts.   Here’s a video demonstrating how to install ThingWorx. (view in My Videos)   Here’s a second video that walks you through configuring ThingWorx with Azure SQL. (view in My Videos)   4. Which versions of Azure SQL does ThingWorx support? Consult the latest system requirements guide here to learn which versions of Azure SQL ThingWorx supports.   5. What database deployment options do I have? In Azure, you can have your SQL Server workloads running in a hosted infrastructure (IaaS) or running as a hosted service (PaaS). Within PaaS, you have multiple deployment options and service tiers within each deployment option, such as Single Database, Elastic Pool sets, and managed Instance. ThingWorx supports all the deployment options to setup Azure SQL as a persistence provider. You can refer to this link on Azure SQL Database versus SQL Server to help you choose an option that works best for your business needs.   6. Why would I want to use an PaaS database? Service tools and built-in features enable a more streamlined and automated means of controlling and operating your database. The need for constant manual control and tweaking of information, recovery tools, compliance and updates is now configured and built into Azure SQL for a more hands-off approach to your storage database. Here is a table to inform you on how Azure SQL PaaS helps.   7. Which features are new to Azure SQL 2019? Azure SQL now offers Always Encrypted data transfer through TLS and auto-failover for managed instance deployment to enable transparent and coordinated failover of multiple databases. Azure SQL also offers a data migration assistant, which detects compatibility issues that can impact functionality when upgrading your database. For more information on features and functionality, see Microsoft SQL documentation or Azure SQL’s latest release notes.   8. Is there any guidance available to help me migrate to Azure SQL? Yes! Microsoft’s Database Migration Service enables seamless migration to Managed Instance with downtime measured in minutes. The process is highly automated and risk-free while streamlining the transition of SQL Server and on-Microsoft database systems such as Oracle to Azure SQL Database. You can learn more about upgrading to Azure SQL here.   9. What purchasing models are available to me?   vCore based (recommended) - For customers that prioritize flexibility and control, this model offers scaling of compute, storage, and I/O resources independently to optimize price based on need. The customer chooses the hardware and service tier based on high-availability design, storage type, fault-isolation methods, and I/O ranges.   DTU based - Three distinct available tiers are differentiated based directly upon compute, memory, and I/O resources. This model bundles the measures together for customers who want pre-configured or simplified resource options. You can refer to more pricing and purchase options here.   10. What should I do if I need technical support? If you select Azure SQL as your persistence provider, then all support requests related to configuring Azure SQL can be logged through PTC Technical Support at https://support.ptc.com or by calling 1-800-477-6435.   You may also want to use the PTC Community to learn and collaborate with the growing PTC developer community. For all other requests related to database management, troubleshooting, monitoring, and administration, we encourage you to reach out to Microsoft directly.   Let me know what you think in the comments below.   Stay connected, Kaya

Setting Up the Azure Load Balancer with a ThingWorx High Availability Deployment Purpose In this post, one of PTC’s most experienced ThingWorx deployment architects, Desheng Xu, explains the steps to configure Azure Load Balancer with ThingWorx when deployed in a High Availability architectural model.   This approach has been used successfully on customer implementations for several ThingWorx 7.x and 8.x versions. However, with some of the improvements planned for ThingWorx High Availability architecture in the next major release, this best practice will likely change (so keep an eye out for updates to come).   Azure Load Balancer The overview article What is Azure Load Balancer? from Microsoft will give you a high-level understanding of load balancers in general, as well as the capabilities and limitations of Azure Load Balancer itself. For our purposes, we will leverage Azure Load Balancer's capability to manage incoming internet traffic to ThingWorx Platform virtual machine (VM) instances. This configuration is known as a Public Load Balancer.   Important Note: Different load balancers operate at different “layers” of the OSI Model. Azure Load Balancer operates at Layer 4 (Transport Layer) – it is indifferent to the specific TCP Payload. As a result, you must either configure both the front-end and back-end to work on SSL, or configure both of them to work on non-SSL communications. “SSL Termination” or “TLS Offload” is not supported by Azure Load Balancer.   Azure offers multiple different load balancing solutions. If you need some guidance on choosing the right one for you, I highly recommending reviewing the Microsoft DevBlog post Azure Load Balancing Solutions: A guide to help you choose the correct option.   High-Level Diagram: ThingWorx High Availability with Azure Load Balancer To keep this article focused, we will not go into the setup of ThingWorx in a High Availability architecture. It will be assumed that ThingWorx is working correctly and the ZooKeeper cluster is managing failover for the Platform instances as expected. For more details on setting up this configuration, the best place to start would be the High Availability Administrator’s Guide.   Planning In this installation, let's assume we have following plan (you will likely need to change these values for your own implementation): Azure Load Balancer will have a public facing domain name: edc.ptc.io Azure Load Balancer will have a public IP: 41.35.22.33 ThingWorx Platform VM instance 1 has a local computer name, like: vm1 ThingWorx Platform VM instance 2 has a local computer name, like: vm2   ThingWorx Preparation By default, the ThingWorx Platform provides a healthcheck end point at /Thingworx/Admin/HA/LeaderCheck , which can only be accessed with a credential configured in platform-settings.json : "HASettings": { "LoadBalancerBase64EncodedCredentials":"QWRtaW5pc3RyYXRvcjphZG1pbg==" } However, Azure Load Balancer does not permit this Health Check with a credential with current versions of ThingWorx. As a workaround, you can create a pings.jsp (using the attached JSP example code) in the Tomcat folder $CATALINA_HOME/webapps/docs . This workaround will no longer be needed in ThingWorx 8.5 and newer releases.   There are two lines that likely need to be modified to meet your situation: The hostname in final String probeURL (line 10) must match your end point domain name. It's edc.ptc.io in our example, don’t forget to replace this with your real hostname! You also need to add a line in your local hosts file and point this domain name to 127.0.0.1 . For example: 127.0.0.1 edc.ptc.io The credential in final String base64EncodedCredential (line 14) must match the credential configured in platform-settings.json. Additionally: Don't forget to make the JSP file accessible and executable by the user who starts Tomcat service for ThingWorx. These changes must be applied to both ThingWorx Platform VM instances. Tomcat needs to be configured to support SSL on a specific port. In this example, SSL will be enabled on port 8443. Please make sure similar configuration is included in $CATALINA_HOME/conf/server.xml <Connector protocol="org.apache.coyote.http11.Http11NioProtocol" port="8443" maxThreads="200" scheme="https" secure="true" SSLEnabled="true" keystoreFile="/opt/yourcertificate.pfx" keystorePass="dontguess" clientAuth="false" sslProtocol="TLS" keystoreType="PKCS12"/> The values in keystoreFile and keyStorePass will need to be changed for your implementation. While pkcs12 format is used in above example, you can use a different certificate formats, as long as it is supported by Tomcat (example: jks format). All other parameters, like maxThreads , are just examples - you should adjust them to meet your requirements.   How to Verify Before configuring the load balancer, verify that health check workaround is working as expected on both ThingWorx Platform instances. You can use following command to verify: curl -I https://edc.ptc.io:8443/docs/pings.jsp The expected result from active node should look like: HTTP/1.1 200 There will be three or more lines in output, depending on your instance configuration but you should be able to see the keyword: HTTP/1.1 200.   Expected result from passive node should look like: HTTP/1.1 503   Load Balancer Configuration Step 1: Select SKU Search for “load balancer” in the Azure market and select Load Balancer from Microsoft Verify the correct vendor before you create a Load Balancer. Step 2: Create load balancer To create a proper load balancer, make sure to read Microsoft’s What is Azure Load Balancer? overview to understand the differences between “basic” and “standard” SKU offerings. If your IT policy only requires SSL communication to the outside but doesn't require a SSL communication in a health probe, then the “basic” SKU should be adequate (not considering zone redundancy). You have to decide following parameters: Region Type (public or Internal) SKU (basic or standard) IP address Public IP address name Availability zone PTC cannot provide specific recommendations for these parameters – you will need to choose them based on your specific business needs, or consult Microsoft for available offerings in your region.   Step 3: Start to configure Once a load balancer is successfully created by Azure, You should be able to see:   Step 4: Confirm frontend IP Click frontend IP configuration at left side and you should be able to see public IP address configuration. Please make sure to register this IP with your domain name ( edc.ptc.io in our example) in your Domain Name Server (DNS). If you unfamiliar with DNS configuration, you should consult with the administrator of your DNS server. If you are using Azure DNS, this Quickstart article on creating Azure DNS Zones and records may help.   Step 5: Configure Backend pools Click Backend pools and click “Add” to add a backend pool definition. Select a name for your Backend pool (using ThingworxBackend in our example). Next step is to choose Virtual network.   Once you select Virtual network, then you can choose which VM (or VMs) you want to put behind this load balancer. The VM should be the ThingWorx VM instance.   In a high availability architecture, you will typically need to choose two instances to put behind this load balancer.   Please Note: The “Virtual machine status” column in this table only shows VM status, but not ThingWorx status. ThingWorx running status will be determined by the health probe configured in the next step.     Step 6: Configure Health Probe Health Probe will be used to determine the ThingWorx Platform’s running status. When a ThingWorx Platform instance is running as the leader, then it will give HTTP status code 200 during a health probe. The Azure Load Balancer will rely on this status code to determine if the platform is running properly.   When a ThingWorx platform VM is not responding, offline, or not the leader in a High Availability setup, then this health probe will provide response with a different HTTP status code other than 200.   For the health probe, select HTTPS for the protocol. In our example port 8443 is used, though another port can be selected if necessary. Then, provide the “/docs/pings.jsp” we created earlier as the probe’s path. You may need to change this path value if you put this file in a different location.   Step 7: Configure Load balancing rules. Select “Load balancing rules” from left side and click “Add”   Select TCP as protocol, in our example we are using 443 as front-end port and 8443 as back-end port. You can choose other port numbers if necessary.   Reminder: Azure Load Balancer is a layer 4 (Transport Layer) router – it cannot differentiate between HTTP or HTTPS requests. It will simply forward requests from front-end to back-end, based on port-forwarding rules defined.   Session persistence is not critical for current versions of ThingWorx as only one active node is currently permitted in a High Availability architecture. In the future, selecting Client IP may be required to support active-active architectures.   Step 8: Verify health probe Once you complete this configuration, you can go to the $CATALINA_HOME/logs folder and monitor latest local_access log. You should see similar entries as pictured below - HTTP 200 responses should be observed from the ThingWorx leader node, and HTTP 503 responses should be observed from the ThingWorx passive node. In the example below, 168.63.129.16 is the internal IP Address of the load balancer in the current region.   Step 9: Network Security Group rules to access Azure Load Balancer On its own, Azure Load Balancer does not have a network access policy – it simply forwards all requests to the back-end pool. Therefore, the appropriate Network Security Group associated with the backend resources within the resource group should have a policy to allow access to the destination port of the backend ThingWorx Foundation server (shown as 8443 here, for example). The following image displays an inbound security rule that will accept traffic from any source, and direct it to port 443 of the IP Address for the Azure Load Balancer.   Enjoy!! With the above settings, you should be able to access ThingWorx via: https://edc.ptc.io/Thingworx (replacing edc.ptc.io with the hostname you have selected).   Q&A Can I configure the health probe running on a port other than the traffic port (8443) in this case? Yes – if desired you can use a different port for the health probe configuration.   Can I use different protocol other than HTTPS for health probe? Yes – you can use different protocol in the health probe configuration, but you will need to develop your own functional equivalent to the pings.jsp example in this article for the protocol you choose.   Can I configure ZooKeeper to support the health probe? No – the purpose of the health probe is to inform the Load Balancer which node is providing service (the leader), not to select a leader. In a High Availability architecture, ZooKeeper determines which VM is the leader and talking with the database. This approach will change in future releases where multiple ThingWorx instances are actively processing requests.   How well does Azure Load Balancer scale? This question is best answered by Microsoft – as a starting point, we recommend reading the DevBlog post: Azure Load Balancing Solutions: A guide to help you choose the correct option.   How do I access logs for Azure Load Balancer? This question is best answered by Microsoft – as a starting point, we recommend reviewing the Microsoft article Azure Monitor logs for public Basic Load Balancer.   Do I need to configure specifically for Websocket and/or AlwaysOn communication? No – Azure Load Balancer is a Layer 4 (Transport Protocol) router - it only handles TCP traffic forwarding.   Can I leverage this load balancer to access all VMs behind it via ssh? Yes – you could configure Inbound NAT rules for this. If you require specific help in configuring this, the question is best answered by Microsoft. As a starting point, we recommend reviewing the Microsoft tutorial Configure port forwarding in Azure Load Balancer using the portal.   Can I view current health probe status on a portal? No – Unfortunately there is no current approach to do this with Azure Load Balancer.

In the summer heat, keep your operators cool with Operator Advisor. Sit by the pool and relax to the tunes of Episode 05 of “ThingWorx on Air.”     High five! We’re back with Episode 05 of “ThingWorx on Air,” our developer-focused IoT podcast.   In today’s episode Jordan Chaisson, a super talented product manager, joins me to share even more about Operator Advisor (OA). You may remember that we introduced OA in our very first episode of “ThingWorx on Air”. Today, we dive deeper into its business value and reveal what’s on its roadmap. Plus, hear the coolest use case she’s seen yet with Operator Advisor!   OA is an accelerator application built on the ThingWorx platform that enables manufacturing operators through digital work instructions and a comprehensive user experience to receive the right data at the right time to minimize scrap and maximize efficiency.   Looking for more? Check out the Operator Advisor Guide or discover where to download Operator Advisor today.   Reach out with any questions, and, as always, stay connected!   -  Kaya

  Energy. Innovation. Passion. That’s how I would describe LiveWorx19 in three words.   From beginning to end, LiveWorx truly was a one-of-a-kind digital transformation event. Whether it was the top-notch demos, the delicious lunch from food trucks, or the unique integration of our products with our partners’ technologies, everywhere I turned, inspiration and excitement were booming.   Here are my top three highlights:   1. Delivering a breakout session Chris Baldwin and I shared exciting new functionality that we’re releasing in ThingWorx to simplify and accelerate deployment of applications across your enterprise. We introduced Solution Central, a new portal in the cloud for application management and deployment (more info to come in a future post, but in the meantime, reach out with any questions).   2. Assisting with Xtropolis, our Demo Floor Speaking directly with you and hearing how you’re using ThingWorx to transform industries like medical devices, beer brewing or furniture manufacturing was pretty impressive. It was also quite rewarding and inspiring to hear your clever questions about your unique IoT use cases while being surrounded by folks performing service on a giant John Deere tractor, attendees having virtual rides on a Polaris vehicle and a team of brewers using ThingWorx and Vuforia to brew some tasty Trillium beer.   3. Participating in Usability Tests Thank you to everyone who participated in the UX Lab. Hearing your direct feedback in usability tests on what you like and what you want to see changed or added to the product helps make it feel like you are right there with us designing ThingWorx—because, well, you are. We take the feedback you share at LiveWorx right back to the office and begin iterating. So, quick shoutout to everyone who participated—thanks for your feedback.     If you weren’t able to attend LiveWorx, check out our CEO, Jim Heppelmann, deliver the opening keynote above and hear him discuss how PTC is interweaving AR, IoT, generative design, PLM and AI to create a digital thread that transforms industrial enterprises across the globe.     After hearing about our overall strategy, you’re looking for more details on our product roadmap, watch the above roadmap session delivered by our EVP of Products, Kathleen Mitford.   One of the most rewarding parts of the show for me was meeting some of you—ThingWorx developers and Ask Kaya readers—in person!   Stay connected, Kaya   P.S. If you want to hear what others thought of LiveWorx, check out what Forbes, Automation World or Diginomica had to say.   P.P.S. If you had as much fun as I did or you’re looking to attend for the first time, tickets for LiveWorx 2020 are already available—check them out here!

  We are counting down the days for you—developers, technologists, futurists—to witness the unparalleled power of PTC’s technology. Hosted by PTC, LiveWorx is the world’s leading digital transformation event to equip you with the knowledge, power and tools you need to begin or accelerate your company’s digital transformation.   I’m excited to share that I will be presenting a breakout session on June 11 th , at 1:15pm EST, around a brand-new functionality we’re offering to improve your ability to manage and deploy your ThingWorx applications.   Want to learn more? Attend LiveWorx 2019 or learn about our livestream options. However you choose to attend, it’s an event that I’m pretty amped for and I can’t wait for you to be, too.   Hope to see you there—it’d be great to meet you in person!   Stay connected, Kaya  

Reminder (and for some, announcement!) that the new ThingWorx 8 sizing guide is available here  https://www.ptc.com/en/support/refdoc/ThingWorx_Platform/8.0/ThingWorx_Platform_8_x_Sizing_Guide

Since the marketplace extension is no longer supported and the drivers may be outdated, you may build your own jdbc package/extension: Download the Extension Metadata file Here Download the appropriate JDBC driver Build the extension structure by creating the directory lib/common Place the JAR file in this directory location: lib/common/<JDBC driver jar file> Modify the name attribute of the ExtensionPackage entity in the metadata.xml file as needed Point the file attribute of the FileResource entity to the name of the JDBC JAR file The metadata also contains a ThingTemplate the name is set to MySqlServer, but can be modified as needed Select the lib folder and metadata.xml file and send to a zip archive Tip: The name of the zip archive should match the name given in the name attribute of the ExtensionPackage entity in the metadata.xml file Import the newly created extension as usual To the JDBC extension, simply create a new thing and assign it the new ThingTemplate that was imported with the JDBC extension Configuration Field Explanation: JDBC Driver Class Name ​Depends on the driver being used Refer to documentation JDBC Connection String ​Defines the information needed to establish a connection with the database Connection string examples can be found in the ThingWorx Help Center ConnectionValidationString ​A simple query that will work regardless of table names to be executed to verify return values from the database   Alternatively, you may download the jdbc connector creator from the marketplace here https://marketplace.thingworx.com/Items/jdbc-connector-extension Then you may just view the mashup and use it to package your jdbc jar into an extension (which can be later imported into ThingWorx).  

  The scenario: Your company has settled on Azure as a cloud platform and you are currently using Azure IoT Edge as your connectivity strategy. You need a quick way to build IoT applications with your Azure devices. You’re looking for industry-proven and time-saving features like Mashup visualization, built-in connectivity to enterprise systems (like SAP or Oracle) with ThingWorx Flow, secure and scalable file transfer to your Azure-connected devices and the ability to create augmented reality (AR) experiences with Vuforia Studio. All of these options are available to you thanks to the ThingWorx-Azure IoT Hub Connector; it’s like the ice cream truck driving by on a hot summer day.   (If you’re wondering why we selected Azure as our preferred infrastructure, check out my previous interview with Neal, a Worldwide ThingWorx Center of Excellence Principal Lead here at PTC.)   I sat down with Ankit, a ThingWorx Product Manager, this week to learn more about the ThingWorx-Azure IoT Hub Connector. When Ankit’s not learning new hobbies like how to surf, snowboard or bike, he’s supporting our Microsoft partnership by enhancing and implementing ThingWorx-Azure functionality. Here’s how our conversation went:   Kaya: What is the Azure IoT Hub? Ankit: The Azure IoT Hub acts as a central message router for bi-directional communication between the cloud (and your ThingWorx applications) and your connected devices. The Azure IoT Hub securely connects, monitors and manages billions of devices. It is an open and flexible cloud platform as a service that supports open-source SDKs and multiple protocols. With ThingWorx, we enable you to authenticate user access per device to ensure your IoT solutions remain secure.   Kaya: I understand your team has created the ThingWorx-Azure IoT Hub Connector. Can you explain what it is and what it does? Ankit: The Azure IoT Hub Connector is an extension that is imported into ThingWorx for a developer to connect the Azure IoT Hub to ThingWorx. This helps ThingWorx to leverage the security and scalability of Azure while retaining the ThingWorx domain expertise to provide fast time to value.   The Connector is built on the ThingWorx Connection Server core. What it essentially does is convert JSON objects from Azure IoT Hub into ThingWorx property types (and vice versa) so that the digital twin data of an Azure device can be native to ThingWorx.   Since the Connector is built on the ThingWorx Connection Server, it is horizontally scalable and leverages features such as health check, metrics (message count and size, property writes) and logging.   Kaya: What was the challenge developers were facing that led us to create the Azure IoT Hub Connector? Ankit: There was no easy way for a developer to use ThingWorx to represent an Azure IoT device. Users weren’t easily able to take advantage of ThingWorx services and functionality on their Azure IoT devices, which were inherently connected to the Azure IoT Hub. Similarly, ThingWorx users were not able to take advantage of Azure services in a “configure-not-code” fashion in ThingWorx.   Kaya: How does the Connector solve this problem to enable you to integrate the two platforms and device models for a better combined solution? Ankit: Once you have an Azure device represented as a “Thing” in ThingWorx, you can use all the features and capabilities of ThingWorx Composer, Mashup Builder, etc. to build applications using the data from that Azure device.   Kaya: That’s pretty great. Ankit: Thanks, agreed. In the next version of the Connector, we’ll integrate more closely with Azure, such that our developers can leverage Azure services as well via ThingWorx, instead of building those services from scratch on Azure all on their own. For example, developers will be able to send software content, like firmware updates, to an Azure device without writing any code on Azure. All of this can be done on ThingWorx using Azure components like Azure IoT Edge Runtime.   Kaya: Awesome. In the meantime, what are the top two or three things a developer can do with the Azure IoT Hub Connector today? Ankit: Today, developers can take advantage of ingress and egress processing as well as file transfer. I’ll explain what these mean. Ingress Processing: Azure IoT devices (i.e. devices that are running Azure SDKs) send messages to the Azure IoT Hub. These messages are typically values of device properties (e.g. temperature). The Azure IoT Hub Connector “listens” for these messages, translates them and passes them to the ThingWorx platform. Egress Processing: Egress messages are messages that arrive from ThingWorx and are pushed to the Azure IoT Hub; an example might be pushing property updates to an Azure IoT device. File Transfer: The Azure IoT Hub Connector supports transferring files between Azure IoT devices and an Azure storage container (i.e. Blob store). An Azure storage container is represented by a ‘FileRepository’ Thing within ThingWorx. This enables developers to transfer files from an Azure storage container to ThingWorx and vice versa.   Kaya: What are two exciting features planned for a future release of the Connector? Ankit: Two exciting features planned for July include software content management (or SCM) and compatibility with ThingWorx Asset Advisor. Software Content Management (SCM): In our next release, we plan to have support for SCM from ThingWorx to an Azure IoT Edge device (an Azure IoT device with IoT Edge Runtime) via Azure IoT Hub. SCM allows users to transfer a variety of content like configuration settings, operating system patches and software updates and/or patches to a software agent on your Azure devices. SCM also allows you to manage your remote assets and keep them patched, secure and up-to-date with the latest features without having to dispatch a technician. This helps to reduce cost and complexity of software distribution and installation. Compatibility with ThingWorx Asset Advisor: Also planned for our next release, you will be able to readily manage Azure IoT devices directly through Asset Advisor to see key device alerts and warnings. This makes it even easier for you to leverage Asset Advisor to rapidly enable remote monitoring of your Azure devices.   Kaya: Exciting stuff. For our readers not familiar with Asset Advisor, check out this episode of my “ThingWorx on Air” podcast to understand what Asset Advisor is and how it works. Okay, next question. Do you have an example of a customer using Azure IoT Hub? Ankit: Absolutely. Colfax, an industrial manufacturing company, is using Azure IoT Hub to improve the efficiency of its IoT efforts across the enterprise. You should check out our case study on Colfax if you haven’t seen it yet.   Kaya: Where should I as a developer go if I want to learn more about the Azure IoT Hub Connector or Azure in general? Ankit: Depending on what you’re looking for, I’d recommend you check out the Help Center for technical guidance or the ThingWorx Azure IoT Hub Connector Release Notes, v. 2.0.0 for release updates.   Kaya: Finally, where can I go to download the ThingWorx Azure IoT Hub? Ankit: You can download it from the PTC Marketplace. Enjoy! Readers, let me know what you think about the Azure IoT Hub Connector in the comments below and reach out with any questions. While we’re excited to deliver what we have planned, our release content may change. In the meantime, for updates, tips and tricks and relevant info, stay connected!

In case it's useful for anyone, I successfully used the Thingworx Importer to import an Entity using version 8.4.1. The import command was in a CURL script (can also be run using e.g. Cygwin on Windows) and the entity data was contained in an XML file. The XML file is attached to this post, and the CURL script is copied into the bottom of the post body.   Notes on using the script: Copy CURL code into import.sh You might need to change the line endings to UNIX, e.g. in Notepad++ menu option Edit > EOL Conversion > Unix Give permissions to run import.sh (chmod +x import.sh) ./import.sh >>>>>>>>>>>> #!/bin/bash APPKEY="2e6704c0-XXXX-XXXX-XXXX-a1589e387d1a" TWX_HTTP_PORT="8018" FILE_PATH_TWX="Things_testImport.xml" PROTOCOL="http://" IP="localhost" URL="$PROTOCOL""$IP"":""$TWX_HTTP_PORT""/Thingworx/Importer?purpose=import" curl -X POST -H 'appKey: '$APPKEY \ -H 'Content-Type: multipart/form-data' \ -H 'Accept: application/json' \ -H 'x-thingworx-session: true' \ -H 'X-XSRF-TOKEN: TWX-XSRF-TOKEN-VALUE' \ -F 'upload=@'$FILE_PATH_TWX \ $URL <<<<<<<<<<<<   Also TWX Importer is explained in this support article.

Please find here an Labview implementation to connect to Thingworx via RestCalls. Have Fun using it. Any Feedback is appreciated. https://github.com/Seppel1985/LabVIEW_TWX_RestAPI

  Helloooooo ThingWorx users,   Ever wanted to see the coolest technology in action? Ever wished you could surround yourself with awesome ThingWorx developers? Maybe you’ve even wished you could meet the ThingWorx product management team!   If that’s the case, you’re in luck! Join me for LiveWorx 2019 from June 10 – 13 in Boston this summer to discover how you can make digital transformation a reality for your organization. See what all the hype’s about here!   I’ll be presenting a rockin’ session on some exciting new functionality coming in ThingWorx to help with enterprise-wide app deployment. See me present with Chris Baldwin on Tues, June 11, @ 1:15pm in our session titled Introducing Solution Central: Your Gateway to Accelerated IIoT Value Across the Enterprise!   For a sneak peek of what’s to come at LiveWorx, here are seven sensational sessions our developers can’t miss! (Note: Dates and times are subject to change.) It's Electric: HowCaterpillar Develops Compelling IIoT Apps That Resonate With Customers & Dealers Mon, June 10, @ 4:30 (45 min) ThingWorx and Microsoft Azure from A to Z Tues, June 11, @ 4:00pm (45 min) It’s All About The Apps: Introducing ThingWorxMashupBuilder 2.0 and More! Wed, June 12, @ 9:00am (45 min) Connecting Asset Advisor to Azure Wed, June 12, @ 3:00pm (45 min) ThingWorx for Scalability with InfluxDB and Beyond! Wed, June 12, @ 3:00pm (45 min) From Pilot to Production: Tips & Tricks for Vuforia Studio Thurs, June 13, @ 12:00pm (45 min)   Hope to see you all there and meet you in person!   Stay connected, Kaya

  Hi everyone,   This week, Anthony Moffa returns to Ask Kaya in a different form from his original appearance explaining the benefits of Thing Presence in ThingWorx. As much as we enjoy reading Anthony in print, you can now hear from the man himself in the “Moffa Monitoring Minute!”   Listen to Episode 04 of “ThingWorx on Air” as he explains what Asset Advisor is and how you can use it to remotely monitor assets, shorten service cycles, and improve visibility of your device fleet.   Want to learn even more about Asset Advisor? Check out this video or read through our website!   Reach out with any questions and just Ask Kaya!   Stay connected, Kaya

  You’ve seen him before. You’ve heard him before. Fans around the globe can’t get enough of him. He’s…   ...Joe Biron—our CTO of IoT!   Hear Joe share his thoughts on the future of the industrial IoT with ThingWorx in Episode 03 of our “ThingWorx on Air” podcast!   Any questions? Just Ask Kaya.   Stay connected!

When predicting a Boolean goal such as Failure in the next hour or any other goal that has a yes or no answer, Thingworx Analytics(TWXA) models will output a 'risk' of the event occurring. TWXA will intelligently pick a threshold beyond which that risk warrants attention. 1. In Analytics Builder, click on the export button 2. This will export a PMML model and download it for you 3. Open up the PMML model, in the output section, you will find a condition that explains the threshold that was selected by TWX Analytics.   In this example case, TWXA chose 0.5 as the best Threshold.   Note: The export button will only be available in Builder for TWXA 8.4+.