IoT & Connectivity Tips

This video gives an introduction to the Descriptive Services: what they are how to install them how to configure them how to use them  

This video shows the steps to install ThingWorx Analytics release 8.3  

In this IoT-AR Tech Tip, we will cover what the microservices do, and their functions overall in the ThingWorx Analytics Server Application.   With the new architecture changes introduced with ThingWorx Analytics 8.1, many users have asked for a more descriptive explanation of the purpose of the microservices that make up the Analytics Server application. In 8.3, Descriptive Microservice has been introduced, and changings in how Predictive Scoring were incorporated. ThingPredictor has been deprecated and its primary functions have been incorporated into ThingWorx Analytics Server’s Prediction microservice.   Many of these microservices can be installed as separate distinct utilities, though there some that are required for base functionality of Analytics Server. This allows custom installation by the end-user to tailor the TWA deployment to their needs.   ThingWorx Analytics Server Microservices   Analytics Microservice – Analytics Server Edge Agent acts as an integration point between the ThingWorx server and the ThingWorx Analytics server. The Edge Agent automatically registers with ThingWorx and instantiates the Things that represent the connected ThingWorx Analytics microservicers. This component is mandatory.   Clustering Microservice - The Clustering Microserver provides services that categorize dataset records into groups (clusters) based on their similarities. Clustering returns results in the form of a PMML model. This component is optional.   Data Microservice - The Data Microserver provides data-handling services. These services include creating a dataset, appending new data, viewing dataset metadata, querying and retrieving information about the distribution of data in the dataset. This component is mandatory.   Predictive Microservice - The Prediction Microserver provides both real time and asynchronous (batch) predictive scoring. The scoring process evaluates each record in a dataset against a prediction model. Each record is assigned a predictive score that reflects the model's predicted outcome for that record. This component is optional. This component replaces ThingPredictor in functionality.   Prescriptive Microservice - The Prescriptive Microserver provides real time prescriptive scoring, which examines how certain changes might affect future outcomes. Fields identified as levers can be varied to determine how specific changes might affect future outcomes. This component is optional.   Profiles Microservice - The Profiling Microserver provides services to identify distinct subpopulations (profiles) within a dataset that share similar characteristics and are different from other subpopulations in statistically significant ways.  Profiles are not required in order to make a prediction, but they contribute to a strategic understanding of the complex factors associated with specific outcomes. This component is optional.   Results Microservice – The Results Microserver provides services for working with results. It also provides services to query input and output fields for Training and Clustering results, which output in PMML format. This component is mandatory.   Training Microservice – The Training Microserver uses machine learning algorithms and techniques to identify meaningful patterns in a dataset and generate a predictive model. A number of parameters are available for tuning the type and combination of learning techniques used. The resulting model is output in PMML format. This component is optional, however, if the Validation Microserver is selected, the Training Microserver will be selected by default.   Validation Microservice – The Validation Microserver provides metrics to evaluate how well a model was able to predict outcomes for a specific goal variable in a dataset. Depending on the goal variable, validation metrics can include RMSE, Matthew's Correlation (MCC), Pearson Correlation, true and false positive rates, accuracy, or a confusion matrix. This component is optional, however if the Training Microserver is selected, the Validation Microserver will be delected by default.   Analytics Worker – The Analytics Worker does the processing work in the system. It picks up jobs from the Zookeeper system and performs the requested work. By default, only one worker is installed, but the number of workers can be scaled up after installation. Workers can process any type of submitted job request. This component is mandatory.   ThingWorx Analytics – Descriptive Analytics Microservice   Descriptive Analytics Microservice is an optional microservice included with the ThingWorx Analytics Server installation package, but can operate as a distinct and separate utility.   Descriptive Analytics provides a library of on-demand services that perform common statistical calculations and facilitate statistical monitoring on raw data. Descriptive Analytics is not required to generate prediction models, but output from these services can provide insights that improve your understanding of your data.   Update January 6, 2020   For information about ThingWorx Analytics 8.5 and newer, please refer to the HelpCenter - Analytics Server System Architecture for 8.5

This video walks through the dataset requirements when working with time series in ThingWorx Analytics Server.  Starting release 52.2 - ending release 8.2.   Also view: - Written version of those steps mentioned in the video - Help Center  

This video go through the steps required to use the Creo Insight extension: - Download and install the required extension - Set required config.pro options - Create provider in Analytics Manager - Publish sensor from Creo - Create analysis Event in Analysis Manager - Retrieve sensor values from ThingWorx in Creo     See also: - https://www.ptc.com/en/support/article?n=CS277514  for a  written version of those steps. - Creo Help Center  

Design and Implement Data Models to Enable Predictive Analytics Learning Path   Design and implement your data model, create logic, and operationalize an analytics model.   NOTE: Complete the following guides in sequential order. The estimated time to complete this learning path is 390 minutes.    Data Model Introduction  Design Your Data Model Part 1 Part 2 Part 3  Data Model Implementation Part 1 Part 2 Part 3  Create Custom Business Logic  Implement Services, Events, and Subscriptions Part 1 Part 2  Build a Predictive Analytics Model  Part 1 Part 2 Operationalize an Analytics Model  Part 1 Part 2  

Getting Started on the ThingWorx Platform Learning Path   Learn hands-on how ThingWorx simplifies the end-to-end process of implementing IoT solutions.   NOTE: Complete the following guides in sequential order. The estimated time to complete this learning path is 210 minutes.   Get Started with ThingWorx for IoT   Part 1 Part 2 Part 3 Part 4 Part 5 Data Model Introduction Configure Permissions Part 1 Part 2 Build a Predictive Analytics Model  Part 1 Part 2

  Operationalize an Analytics Model Guide Part 1   Overview   This project will introduce ThingWorx Analytics Manager. Following the steps in this guide, you will learn how to deploy the model which you created in the earlier Builder guide. We will teach you how to utilize this deployed model to investigate whether or not live data indicates a potential engine failure. NOTE: This guide’s content aligns with ThingWorx 9.3. The estimated time to complete ALL 2 parts of this guide is 60 minutes.    Step 1: Analytics Architecture   You can leverage product-based analysis Models developed using PTC and third-party tools while building solutions on the ThingWorx platform. Use simulation as historical basis for predictive Models Create a virtual sensor from simulation Design-time versus operational-time intelligence It is important to understand how Analytics Manager interacts with the ThingWorx platform.   Build Model   In an IoT implementation, multiple remote Edge devices feed information into the ThingWorx Foundation platform. That information is stored, organized, and operated-upon in accordance with the application's Data Model. Through Foundation, you will upload your dataset to Analytics Builder. Builder will then create an Analytics Model.     Operationalize Model   Analytics Manager tests new data through the use of a Provider, which applies the Model to the data to make a prediction. The Provider generates a predictive result, which is passed back through Manager to ThingWorx Foundation. Once Foundation has the result, you can perform a variety of actions based on the outcome. For instance, you could set up Events/Subscriptions to take immediate and automatic action, as well as alerting stakeholders of an important situation.       Step 2: Simulate Data Source   For any ThingWorx IoT implementation, you must first connect remote devices via one of the supported connectivity options, including Edge MicroServer (EMS), REST, or Kepware Server. Edge Connectivity is outside the scope of this guide, so we'll use a data simulator instead. This simulator will act like an Engine with a Vibration Sensor, as described in Build a Predictive Analytics Model. This data is subdivided into five frequency bands, s1_fb1 through s1_fb5. From this data, we will attempt to predict (through the engine's vibrations) when a low grease emergency condition is occuring.   Import Entities   Import the engine simulator into your Analytics Trial Edition server. Download and unzip the attached amqs_entities.zip file. At the bottom-left of ThingWorx Composer, click Import/Export > Import.     Keep the default options of From File and Entity, click Browse, and select the amqs_entities.twx file you just downloaded.   Click Import, wait for the Import Successful message, and click Close.   From Browse > All, select AMQS_Thing from the list.   At the top, click Properties and Alerts to see the core functionality of the simulator. NOTE: The InfoTable Property is used to store data corresponding to the s1_fb1 through s1_fb5 frequency bands of the vibration sensor on our engine. The values in this Property change every ten seconds through a Subscription to the separate AMQS_Timer Thing. The first set of values are good, in that they do NOT correspond to a low grease condition. The second set of values are bad, in that they DO correspond to a low grease condition. These values will change whenever the ten-second timer fires.   View Mashup   We have created a sample Mashup to make it easier to visualize the data, since analyzing data values in the Thing Properties is cumbersome. Follow these steps to access the Mashup. On the ThingWorx Composer Browse > All tab, click AMQS_Mashup.   At the top, click View Mashup.    Observe the Mashup for at least ten-seconds. You'll see the values in the Grid Advanced Widget change from one set to another at each ten-second interval.     NOTE: These values correspond to data entries from the vibration dataset we utilized in the pre-requisite Analytics Builder guide. Specifically, the good entry is number 20,040... while the bad entry is number 20,600. You can see in the dataset that 20,400 corresponds to a no low grease condition, while 20,600 corresponds to a yes, low grease condition.   Step 3: Configure Provider   In ThingWorx terminology, an Analysis Provider is a mathematical analysis engine. Analytics Manager can use a variety of Providers, such as Excel or Mathcad. In this quickstart, we use the built-in AnalyticsServerConnector, an Analysis Provider that has been specifically created to work seamlessly in Analytics Manager and to use Builder Models. From the ThingWorx Composer Analytics tab, click ANALYTICS MANAGER > Analysis Providers, New....   In the Provider Name field, type Vibration_Provider. In the Connector field, search for and select TW.AnalysisServices.AnalyticsServer.AnalyticsServerConnector.   4. Leave the rest of the options at default and click Save.   Step 4: Publish Analysis Model   Once you have configured an Analysis Provider, you can publish Models from Analytics Builder to Analytics Manager. On the ThingWorx Composer Analytics tab, click ANALYTICS BUILDER > Models.   Select vibration_model_s1_only and click Publish.   On the Publish Model pop-up, click Yes. A new browser tab will open with the Analytics Manager's Analysis Models menu.      4. Close that new browser tab, and instead click Analytics Manager > Analysis Models in the ThingWorx Composer Analytics navigation. This is the same interface as the auto-opened tab which you closed.   False Test   It is recommended to test the published Model with manually-entered sample data prior to enabling it for automatic analysis. For this example, we will use entry 20,400 from the vibration dataset. If the Model is working correctly, then it will return a no low grease condition. In Analysis Models, select the model you just published and click View.   Click Test.   In the causalTechnique field, type FULL_RANGE. In the goalField field, type low_grease. For _s1_fb1 through _s1_fb5, enter the following values: Data Row Name Data Row Value _s1_fb1 161 _s1_fb2 180 _s1_fb3 190 _s1_fb4 176 _s1_fb5 193 6. Click Add Row. 7. Select the newly-added row in the top-right, then click Set Parent Row. 8. Click Submit Job. 9. Select the top entry in the bottom-left Results Data Shape field. 10. Click Refresh Job. Note that _low_grease is false and and _low_grease_mo is well below 0.5 (the threashold for a true prediction).   You have now successfully submitted your first Analytics Manager job and received a result from ThingPredictor. ThingPredictor took the published Model, used the no low grease data as input, and provided a correct analysis of false to our prediction.   True Test Now, let's test a true condition where our engine grease IS LOW, and confirm that Analytics Manager returns a correct prediction. In the top-right, select the false data row we've already entered and click Delete Row. For _s1_fb1 through _s1_fb5, change to the following values: Data Row Name Data Row Value _s1_fb1 182 _s1_fb2 140 _s1_fb3 177 _s1_fb4 154 _s1_fb5 176 3. Select the top entry in the bottom-left Results Data Shape field. 4. Click Refresh Job. Note that _low_grease is true and and _low_grease_mo is above 0.5.                 5. Click Submit Job.         6. Select the top entry in the bottom-left Results Data Shape field.         7. Click Refresh Job. Note that _low_grease is true and and _low_grease_mo is above 0.5.          You've now manually submitted yet another job and received a predictive score. Just like in the dataset Entry 20,600, Analytics Manager predicts that the second s1_fb1 through s1_fb5 vibration frequencies correspond to a low grease condition which needs to be addressed before the engine suffers catastrophic failure.   Enable Model   Since both false and true predictions made by the Model seem to match our dataset, let's now enable the Model so that it can be used for automatic predictions in the future. In the top-left, expand the Actions... drop-down box.   Select Enable.     Click here to view Part 2 of this guide.   

Build a Predictive Analytics Model Guide Part 2   Step 5: Profiles   The Profiles section of ThingWorx Analytics looks for combinations of data which are highly correlated with your desired goal. On the left, click ANALYTICS BUILDER > Profiles. Click New....The New Profile pop-up will open. NOTE: Notice the Text Data Only section which is new in ThingWorx 9.3.         3. In the Profile Name field, enter vibration_profile. 4. In the Dataset field, select vibration_dataset. 5. Leave the Goal field set to the default of low_grease. 6. Leave the Filter field set to the default of all_data. 7. Leave the Excluded Fields from Profile field set to the default of empty. 8. Click Submit. 9. After ~30 seconds, the Signal State will change to COMPLETED. The results will be displayed at the bottom.                 The results show several Profiles (combinations of data) that appear to be statistically significant. Only the first few Profiles, however, have a significant percentage of the total number of records. The later Profiles can largely be ignored. Of those first Profiles, both Frequency Bands from Sensor 1 and Sensor 2 appear. But in combination with the result from Signals (where Sensor 1 was always more important), this could possibly indicate that Sensor 1 is still the most important overall. In other words, since Sensor 1 is statistically significant both by itself and in combination (but Sensor 2 is only significant in combation with Sensor 1), then Sensor 2 may not be necessary.     Step 6: Create Model   Models are primarily used by Analytics Manager (which is beyond the scope of this guide), but they can still be used to measure the accuracy of predictions. When Models are calculated, they inherently withhold a certain amount of data. The prediction model is then run against the withheld data. This provides a form of "accuracy measure", which we'll use to determine whether Sensor 2 is necessary to the detection of a low grease condition by creating two different Models. The first Model (which you will create below) will contain all the data, while the second Model (in the next step) will exclude Sensor 2. On the left, click ANALYTICS BUILDER > Models.   Click New….The New Predictive Model pop-up will open.   3. In the Model Name field, enter vibration_model. 4. In the Dataset field, select vibration_dataset. 5. Leave the Goal field set to the default of low_grease. 6. Leave the Filter field set to the default of all_data.         7. Leave the Excluded Fields from Model section at its default of empty.       8. Click Submit. 9. After ~60 seconds, the Model Status will change to COMPLETED.   View Model   Now that the prediction model is COMPLETED, you can view the results. Select the model that was created in the previous step, i.e. vibration_model. Click View… to open the Model Information page.   Review the visualization of the validation results. Note that your results may differ slightly from the picture, as the automatically-withheld "test" portion of the dataset is randomly chosen. Click on the ? icon to the right of the chart for details on the information displayed.   The desired outcome is for the model to have a relatively high level of accuracy. The True Positive Rate shown on the Receiver Operating Characteristic (ROC) chart are much higher than the False Positives. The curve is relatively high and to the left, which indicates a high accuracy level. You may also click on the Confusion Matrix tab in the top-left, which will show you the number of True Positive and True Negatives in comparison to False Positives and False Negatives.     Note that the number of correct predictions is much higher than the number of incorrect predictions.     As such, we now know that our Sensors have a relatively good chance at predicting an impending failure by detecting low grease conditions before they cause catastrophic engine failure.     Step 7: Refine Model   We will now try comparing this first Model that includes both Sensors to a simpler Model using only Sensor 1. We do this because we suspect that Sensor 2 may not be necessary to achieve our goal. On the left, click ANALYTICS BUILDER > Models.   Click New…. In the Model Name field, enter vibration_model_s1_only. In the Dataset field, select vibration_dataset. Leave the Goal field set to the default of low_grease. Leave the Filter field set to the default of all_data.   On the right beside Excluded Fields from Model, click the Excluded Fields button. The Fields To Be Excluded From Job pop-up will open. 8. Click s2_fb1 to select the first Sensor 2 Frequency Band. 9. Select the rest of the Frequency Bands through s2_fb5 to choose all of the Sensor 2 frequencies. 10. While all the s2 values are selected, click the green "right arrow", i.e. the > button in the middle. 11. At the bottom-left, click Save. The Fields To Be Excluded From Job pop-up will close.           12. Click Submit. 13. After ~60 seconds, the Model State will change to COMPLETED. 14. With vibration_model_s1_only selected, click View....   The ROC chart is comparable to the original model (including Sensor 2). Likewise, the Confusion Matrix (on the other tab) indicates a good ratio of correct predictions versus incorrect predictions.     NOTE: These Models may vary slightly from your own final scores, as what data is used for the prediction versus for evaluation is random. ThingWorx Analytics's Models have indicated that you are likely to receive roughly the same accuracy of predicting a low-grease condition whether you use one sensor or two! If we can get an accurate early-warning of the low grease condition with just one sensor, it then becomes a business decision as to whether the extra cost of Sensor 2 is necessary.   Step 8: Next Steps   Congratulations! You've successfully completed the Build a Predictive Analytics Model guide, and learned how to:   Load an IoT dataset Generate machine learning predictions Evaluate the analytics output to gain insight    This is the last guide in the Getting Started on the ThingWorx Platform learning path.   This is the last guide in the Monitor Factory Supplies and Consumables learning path.   The next guide in the Design and Implement Data Models to Enable Predictive Analytics learning path is Operationalize an Analytics Model.     Additional Resources   If you have questions, issues, or need additional information, refer to:   Resource Link Support Analytics Builder Help Center    

Build a Predictive Analytics Model Guide Part 1   Overview   This project will introduce ThingWorx Analytics Builder. Following the steps in this guide, you will create an analytical model, and then refine it based on further information from the Analytics platform. We will teach you how to determine whether or not a model is accurate and how you can optimize both your data inputs and the model itself.   NOTE: This guide's content aligns with ThingWorx 9.3. The estimated time to complete ALL 2 parts of this guide is 60 minutes.      Step 1: Scenario   MotorCo manufactures, sells, and services commercial motors. Recently, MotorCo has been developing a new motor, and they already have a working prototype. However, they've noticed that the motor has a chance to FAIL CATASTROPHICALLY if it's not properly serviced to replace lost grease on a key moving part. In order to prevent this type of failure in the field, MotorCo has decided to instrument their motors with sensors which record vibration. The hope is that these sensors can detect certain vibrations which indicate required maintenance before a failure occurs. MotorCo has decided to utilize ThingWorx Analytics to scan their prototype data for any insights they can gain to address this issue. Challenges: There is a known failure mode of a prototype, but it is not currently possible to predict when the failure might happen Until this failure mode is mitigated, the prototype cannot move into full production While connected data is being monitored, what to do with that data is not currently known The additional sensors are adding to the overall cost of the product   Step 2: Settings   If you manually installed the Analytics extension, then you need to complete the steps below to finalize the connection.  On the ThingWorx Composer Analytics tab, click ANALYTICS BUILDER > Settings. In the Analytics Server field, search for and select your Server Thing Entity, such as AnalyticsServer_Thing or localhost-AnalyticsServer. Click Verify Configuration.   Click Save. WARNING: You MUST CLICK SAVE or the configurations will be lost when you move to Data in the next step.   Step 3: Upload Data   We will now load the data that ThingWorx Analytics will use to generate a model. Download the attached analytics_vibration.zip to your computer. Unzip the analytics_vibration.zip file to access the vibration_data_and_header.csv and vibration_metadata.json files. On the left, click ANALYTICS BUILDER > DATA.   Under Datasets, click New.... The New Dataset pop-up will open.         5. In the Dataset Name field, enter vibration_dataset. 6. In the File Containing Dataset Data section, search for and select vibration_data_and_header.csv. 7. In the File Containing Dataset Field Configuration section, search for and select vibration_metadata.json.   8. Click Submit. Note that it will take a variable amount of time for the data-upload to complete, based on the size of your dataset.              Step 4: Signals   The Signals section of ThingWorx Analytics looks for the most statistically correlated single field in the dataset which relates to your selected goal. This doesn't necessarily indicate that it is the cause of your goal, whether maximizing or minimizing. It just means that the dataset indicates that this single field happens to correlate with the goal that you desire. On the left, click ANALYTICS BUILDER > Signals.   At the top, click New…. A New Signals pop up will open.   3. In the Signal Name field, enter vibration_signal. 4. In the Dataset field, select vibration_dataset. 5. Leave the Goal field set to the default of low_grease. 6. Leave the Filter field set to the default of all_data. 7. Leave the Excluded Fields from Signal field set to the default of empty. 8. Click Submit. 9. Wait ~30 seconds for Signal State to change to COMPLETED The results will be displayed at the bottom.             The results show that the five Frequency Bands for Sensor 1 are the most highly correlated with determining our goal of detecting a low grease condition. For Sensor 2, only bands one and four seem to be at all related, and bands two, three, and five are hardly related at all.   Click here to view Part 2 of this guide.