IoT & Connectivity Tips

This video begins Module 3: Data Profiling of the ThingWorx Analytics Training videos. It describes the process of examining your data to make sure that it is suitable for the use case you would like to explore.

With ThingWorx, we can already use univariate anomaly alerts (on a single sensor value). However, in many situations, the readings from an individual sensor may not tell you much about the overall issue and a multivariate anomaly detector can be more useful. This post is intended to provide an overview of the Azure Anomaly Detector and how it can be integrated with ThingWorx. The attachment contains: A document with detailed instructions about the setup; A .csv file with the multivariate timeseries dataset; A .twx file with some entities that need to be imported in ThingWorx as well as the CSVParser extension that needs to be installed; A .zip file that will need to uploaded in an Azure Blob Container at some point in the setup

Beginning with version 8.4.0 ThingWorx Analytics Server can now automatically create metadata (Json file) based on the uploaded csv. file. Underneath video demonstrates the steps for automated metadata detection.

This video concludes Module 6: Predictive Models & Model Validation of the ThingWorx Analytics Training videos. 

    Generate engine-failure predictions and gain insight into your data with machine learning.   GUIDE CONCEPT   This guide will upload captured data from an Edge MicroServer (EMS) "Engine Simulator" to ThingWorx Analytics Builder.   Following the steps in this guide, you will create an analytical model, and then refine it based on further information from the Analytics platform.   We will teach you how to determine whether or not a model is accurate and how you can optimize both your data inputs and the model itself.   NOTE: This guide's content aligns with ThingWorx 9.3. The estimated time to complete ALL parts of this guide is 60 minutes     YOU'LL LEARN HOW TO   Load an IoT dataset Generate machine learning predictions Evaluate the analytics output to gain insight     Step 1: Scenario   In this guide, we’re continuing the same MotorCo scenario, where an engine can fail catastrophically in a low-grease condition.   In previous guides, you’ve gathered and exported engine vibration-data from an Edge MicroServer (EMS).   The goal of this guide is to now import that previously-exported Comma-Separated Values (.csv) data into ThingWorx Analytics, and then create an analytical model for predictive maintenance.   Analytical model creation can be extremely helpful for the automotive segment in particular. For instance, each car that comes off the factory line could have an EMS constantly sending data from which an analytical model could automatically detect engine trouble.   This could enable your company to offer an engine monitoring subscription service to your customers.   This guide will show you how to build an analytic model of your engine to facilitate this monitoring service.     Step 2: Upload Simulated Data   This guide assumes that you are using either the hosted trial (with has both Foundation and Analytics pre-installed) or a combination of the Foundation and Analytics downloadable installers.   To confirm that Foundation is communicating with Analytics, perform the following steps:   On the ThingWorx Foundation left-side navigation column, click Analytics > Analytics Builder > Settings.   At the top-right in the Analytics Server Version field, ensure that you see an appropriate version number.     NOTE:  If you use your own dataset, it's possible that you're results in the following steps will differ from those created by the provided-dataset. If you were unable to generate a 30,000+ entry dataset in the last guide, then you may download testCSVfile.csv attached here,instead. You will also need to download and extract vibration_metadata.zip which describes each column of the dataset. On the left, click Analytics Builder > Data.   Under Datasets, click New....   In the Dataset Name field, enter simulated_dataset. In the File Containing Dataset Data section, search for and select testCSVfile.csv. In the File Containing Dataset Field Configuration section, search for and select vibration_metadata.json.   Click Submit. Note that the time it takes to import the dataset is determined by its size.       Step 3: Simulated Signals and Profiles    The Signals section of ThingWorx Analytics looks for the most statistically correlated single field in the dataset which relates to your selected goal.   This doesn't necessarily indicate that it is the cause of your goal, whether maximizing or minimizing. It just means that the dataset indicates that this single field happens to correlate with the goal that you desire.   On the left, click Analytics Builder > Signals.   At the top, click New….   In the Signal Name field, enter simulated_signal. In the Dataset field, select simulated_dataset.   Click Submit. Wait ~30 seconds for Signal State to change to COMPLETED     Unfortunately, our results aren't very good. Or, more accurately, they're too good.   Our simulated dataset has some noise in it from adding random values to our five frequency bands on each our two sensors. However, ThingWorx Analytics has instantly seen through that noise and discarded it. Instead, it's only detected that s2_fb5 isn't relevant.   If you look back at the Use the EMS to Create an Engine Simulator guide, you'll see that s2_fb5 has the same base value between both a "good grease" and a "bad grease" condition, i.e. a base of 190.   This does show already that Analytics is working, though. Since s2_fb5 didn't change between good and bad grease conditions, our Signal analysis is indicating that it's not relevant to our model.   Profiles   Now, let's do the same for a Profile.   The Profiles section of ThingWorx Analytics looks for combinations of data which are highly correlated with your desired goal.   On the left, click Analytics Builder > Profiles.   Click New....   In the Profile Name field, enter simulated_profile. In the Dataset field, select simulated_dataset.   Click Submit. Wait ~30 seconds for the Profile State to change to COMPLETED.     Just like with Signals, our Profile is too good. In fact, Analytics is indicating that just s1_fb2 by itself is the primary indicator of good vs. bad grease conditions.   This is likely due to random chance. The random noise added to s1_fb2 just happened to be slightly less than the other frequency bands, so everything else was discarded.   Regardless, ThingWorx Analytics is quickly seeing through our simulated data.   Next, we'll actually create a Model using the simulated dataset.     Click here to view Part 2 of this guide  

  Use the Edge MicroServer (EMS) to simulate an engine with vibration sensors.   GUIDE CONCEPT   The Edge MicroServer (EMS) facilitates connectivity from Edge devices to ThingWorx Foundation.   It’s often easier, though, to start development with simulated Edge values rather than hooking up sensors.   This guide will show you how to simulate vibration values of an engine using the EMS.     YOU'LL LEARN HOW TO   Modify an EMS Template Provision Thing Properties and Values from an EMS rather than Foundation Send information from an EMS to Foundation Store large amounts of data in an InfoTable Property Create a simulator for testing   NOTE:  The estimated time to complete all parts of this guide is 30 minutes.     Step 1: Scenario   MotorCo manufactures, sells, and services commercial motors.   Recently, MotorCo has been developing a new motor, and they already have a working prototype.   However, they’ve noticed that the motor has a chance to FAIL CATASTROPHICALLY if it’s not properly serviced to replace lost grease on a key moving part.     In order to prevent this type of failure in the field, MotorCo has decided to instrument their motors with sensors which record vibration.   The hope is that these sensors can detect certain vibrations which indicate required maintenance before a failure occurs.   In this guide, you’ll begin this monitoring process by using ThingWorx Foundation to monitor and record vibration data from the prototype motor. In particular, you will learn how to provision Thing Properties and Values from an EMS, as well as how to permanently store these values for analysis in an Info Table Property.   These types of modifications to an EMS can be extremely helpful for the automotive segment in particular. For instance, each car that comes off the factory line could have custom, auto-generated EMS scripting that would dynamically create Foundation information for each car in the fleet. This could be a massive time-savings versus manually generating Thing Properties directly within Foundation.   Because the motor is still in the process of being instrumented with sensors, you’ll get all the functionality in-place beforehand by constructing a motor simulator using the Edge MicroServer (EMS).     Step 2: Modify config.lua   In the previous Use the Edge MicroServer (EMS) to Connect to ThingWorx  guide, you installed the EMS on a Windows PC, configured it to talk to ThingWorx Foundation, and then created an EdgeThing on Foundation to complete the connectivity.   This guide assumes that you have already completed that Windows EMS guide and have an active EMS connection to the EdgeThing.   Perform the following steps to modify this connection to increase the task rate of the EMS, which we'll use in the following steps to update Properties more quickly.   On your Windows PC, select the Windows PowerShell window where the luaScriptResource.exe program is running.   Type Ctrl-C to close the luaScriptResource.exe operation, i.e. hold the Control key and hit the C key.   Minimize the luaScriptResource.exe PowerShell window, and activate the wsemse.exe PowerShell window.   Type Ctrl-C to close the wsems.exe operation.   Return to Foundation, and note that EdgeThing is not connected.   Navigate to the C:\CDEMO_EMSE\etc directory.   Open config.lua in your prefered text-editor.   Change scanRate to 1000. Add the following line below the scanRate line: taskRate = 1000,   The final code of config.lua should be the following Note that the EMS may have slightly modified your config.lua file, such as adding a data_security_key line. Leave these EMS-generated modifications alone. scripts.log_level = "WARN" scripts.script_resource_ssl = false scripts.script_resource_authenticate = false scripts.EdgeThing = { file = "thing.lua", template = "YourEdgeThingTemplate", scanRate = 1000, taskRate = 1000, sw_update_dir = "C:\\CDEMO_EMS\\updates" } Save the config.lua file.     Click here to view Part 2 of this guide.

This demo walks through how Range Count works. The Range Count service calculates the difference between the maximum and minimum value.  Agenda of the demo: 1. Create a demo Thing 2. Add a new property to the Thing 3. Add the property statistical calculation type Range Count to the Thing 4. Validate the statistical calculation service via the added calculation type Range Count 5. Validate the statistical calculation service via the Service QueryTimedValuesForProperty      

This video concludes Module 3: Data Profiling of the ThingWorx Analytics Training videos. It shows you a few examples of questions that should be asked of a subject-matter expert (SME) to better understand the information contained in a dataset. Using answers to these questions, you will  use a tool such as Microsoft Excel to modify a given dataset, and prepare it for future exercises in this course.

    Step 6: Map Data   Now that the event is created, we need to map the Properties of EdgeThing to the fields required to invoke an Analysis Job.   We'll start with the Inputs.   Select the previously-created Event, and click Map Data....   Click Inputs Mapping.   In Source Type, select Thing. In Source, search for and select EdgeThing.   On the left, scroll down and select s1_fb1. Note that you do NOT want the s1_fb1 that is part of the InfoTable Property, because the Info Table Property only stores recorded data, not live data. On the right, select _s1_fb1, the first frequency band required for the Model to make a prediction.   Click the Map button in the center.   Repeat this mapping process for for s1_fb2 through s1_fb5.   Map causalTechnique to causalTechnique in the same manner. This is a String Property in EdgeThing with a Default Value of "FULL_RANGE". Map goalField to goalField in the same manner. This is a String Property in EdgeThing with a Default Value of "low_grease".   Map Results   Now that the Inputs are mapped, we also want to map the Results.   Click Results Mapping on the left.   Map _low_grease to Result_low_grease. Map _low_grease_mo to Result_low_grease_mo.   Click Close to close the mapping pop-up.   Enable Event   Now that we've done the mapping from Foundation to Analytics, let's Enable the Analysis Event so that it can automatically generate and process Analysis Jobs. Select the mapped Analysis Event. Select Enable.   Now that you have enabled the Analysis Event, the new data will be submitted to Analytics Manager whenever EdgeThing's s1_fb1 Property changes.   An Analysis Job will automatically run, with a predictive score sent back and stored in EdgeThing's Result_low_grease (Boolean) and Result_low_grease_mo (Number) Properties.     Step 7: Check Jobs   In this step, we'll confirm that the automatic analysis of information coming from remote devices is operational.   On the ThingWorx Composer Analytics tab, click Analytics Manager > Analysis Jobs.   Uncheck Filter Completed Jobs.   Select a Job and click View.... Click Results.   NOTE: You will see true or false, corresponding to either a low grease or no low grease condition. Using this technology, you could create a paid customer service, where you offered to monitor remote engines, in return for automatically shutting them down before they experience catastrophic engine failure.   For that example implementation, you would utilize the EdgeThing.Result_low_grease BOOLEAN Property to trigger other actions.   For instance, you could create an Alert Event which would be triggered on a true reading.   You could then have a Subscription which paid attention to that Alert Event, and performed an action, such as sending an automatic shutdown command to the engine when it was experiencing a likely low grease event.   NOTE: We recommend that you return to the ThingWorx Composer Analytics > Analytics Manager > Analysis Events tab and Disable the Event prior to continuing. Since the simulator generates an Event every ~1 seconds, this can create a large number of Events, which can fill up your log.       Step 8: Next Steps   Congratulations. You've completed the Manage an Engine Analytical Model guide. In this guide you learned how to:   Define an Analysis Provider that uses the built-in Analytics Server Connector Publish a Model from Analytics Builder to Manager Create an Analysis Event which takes data from ThingWorx Foundation and decides whether or not a failure is likely   The next guide in the Vehicle Predictive Pre-Failure Detection with ThingWorx Platform learning path is Engine Failure-Prediction GUI.   Learn More   We recommend the following resources to continue your learning experience:    Capability     Guide Build Implement Services, Events, and Subscriptions Guide   Additional Resources   If you have questions, issues, or need additional information, refer to:    Resource              Link Community Developer Community Forum Support Analytics Manager Help Center      

This video continues Module 6: Predictive Models & Model Validation of the ThingWorx Analytics Training videos. It describes the remaining machine learning algorithms used by ThingWorx Analytics to build predictive models that weren't covered in Part 1. In addition, this video describes the different kinds of ensembles you can build that utilize multiple algorithms. 

  Build a Predictive Analytics Model - Video Guide   This project will introduce ThingWorx Analytics Builder via a convenient video-guide. Following the steps in this video, you will create an analytical model, and then refine it based on further information from the Analytics platform. We will teach you how to determine whether or not a model is accurate and how you can optimize both your data inputs and the model itself.     ThingWorx Analytics Server Installation with SSL   This video demonstrates the installation of Analytics Server with SSL protections enabled. It includes information about generating the necessary certificates and truststores to enable SSL for each component that connects with the server.     ThingWorx Platform Analytics Installation with SSL   This video demonstrates the Platform Analytics installation with SSL protections enabled. It provides information about generating the necessary certificates and truststores to enable SSL for all connected components, including RabbitMQ and Flink.     Toolbar Widget | ThingWorx 9   Watch this video to learn how to add a Toolbar widget to a mashup in ThingWorx. You'll learn how to define actions using a data service and create bindings to control and configure a Grid widget.     ThingWorx SSO: Login demonstration from Azure AD to ThingWorx   This video demonstrates the ThingWorx SSO login procedure using Azure AD. The login procedure is shown from a user point of view. Then a behind the scenes view looks at the design in ThingWorx Composer.     Menu Bar Widget ThingWorx 9   Watch this video to learn how to create a mashup layout that uses a Menu Bar widget for navigation. You'll learn how to create a layout, define menu items, and configure the widget.     ThingWorx AD FS SSO Setup   This video provides a walk-through of the steps required to set up SSO for ThingWorx in an environment where AD FS is both the CAS and the IdP. The focus is on the AD FS setup steps.     Create Your Application UI   Following the steps in this video-guide, you will learn how to use the ThingWorx Mashup Builder tool to create a Graphical User Interface (GUI) for your IoT Application. We will teach you how to rapidly create and update a Mashup, which is a custom visualization built to display data from devices according to your application's business and technical requirement     Get Started with ThingWorx for IoT   Explore the ThingWorx Foundation Internet-of-Things application building platform in a convenient, instructional video guide format.     Thingworx Mashup 101 - Do's and Don'ts   This session covers the most common and useful tips about how to correctly use Mashup builder, Widgets and Layouts – and what to avoid - to create applications with good principles of UI/UX and easier to maintain.     What's New in ThingWorx 9.1   The industry’s most complete IIoT platform just got better. Loaded with a full range of new and updated features and functionality, you can expect powerful platform capability enhancements across the board.       Standardize Connectivity to Devices, Applications, & Systems for Centralized IIot Data   You can’t implement IIoT solutions if you can’t connect to your assets, but in complex environments connectivity can seem like an insurmountable challenge. ThingWorx makes it easy to establish standardized connectivity, so you can create a secure, single-source for accessing industrial data across your IT and OT systems     Bar Chart Widget | ThingWorx 9   Watch the following video on how to add the Bar Chart widget to a mashup and configure basic widget properties.     Button Widget | ThingWorx 9   Watch the this video to learn how to add the Button widget to a mashup and use its Clicked event to trigger data services.     Line Chart Widget | ThingWorx 9   Watch the this video to learn how to add the Line Chart widget to a mashup and bind a data source.     ReImagine Your Application UI With Collection and Custom CSS   Create compelling, modern application user interfaces in ThingWorx with the latest enhancements to our Mashup visualization platform - Collection and Custom CSS. In this webinar with IoT application designer Gabriel Bucur, we'll show how the new Collection widget makes it easy to replicate visual content in your UI for menu systems, dashboards, tables, and more.     Testing Your Edge Application   Native testing is an often-overlooked aspect of edge application development. This session shares how to perform thorough testing before you push your application into live production.     Predictive Maintenance with Thingworx 101   Adopting a predictive maintenance strategy for the first time can often feel daunting, but it doesn't have to. Learn how ThingWorx can be used to leverage edge devices and turn monitoring activity into action.     ThingWorx Mashup Pitfalls: What to Avoid and What Not to Do   Designing Mashups takes time. Moreover, expanding upon a Mashup that was built for a small-scale application can get cumbersome.     IoT Security: Keeping Devices Safe in a High Risk World   Understanding access controls can be difficult. With the built-in security features and concepts in ThingWorx, you can easily give your devices access on-demand.     Edge Connectivity in Unreliable Networks   The Store and Forward feature within ThingWorx Industrial Connectivity assures users that critical data collected at the edge won't get lost during an outage.  

The intend of this post. This post is for the user who want to validate that, the ThingWorx Analytics Services related to Confidence Models work successfully. Underneath video walk through the steps to validate the Services via a non-supported PTC Mashup. The intend of this video is uniquely to validate that, Services related to Confidence Models works successfully.  What package files are used in the video? The Mashup entities and dataset used in the video, is attached to this post. Feel free to download the files and test on your machine. Why use Confidence Models? A confidence model is a way of adding confidence interval information to a predictive model. Statistically, for a given prediction, a confidence model provides an interval with upper and lower bounds, within which it is confident, up to a certain level, that the actual value occurs. During predictive scoring, this measure of confidence provides additional information about the accuracy of the prediction. More information about Confidence Models can be found here at PTC Help Center 

This video shows the steps to install ThingWorx Analytics Server 8.5.1 as well as the ThingWorx Analytics Extension.  

This video continues Module 1: ThingWorx Analytics Overview of the ThingWorx Analytics Training videos. It covers some of the functionality of the ThingWorx platform, as well as ThingWorx Analytics capabilities.

    Step 7: Add Grid   It might also be helpful to display the data you've used to build the ThingWorx Analytics model.   In the future, this might be split out into an entirely separate page/Mashup that is exclusively devoted to backend model creation, but that would be beyond the scope of this guide.   For now, we'll simply display that collected data via the Grid Widget      1. On the EEFV_Mashup, drag-and-drop a Grid Advanced Widget onto the bottom-left Canvas section..     2. On the top-right Data tab, click the green </> button beside Things_EdgeThing. Note that this will open the Add Data pop-up, but with EdgeThing pre-selected.   3. In the Services Filter field, type getproperties.   4. Click the right-arrow beside GetProperties to add it to Selected Services on the right.   5. Check Execute on Load.     6. Click Done.   7. Under the Data tab, expand GetProperties to reveal the options.     8. Drag-and-drop Things_EdgeThing > GetProperties > infoTableProperty onto the Grid Advanced Widget.     9. On the Select Binding Target pop-up, click Data.     10. Click Save.   11. Click View Mashup.         Step 8: Add Controls   Throughout this Learning Path, it has been recommended that you stop Analysis Events when not actively using their functionality.   However, this requires going into the backend of ThingWorx Analytics to disable the event, which is not ideal.   Instead, let's enable or disable the Analysis Event from inside the Mashup by adding some Button Widgets to directly interface the Analytics backend for us.       1. Click the bottom-right Canvas section to select it.         2. In Mashup Builder top-left, click the Layout tab.         3. Under Positioning, click the Static radio-button.         4. Click the Widgets tab.       5. Drag-and-drop a Button Widget onto the bottom-right section.         6. Drag-and-drop another Button Widget onto the bottom-right section.       7. Drag-and-drop a Text Field Widget onto the bottom-right section.       8. Click Save.       Bring in More Data   Now that we have Buttons to trigger enable/disable, as well as a Text Field to display information, we now need to bring in some additional Mashup Data Services to interact with the ThingWorx Analytics backend.       1. Click the green + button at the top of the Data tab.       2. In the Entity Filter field, search for and  select TW.AnalysisServices.EventManagementServicesAPI.       3. In the Services Filter field, search for and add QueryAnalysisEvents by clicking the right arrow.       4. Check Execute on Load.         5. In Services Filter, search for and select EnableAnalysisEvent by clicking the right arrow. Note that you should NOT check "Execute on Load", as we'll trigger this Service only when the Button is clicked.     6. In Services Filter, search for and select DisableAnalysisEvent by clicking the right arrow. Likewise, do NOT check "Execute on Load" here either.       7. Click Done.         8. Click Save.     Display Event Key   To enable or disable Analytics Events, we need to know the eventId, which is returned by QueryAnalysisEvent Service as the parameter labeled key.   We'll bind that to the Text Field Widget for later usage in enabling/disabling.        1. Change the top-button's Label Property to Enable Analytics Event.       2. Change the bottom-button's Label Property to Disable Analytics Event.         3. Under the Data tab, expand QueryAnalysisEvents > Returned Data > All Data to reveal the options. .       4. Drag-and-drop QueryAnalysisEvents > Returned Data > All Data > key to the TextField Widget.         5. On the Select Binding Target pop-up, click Text.         6. Click Save.     Enable/Disable Analytics Events   Now that we know the key/eventId, we can call the EnableAnalysisEvent and DisableAnalysisEvent services.       1. Under the Data tab, expand EnableAnalysisEvent > Parameters to reveal eventId.       2. Click the Text Field Widget to select it, and then click the top-left drop down to reveal the options.         3. Drag-and-drop the Text Field's Text Property onto EnableAnalysisEvent > Parameters > eventId.         4. Repeat steps 1-3 for DisableAnalysisEvent.         5. Click the Enable Analytics Event Button Widget to select it, then click the top-left to reveal the drop down option .       6. Drag-and-drop the Clicked Event onto the EnableAnalysisEvent Service under the Data tab.         7. Repeat steps 5-6 for DisableAnalysisEvent, using the other Button Widget.         8. Click Save     Step 9: View Mashup   Throughout this guide, we've added various additional functionality to our MVP Mashup. At this point, you could continue to update the Mashup as you see fit.   For instance, you could change the background color of the top-left section to better match the header. Or you could further modify the original Mashup shown in the Contained Mashup Widget so that it better fits in the allowed space. You could add another Label Widget to the Header section to also display the company's motto / tag-line.   Regardless, when you are done with modifications, Save and click View Mashup.     Note that you can left-click-and-drag on the Time Series Chart to select particular time ranges. Or you could add a Time Selector Widget to the bottom-right section to control it there.   Similarly, you could add controls for the Grid Widget to only show the Identifier ranges in which you were interested.   Or you could split out the Model-creation values to a completely separate Mashup as previously discussed.   The extent to which you develop your Mashup is entirely up to you.        Step 10: Next Steps   Congratulations. You've completed the Enhanced Engine Failure Visualization guide. In this guide, you learned how to:   Create a Mashup with a Header Divide your Mashup into Sub-sections Use a Contained Mashup to reuse development Store historical data in a Value Steam Display historical data in a Line Chart Show spreadsheet data via a Grid Advanced Widget Tie Mashup controls into the ThingWorx backend   This is the last guide in the Vehicle Predictive Pre-Failure Detection with ThingWorx Platform learning path.   Learn More   We recommend the following resources to continue your learning experience:   Capability  Guide Build Implement Services, Events, and Subscriptions Guide   Additional Resources   If you have questions, issues, or need additional information, refer to:   Resource Link Community Developer Community Forum Support Analytics Manager Help Center

  Use the Statistical Calculation Thing Shape to Execute Common Statistical Functions   GUIDE CONCEPT   This project will introduce the Statistical Calculation Thing Shape.   The steps in this guide outline how to utilize Descriptive Analytics in ThingWorx Analytics to perform common mathematical analyses on data sets. You will learn how to use the Statistical Calculation Thing Shape's built-in functionality to calculate the mean, median, mode, and other useful insights.     YOU'LL LEARN HOW TO   Create a Value Stream and Data Shape Create a Thing with the Statistical Calculation Thing Shape Modify a Property to record values to the Value Stream Utilize various built-in services to perform the Mean, Median, Mode, and Standard Deviation calculations   NOTE: The estimated time to complete this guide is 30 minutes.     Step 1: Introduction   Descriptive Analytics enables users to perform on-demand common statistical calculations and enable statistical monitoring. Output from these Services can then easily be added to IoT applications built with the ThingWorx platform.   For example, output generated by Descriptive Analytics can be used to build solution-specific visualizations (Mashups or Widgets), create Alerts based on logged Property values, or generate transformed data for machine learning processes.   Descriptive Analytics includes two microservers, each with its own set of statistical Services. This guide will deal specifically with the Statistical Calculation Thing Shape.   This Thing Shape will add a variety of built-in Services to any Thing or Thing Template you choose, such as calculations for Mean, Median, Mode, or Standard Deviation.   To perform these statistical calculations, a Property must have time-series data logged to a Value Stream.   In addition, the Statistical Calculation Thing Shape has been optimized to perform calculations only on particular time ranges and with a maximum entry-count. This helps minimize some of the performance hits that can occur from repeatedly accessing Value Streams.   You perform this "data grooming" via the QueryTimedValuesForProperty Service, which is also part of the Statistical Calculation Thing Shape.   In addition to the name of a Property with values logged to a Value Stream, you also provide a Start Date, End Date, and Max Item Count.   The QueryTimedValuesForProperty Service then formats the values from the Value Stream to work with for the other built-in statistical calculation Services.   Step 2: Create Prerequisites   The inputs to the built-in Services of the Statistical Calculation Thing Shape require time-series data stored in a Value Stream. Setting a Thing's Properties to be Logged will store valid time-series data on which to perform Descriptive Analytics.   Create Value Stream   Follow the steps below to create a Value Stream that you will later tie to a Thing.   On the ThingWorx Composer Browse tab, click DATA STORAGE > Value Streams, + New.   Select ValueStream and click OK.   In the Name field, enter scts_valuestream. If Project is not already set, search for and select PTCDefaultProject.   At the top, click Save.   Create Data Shape   You will need a Data Shape to format the timed_values Property we will create in the next step.   On the ThingWorx Composer Browse tab, click MODELING > Data Shapes, + New.   In the Name field, enter scts_timed_values_datashape. If Project is not already set, search for and select PTCDefaultProject.   At the top, click Field Definitions.   Click + Add. On the right slide-out, in the Name field, enter value. Change the Base Type to NUMBER.   At the top-right, click the "check with a +" icon for Done and Add. In the Name field, enter timestamp. Change the Base Type to LONG.   At the top-right, click the "check" icon for Done. At the top, click Save.       Step 3: Create Thing   Next, you will create a Thing and tie the previously-created Value Stream to it.   You will assign the Statistical Calculation Thing Shape to get a variety of built-in analytics Services to manipulate the data. You will also create a series of Properties to facilitate the new Services.   On the ThingWorx Composer Browse tab, click MODELING > Things, + New.   In the Name field, enter scts_thing. If Project is not already set, search for and select PTCDefaultProject. In the Base Thing Template field, search for and select GenericThing. In the Implemented Shapes field, search for and select StatisticalCalculationThingShape. In the Value Stream field, search for and select scts_valuestream.   At the top, click Save.   Add Properties   Next, you will add a series of Properties to scts_thing.   Perform the following steps repeatedly until all Properties have been added.   At the top, click Properties and Alerts.   Click + Add. On the right slide-out, in the Name field, enter numbers. Change the Base Type to NUMBER. Click Persistent. Click Logged.   At the top-right, click the "check with a +" icon for Done and Add. Repeat steps 3-7 until all of the properties in the table below have been added. NOTE: For the final standarddev_result Property, do NOT click Done and Add; you'll just click the check for Done instead. Property Name Base Type Data Shape Persistent  Logged start_time DATETIME N/A Checked NOT checked end_time DATETIME N/A Checked NOT checked timed_values INFOTABLE scts_timed_values_datashape Checked NOT checked mean_result NUMBER N/A Checked NOT checked median_result NUMBER N/A Checked NOT checked mode_result INFOTABLE none, i.e. leave it blank Checked NOT checked standarddev_result NUMBER N/A Checked NOT checked     At the top-right, click the "check" icon for DONE. At the top, click Save.   Step 4: Set Properties   You will now set the properties to guarantee that each calculation gives us a different answer.   Mean is the average. Median is the "middle" number from the dataset. Mode is the most common number. Standard Deviation is a measure of the "dispersion" of the dataset.   You will use the following dataset: 1, 5, 9, 5, 9, 1, 9.   The mean is 39 / 7 = 5.571...   The median is 5, as 5 is the middle of 1, 5, and 9.   Mode is 9, because 9 appears most commonly in the dataset.   Standard Deviation is 3.5989...   Perform the following steps to enter the above values into the numbers property.   Set numbers   Under the Value column and on the numbers property row, click the "pencil" icon for Set value of property.   On the right slide-out, enter 1.   At the top-right, click the "check" icon for Done. At the top, click Save.   Repeat steps 1-4 above, changing the value each time according to the table below: Value Change Count  Entered Value 2nd 5 3rd 9 4th 5 5th 9 6th 1 7th 9   You also need to set the start_time and end_time. These are dates used to define the time-period in which the Statistical Calculation Thing Shape will search for values.   For instance, you could set the calculations to run at midnight, and then use the past 24-hours as your time-period.   For this example, set dates to be 24 hours prior to the time at which you set the above values of the numbers property, through 24 hours in the future.   Set start_time   Under the Value column and on the start_time Property row, click the "pencil" icon for Set value of property.   On the right slide-out, search for and select the previous day.   At the top-right, click the "check" icon for Done. At the top, click Save.   Set end_time   Under the Value column and on the end_time property row, click the "pencil" icon for Set value of property.   On the right slide-out, search for and select the following day. At the top-right, click the "check" icon for Done. At the top, click Save.     Click here to view Part 2 of this guide.

This video continues Module 6: Predictive Models & Model Validation of the ThingWorx Analytics Training videos. It covers some modeling techniques to help build better predictive models. It discusses the dangers of models that overfit data, and how to avoid overfitting. 

  Create an Engine Failure Prediction GUI to warn about customer issues.   GUIDE CONCEPT   This guide will use ThingWorx Foundation's Mashup Builder to create a Graphical User Interface (GUI) to display results from Analytics Manager comparisons of your analytical model to real-time data.   Following the steps in this guide, you will learn how to utilize Widgets and backend data to quickly visualize customer failure conditions.     YOU'LL LEARN HOW TO   Create a Mashup Add Widgets to represent different data Bring Backend Data into the Mashup Tie data to Widgets View Analytical Results in a convenient GUI   NOTE:  The estimated time to complete all parts of this guide is 30 minutes.     Step 1: Scenario   In this guide, we're finishing up with the MotorCo scenario where an engine can fail catastrophically in a low-grease condition.   In previous guides, you've gathered and exported engine vibration-data from an Edge MicroServer (EMS) and used it to build an engine analytics model. You've even put that analytical model into service to give near-immediate results from current engine-vibration readings.   The goal of this guide is to create a GUI to visualize those predicted "low grease" conditions to facilitate customer warnings.     GUI-creation to visualize analytical model deployment can be extremely helpful for the automative segment in particular. For instance, each car that comes off the factory line could have an EMS constantly sending data from which an analytical model could automatically detect engine trouble.   This could enable your company to offer an engine monitoring subscription service to your customers.   This guide will show you how to visualize the results of an engine analytic model for a "smart, connected products" play.     Step 2: Create Mashup   Mashups are ThingWorx Foundation's method of creating Graphical User Interfaces (GUIs).   Mashups are created through the Mashup Builder interface.   Before you can use this drag-and-drop interface, you must first create a new Mashup.      1. In ThingWorx Foundation Composer, click Browse > Visualization > Mashups.      2. Click + New.        3. Leave the defaults and click OK.        4. In the Name field, type EFPG_Mashup.      5. If Project is not already set, search for and select PTCDefaultProject.        6. At the top, click Save.        7. At the top, click Design. You are now in the Mashup Builder interface, where you can drag-and-drop graphical elements (Widgets) to create your GUI.      8. At the top-left, click the Layout tab.        9. Change Positioning to Static. This removes any auto-positioning or dynamic-sizing from our Mashup, so that we can place and size Widgets manually.      10. At the top, click Save again.       Step 3: Add Widgets   We now want to add several Widgets to our Mashup by dragging-and-dropping them into the central Canvas area.   Return to the Widgets tab in the top-left.       2. In the Filter Widgets field, type label.        3. Drag-and-drop five (5) Label Widgets onto the central Canvas area.      4. Select the Label Widgets, go to the Properties in the bottom-left, and change the Label's LabelText to s1_fb1 through s1_fb5.         5. Filter Widgets for text field, and then drag-and-drop five (5) Text Field Widgets onto the central Canvas area.        6. Drag-and-drop two (2) more Label Widgets onto the central Canvas area.      7. Change their LabelText Properties to Result_low_grease and Result_low_grease_mo.        8. Drag-and-drop two (2) more Text Field Widgets onto the central Canvas area.        9. In the top-left Widget's tab, change Category to Legacy.      10. Drag-and-drop an Auto Refresh Widget onto the Canvas.        11. At the top, click Save.         Step 4: Add Data   Now that we have a rough set of Widgets in place to display Data in our Mashup, we need to bring in backend Data from Composer.      1. In the top-right, ensure the Data tab is active.      2. Click the green + button.        3. In the Entity Filter field, search for and select EdgeThing.      4. In the Services Filter field, type getprop.      5. Click the right-arrow beside GetProperties.      6. On the right, check Execute on Load.        7. In the bottom-right, click Done.      8. On the right, expand GetProperties.        9. At the top, click Save.       Step 5: Bind Data   Now that backend Data is accessible inside Mashup Builder, we need to bind it to the various Widgets.   Drag-and-drop Data > Things_EdgeThing > GetProperties > s1_fb1 to the top-left Text Field under the s1_fb1 Label Widget.           2. On the Select Binding Target pop-up, click Text.         3. Repeat for s1_fb2 through s1_fb5 on the 2nd-5th Text Field Widgets.       4. Drag-and-drop Data > Things_EdgeThing > GetProperties > Result_low_grease to the top-right Text Field under the Result_low_grease Label Widget, and select Text on the Binding pop-up.       5. Repeat for Result_low_grease_mo on the Text Field just below.       6. Click the Auto Refresh Widget to select it.       7. In the bottom-left Properties, set RefreshInterval to 1.         8. With the Auto Refresh Widget still selected, click the top-left corner to reveal a drop-down.       9. Drag-and-drop the Refresh Event onto Things_EdgeThing > GetProperties.       10. On the right, click the GetProperties Mashup Data Service to reveal all of its interactions in the bottom-center Connections window.              11. At the top, click Save.       Step 6: View Mashup   In the last guide, we disabled the Analysis Event after confirming that appropriate analytical jobs were being created and completed whenever new data from our EMS Engine Simulator entered Foundation.   We now need to re-enable that Event so that we can see the results in our Mashup.       1. Return to Analytics > Analytics Manager > Analysis Events.       2. Select the Event, and click Enable.   View Analytical Results   With the Analysis Event enabled, we can now view our Mashup to watch engine-failure-predictions be displayed in real time.       1. Return to EFPG_Mashup.       2. At the top, click View Mashup.       3. Wait and notice how the refresh occurs every second, including both true and false analytics results for the "low grease" condition.      4. When you are satisfied that you have observed enough true/false results, return to Analytics > Analytics Manager > Analysis Events and Disable the event.   Through this Learning Path, you have done all of the following:   Connect a remote device to Foundation using the Edge MicroServer (EMS) Fed relevant product-data to the Foundation backend Formatted and exported that data in a manner which Analytics could understand Imported that data and used it to build an Analytics Model of your remote device Started feeding real-time remote data into the model to achieve immediate failure-prediction results Create a GUI to easily visualize those results   You now have a completed Minimum Viable Product (MVP) for a Service Play with MotorCo's new engines.   These new engines may have a known-failure condition, but (because you've instrumented those engines to provide constant data) you can monitor them and predict failures before they actually happen.   This could easily be an up-sell scenario for MotorCo's customers. They could simply purchase the engine. Or they could both purchase the engine and enroll in a service contract where you notified them of impending issues.   What was originally a single-point source of income is now ongoing        Step 7: Next Steps   Congratulations. You've completed the Engine Failure-Prediction GUI guide. In this guide you learned how to:   Create a Mashup Add Widgets to represent different data Bring Backend Data into the Mashup Tie data to Widgets View Analytical Results in a convenient GUI   The next guide in the Vehicle Predictive Pre-Failure Detection with ThingWorx Platform learning path is Enhanced Engine Failure Visualization.   Learn More   We recommend the following resources to continue your learning experience:   Capability Guide Build Implement Services, Events, and Subscriptions Guide   Additional Resources   If you have questions, issues, or need additional information, refer to:   Resource Link Community Developer Community Forum Support Analytics Manager Help Center

This video continues Module 6: Predictive Models & Model Validation of the ThingWorx Analytics Training videos. It then begins to describe some of the performance metrics used to evaluate predictive models. 

This video concludes Module 5: Descriptive Analytics of the ThingWorx Analytics Training videos. It covers signals, profiles, and clusters, and how these forms of descriptive analytics provide crucial insight into your data.

This video concludes Module 4: Data Transformation & Feature Engineering of the ThingWorx Analytics Training videos. It covers Descriptive Services and Derived Properties, and how they can be leveraged to create helpful alerts and make data transformation easier.