Build an Engine Analytical Model Part 3

Step 7: Real World Model

We’ll now rerun model creation with the Real World data.

Even though Signals and Profiles are possibly telling us that only Sensor 1 is needed, the first Model you’ll create will contain all the data, while the second Model will exclude Sensor 2.

We’ll then compare the Models to see which one is going to work the best for predicting engine failures.

1. On the left, click Analytics Builder > Models.
2. Click New….
   
    

   
   
   

3. In the Model Name field, enter vibration_model.
4. In the Dataset field, select vibration_dataset.
   
    

   
   
   

5. Click Submit.
   • After ~60 seconds, the Model Status will change to COMPLETED.
6. Select the model that was created in the previous step, i.e. vibration_model.
7. Click View… to open the Model Information page.

• Note that your model may differ slightly from the picture below, as the automatically-withheld "test" data is randomly chosen.

Unlike our simulated dataset, this real-world data is not perfect. However, it’s still pretty good, and is much more representative of what a real-world scenario would indicate.

The True Positive Rate shown on the Receiver Operating Characteristic (ROC) chart are much higher than the False Positives.

The curve is relatively high and to the left, which indicates a high accuracy level.

You may also click on the Confusion Matrix tab in the top-left, which will show you the number of True Positive and True Negatives in comparison to False Positives and False Negatives.

NOTE: The number of correct predictions is much higher than the number of incorrect predictions.

As such, we now know that our Sensors have a relatively good chance at predicting an impending failure by detecting low grease conditions before they cause catastrophic engine failure.

Refined Model

We can now compare this first Model that includes both Sensors to a Model using only Sensor 1, since we suspect that Sensor 2 may not be necessary to achieve our goal.

1. On the left, click Analytics Builder > Models.
2. Click New….
3. In the Model Name field, enter vibration_model_s1_only.
4. In the Dataset field, select vibration_dataset.
   
    

   
   
   

5. On the right beside Excluded Fields from Model, click the Excluded Fields button.
   
    

   
   
   

6. Select s2_fb1 through s2_fb5.
   
    

   
   
   

7. While all the s2 values are selected, click the green "right-arrow", i.e. > button, in the middle.
   
    

   
   
   

8. At the bottom-left, click Save.
   
    

   
   
   

9. Click Submit.
   • After ~60 seconds, the Model State will change to COMPLETED.
10. With vibration_model_s1_only selected, click View….
    
     

    

The ROC chart is comparable to the original model (including Sensor 2).

Likewise, the Confusion Matrix (on the other tab) indicates a good ratio of correct predictions versus incorrect predictions.

NOTE: These Models may vary slightly from your own final scores, as what data is used for the prediction versus for evaluation is random.

ThingWorx Analytics’s Models have indicated that you are likely to receive roughly the same accuracy of predicting a low-grease condition whether you use one sensor or two!

If we can get an accurate early-warning of the low grease condition with just one sensor, it then becomes a business decision as to whether the extra cost of Sensor 2 is necessary.

Step 8: Next Steps

Congratulations! You've successfully completed the Build an Engine Analytical Model guide, and learned how to:

• Load an IoT dataset
• Generate machine learning predictions
• Evaluate the analytics output to gain insight