cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

The PTC Community email address has changed to community-mailer@ptc.com. Learn more.

Simple test case: Anomaly Detection is detecting an anomaly when it shouldn't

nahuel
14-Alexandrite

Simple test case: Anomaly Detection is detecting an anomaly when it shouldn't

Hi community,

 

To test the anomaly detection with a simple use case I followed the steps in this guide, imported the anomaly mashup and to not wait for too long I updated the cmoTimer update rate to 50ms.

 

I didn't get to the stage of modifying, for example the sine amplitude, because prior to any modification the mashup showed me that is detecting as an anomaly the same sine used for training.

AnomalyDetectionExperiment_DefaultSine_50ms.png

I think that updating the update rate to 50ms shouldn't be the reason of the false positive detection but I could be wrong.

Any thoughts why the anomaly detection is firing when it shouldn't?

 

Regards

1 ACCEPTED SOLUTION

Accepted Solutions
cmorfin
19-Tanzanite
(To:nahuel)

Hi @nahuel 

 

Please find attached my entity.

I did some other tests and notice some blip when the timer was at 5 ms, so there is maybe something with load.

Also at 50 ms, sometimes, not always, I see a handful of anomaly at the beginning of monitoring but they disappear after that. So you may want to let it run for a while to check if you have the same behaviour or not.

 

Thanks

Christophe

 

View solution in original post

11 REPLIES 11
cmorfin
19-Tanzanite
(To:nahuel)

Hi @nahuel 

 

I just made a test with this sample entities with the timer at 50 ms, but it appears to behave properly on my side.

Could you please confirm what version of ThingWorx and ThingWorx Analytics you are using ?

I tested in TWX 9.1.2 - TWA 9.1.0 - maybe you could try in this version too.

Also have you checked the Alert History to confirm anomaly are reported there (to exclude a display issue)

 

Also the screenshot shows that the calibrating / training time are at 21:42, but the signal data shows at 18:42.
I am not sure that has an impact but it would seem we have a variety of timezone involved. Can you confirm the OS and timezone of both ThingWorx and ThingWorx Analytics server ?

 

Regards

Christophe

nahuel
14-Alexandrite
(To:cmorfin)

Hi @cmorfin,

 

On my side I have the following versions
TWX 9.1.0-b10877
TWA 9.1.0-r4

 

I checked and the anomalies are being reported to the Alert History stream.

 

Regarding the difference in time, both, TWX and TWA, are installed in the same Windows Server 2016 Datacenter machine, and the timezone is set to UTC.
I looked at the Anomaly_Dashboard_Property mashup and I think that the difference in time is related to the FormatDate service (of the AnomalyServices thing) and that the server timezone is UTC and my browser timezone is UTC-3.
I looked at the FormatDate service, and the dateFormat string does not have the timezone into account.
I tried bypassing that service and binding directly to the Calibration Start Time widget and the time is displayed correctly (the format is different but the value is ok). Check the screenshot below.

nahuel_0-1624986677692.png

 

I repeated the test for 50ms (one more time), 100ms and 1sec - as the update rate value was increased (from 50ms to 1000ms) the number of false positives was reduced but there were always a few. Attached you can find the screenshots for each one.

 

Any suggestions on how to keep troubleshooting?

cmorfin
19-Tanzanite
(To:nahuel)

Hi @nahuel 

 

Could you post the configuration details of your anomaly alert ?

The certainty parameter will have an impact on the number of false positive, so you may want to adjust this.

 

Regards

Christophe

nahuel
14-Alexandrite
(To:cmorfin)

Hi, @cmorfin.

 

Sure, here it is.

nahuel_0-1625066235234.png

As suggested in the guide, it is the default configuration, and it is the same for each experiment (50ms, 100ms, 1000ms).

 

Could the reason be related to hardware sizing or overload?
The VM on which TWX and TWA is installed has the following characteristics

nahuel_3-1625066335282.png

And since this VM is a dev one, it has more PTC components installed too. Here is the list of installed components:
+ TWX
+ TWA
+ Vuforia Experience Service
+ Vuforia Studio
+ Kepware

 

Regards,

Nahuel

nahuel
14-Alexandrite
(To:nahuel)

Hi @cmorfin.

 

Any thoughts on how to keep troubleshooting? Any log or configuration to look at?

 

Regards,

Nahuel

nahuel
14-Alexandrite
(To:nahuel)

Hi @cmorfin ,

 

Since the anomaly detection for the sine wave with an update rate of 50ms worked on your side, could you export and share the Thing on which the alert was configured? That way I could import the exported Thing (that includes the correctly trained model) and test if it works correctly on my side.

 

Kind regards,

Nahuel

cmorfin
19-Tanzanite
(To:nahuel)

Hi @nahuel 

 

Please find attached my entity.

I did some other tests and notice some blip when the timer was at 5 ms, so there is maybe something with load.

Also at 50 ms, sometimes, not always, I see a handful of anomaly at the beginning of monitoring but they disappear after that. So you may want to let it run for a while to check if you have the same behaviour or not.

 

Thanks

Christophe

 

nahuel
14-Alexandrite
(To:cmorfin)

Hi @cmorfin ,

 

Thanks for the reply.

I've been doing some tests on my side, and with the following setup I got that the fastest update rate I can get without having glitches is 200ms.

nahuel_1-1628113741524.png

 

Just as a reference purpose, would you mind sharing your setup?

 

Regards,

Nahuel

cmorfin
19-Tanzanite
(To:nahuel)

Hi @nahuel 

 

I am running this on a Windows 10 machine with 32 Gb of RAM and 6 core (12 logical processors)

 

Regards

Christophe

 

slangley
23-Emerald II
(To:nahuel)

Hi @nahuel

 

If your issue has been resolved with help from one of the previous responses, please mark the appropriate one as the Accepted Solution for the benefit of others with the same problem.

 

Regards.

 

--Sharon

nahuel
14-Alexandrite
(To:slangley)

Hi @slangley I'm still running some experiments on this. I'll mark the appropriate response as the Accepted Solution when I solve this issue