Difference Between Signals from Models and Signals from Dataset

[joao_espadanal]

Hello community,

(Post about thingworkx analytics)

I was curious about the difference from signals that come from a model (where you build a model and then create a signal from inside the model view) and signals that are made from the signals section.

From how I understood signals are calculated, there should be no difference (i.e. no influence from the model), although, after running some tests, we have noticed they are indeed different. The model choice does not appear to influence the signal values, but values that come from the two different section are different. 

Could someone please explain me this difference ?

Which of the two should I use ? 

Thank you for all your time,
Bernardo

[cmorfin]

Hi

When you create the signal form inside the model menu, the signal takes the same configuration as the model itself: same dataset, exclusion, filter ... this is not configurable

When creating the signal from the top menu, you can configure all the field to build a specific signal.

Hope this helps

Christophe

[BernardoAlb]

Hi Christophe, 

First of all, thank you for your time,

The thing is, we built a signal from the menu with a specific set of exclusion, a defined filter and a dataset.
Then we built a model with the exact same exclusion, the exact same filter and the exact same dataset and the two signals give different results.

To be more precise, the Mutual Information values for the fields were all diferent, and also was their order (so it is not just a scale difference).
Although, within each field, the values for average goal, difference to average and number of records where all the same. Only the MI values for each field were different.

So I'm assuming they are calculated differently,

Again, thank you for your time, 

Regards,
Bernardo

[BernardoAlb]

And sorry for the confusion with the duplicate post, I will be answering in this one 

[cmorfin]

Hi Bernardo

Is it possible for you to upload :

- your dataset csv

- your datase json

- goal name

- model configuration

- exclusion list

-filter definition

- Version of ThingWorx

- Version of ThingWorx Analytics

- Version of ThingWorx Analytics extension

We can try to make some test here.

Remember though that data uploaded here are publicly available, so if this is not ok for you, I would advise to open a case to Technical Support so this can be taken offline.

It could also be worth creating the model with validation holdout set to 0 in the Advanced Configuration parameters. Keeping exclusion list, filter and goal identical with the top signal to compare.
I am not sure but it is possible that the signal done at model level are made on the trained data only and not the full dataset, The difference between full dataset and trained data is the validation holdout, so setting it to 0 will ensure we use the same 2 dataset.

Kind regards

Christophe

[BernardoAlb]

Hi Christophe,

I cannot upload the data as it is confidential. 

I actually had that suspicion about the validation, but I tried it and it is not possible to use a validation holdout of 0 as 1% is the minimum accepted value.

It doesn't also appear to make a difference on the signal if I use a holdout of 1 or 20. 

Either way, thank you very much for your time,

Regards,
Bernardo

[cmorfin]

Hi Bernardo

In this case I can only advise to open a new case to Technical Support so we can look at it.

Thanks

Christophe