Real-time Thingworx Analytics and using Power BI web tools in Azure

[rad1]

It seems 3 Tomcat service needs to run for Analytics to work:

• Thingworx itself
• Analytics microservices
• DataConnect  (which requires installation of Apache Spark 1.4.0 with Hadoop 2.4)

This picture shows the process.

Is Thingworx Analytics real-time or batch based? Base on above diagram there seems to be polling for data.

I would like to see how to achieve something like Microsoft Azure ML to provide predictive analysis.

The data needed for ML will come from eg from HDInsight Spart Streaming

and I would like to use Power BI to do some visualization by connecting to HDInsight Spark and using DirectQuery

Spark BI using data visualization tools on Azure HDInsight | Microsoft Docs

I was wondering how I can achieve near real-time analytics like Microsoft Azure does and how Power BI
web tools can access Apeche Spark DB.

Is it possible to use DataConnect with HDInsight Spark instance in Azure?

If possible and if databases are compatible how to link Thingworx and Analytics server with HDInsight Spark instance in Azure?

If Power BI can reach DataConnect's  Apache Spark I am OK with that.

So the overall goal is to do real-time analytics using Thingworx Analytics and somehow visualize data using Power BI web tools.

Can I embed Power BI widgets/dashboards into Thingworx's mashups?

Thanks,

Rad

[cmorfin]

Hi Rad

I try to answer some of your questions here.

The diagram you posted for the architecture of ThingWorx > DataConnect > ThingWorx Analytics is typically used to collect historic data from ThingWorx to send to ThingWorx analytics in order to build a predictive model. As such this is not real time.

To perform real time scoring you will use ThingPredictor that will then take in new values and predict the goal in near real time . See this article for video on seting up and using ThignPredictor Article Viewer | PTC . Help Center wil lalso be very valuable : https://support.ptc.com/help/thingworx_hc/thingworx_analytics_8/#page/thingworx_analytics_8%2Fthingpredictor%2Fthingpredictor-overview.html%23

In term of connecting DataConnect or Analytics to an external Spark, I don't think this will be possible. Spark is a component used but it does not do all the job, Analytics adds its own twist to it. This means also that connecting to ThingWorx Analytics Spark won't be useful (not sure if it is possible ) either since it does not contains the results of ThingWorx Analytics.

I can't comment on using PowerBI, maybe others might have some input on this.

Hope this helps

Kind regards

Christophe