thingworx hadoop connectivity

[Tejasvi]

Hi, does thingworx support hadoop? is the connection possible? 

[mgoel]

@Tejasvi

Thank you for opening this thread. If we talk about storage then ThingWorx does not support Hadoop for Storage. Could you please see below threads to get some idea:

https://community.ptc.com/t5/ThingWorx-Developers/Does-ThingWrox-provide-any-predefined-mechanism-to-migrate-data/m-p/532022

https://community.ptc.com/t5/ThingWorx-Developers/How-to-transfer-data-from-ThingWorx-to-Hadoop/m-p/505563

Do let us know in case of any question.

Regards-Mohit

[supandey]

Hi @Tejasvi are you looking to connect HDFS to ThingWorx? As Mohit has already noted that currently there are no OOTB configuration available, however  using something like JDBC Connector Extension available for download from PTC Marketplace  you may try connecting to Hadoop from ThingWorx using SQL e.g. Apache Hive, Oracle Big Data SQL, etc.

[Tejasvi]

Hi Supandey,

We have actually used the JDBC connector extension, we also used the thingworx extension SDK available in the market place to create an extension for thing template to establish connection b/w thingworx and hadoop, but the connection is not getting established.

We are getting the isConnected property to be false in thingworx, though we are able to see in the hive log that the connection is established.

Also in thingworx we are able to see the table name and column values in the table/column tab. But whenever we write a custom service, we get the error message 'unable to invoke the method'. Generic services are getting executed. 

Provided screenshots for the same. 

[supandey]

Hi @Tejasvi what sort of SQL command are you using? The error you have shown seems to be related to the syntax and not about the connectivity.

Additionally, what kind of user are using to execute those services?

[smainente]

Hi Tejasvi,

The Hive JDBC Driver supports a limited subset of the JDBC API : if you look at the source of org\apache\hive\jdbc\HiveConnection.java, about 40 methods are throwing a "Method not supported" SQLException.

Apparently the ThingWorx QueryHandler in 8.2 uses transactions and this is not supported by the Hive JDBC driver. I informed R&D (PSPT-5132) but I don't know yet if it can be a candidate for enhancement.