Hadoop Extension for Thingworx

[bikramp]

Hi

I have a need to store large amount of data. Since there is a limitation of 15 mill records in streams, could you please advise an alternative to store the data?

Does Thingworx provide hadoop extension?

Regards

Saurabh

[adam11]

Hi Saurabh,

We do not provide a Hadoop extension, but it would be possible for you to develop your own. What makes you think there is a limit on the # of records for

Streams

?

Regards,

Adam

[bikramp]

Hi Adam

So does that mean that a stream can keep unlimited number of records? Is there any upper limit to it? Also, how does the data processing take place if at all streams can handle trillions of record.

I am talking about ~ 3,744,000,000 records per day.

Does this amount of data slow down the overall processing of thingworx?

Regards

Saurabh

[bikramp]

Hi

Can anyone help me in this regard?

I am receiving data approximately ~ 3,744,000,000 records per day.

And I need to store data for atleast 1 year.

Please advise.

Regards

Saurabh Singhal

[adam11]

Saurabh,

I've reached out to development to get some recommendations about how to best store this data.