Community Tip - Did you get an answer that solved your problem? Please mark it as an Accepted Solution so others with the same problem can find the answer easily. X
Question: What should I know about using ThingWorx with InfluxDB to store my time series data?
Hi, ThingWorx users!
It’s here! Thanks for waiting patiently since my previous post announcing ThingWorx’ new support of InfluxDB as a time series persistence provider.
As of our 8.4 release, you can now use InfluxDB to store your ThingWorx time series data with incredible power and ease.
Want to learn more? Check out the following FAQs:
1. What is InfluxDB? Who is InfluxData?
InfluxDB is a time series database designed to handle high write and query loads. It is meant to be used as a backing store for any use case involving large amounts of timestamped data, like monitoring, application metrics, IoT sensor data, and real-time analytics that you’d find in ThingWorx.
InfluxDB is created by InfluxData, an awesome company that we are proud to call a PTC partner.
2. When would I want to use InfluxDB for IIoT?
While the ThingWorx IIoT platform supports multiple databases to persist IIoT data and is agnostic when it comes to the storage layer, InfluxDB is the ideal choice for time series. When the number of connected devices increases, along with the amount of streaming data, the need to have a high-scale telemetry database choice is obvious.
For very high scale data ingestion, InfluxDB should be used as a persistent provider with the ThingWorx platform for multiple reasons. Its flexibility and ease of use provides native support for standard time series functions, including: sampling, interpolation, time bucketing, aggregation, selector, transformation, predictor, etc. It does all of this while supporting a high compression of data (~45x) with the ability to handle thousands of writes per second and read thousands of rows in milliseconds.
Check out this article by our Enterprise Deployment Center (EDC) explaining why InfluxDB is great for small ThingWorx applications.
3. What are the three different flavors of InfluxDB?
InfluxDB Open Source (TICK Stack), InfluxDB Enterprise & InfluxDB Cloud. Here’s more info on each:
To learn more about the different modules of InfluxDB (Telegraf, Chronograf, Kapcitor), check out InfluxData Introduction for documentation or InfluxData Products for product info.
4. What is the difference between InfluxDB opensource and enterprise?
5. Where can I find the pricing overview for buying enterprise licenses for InfluxDB?
The PTC product and go-to-market team have defined commercial pricing for InfluxDB Enterprise. For help with pricing, reach out to Chris Wensley (cwensley@ptc.com) and Anders Hinrichsen (anders@influxdata.com).
6. How do I configure InfluxDB with ThingWorx?
We’ve outlined the steps for you in the ThingWorx Help Center and created a quick video to instruct you on how to install InfluxDB with ThingWorx.
To see the current version of InfluxDB that we support, read our ThingWorx 9.0 System Requirements guide.
7. How do I configure InfluxDB and ThingWorx in a high availability scenario?
With the ability to leverage multiple data stores, we work to provide the flexibility to best meet the needs of your IT preferences and investments. InfluxDB helps us do that. To configure ThingWorx for High Availability, please refer to this section of the ThingWorx Platform 9 Help Center. To configure InfluxDB for High Availability at the database level, please refer to InfluxData’s documentation on how to Install and deploy InfluxDB Enterprise clusters.
8. Where can I learn more about how to monitor and manage InfluxDB?
Monitoring info for InfluxDB can be found here: Monitoring Tools for TICK Stack.
9. How can I tune and optimize InfluxDB with ThingWorx?
The best approach for running InfluxDB with PTC ThingWorx 8.4 (or later) is to treat the workload and configuration just as you would in a stand-alone deployment. We suggest to stick to the recommendations in the InfluxDB and TICK stack documentation.
10. How do I perform backup and recovery of ThingWorx with InfluxDB?
Please see the ThingWorx Platform Backup and Recovery Planning Technical Brief to plan for back and recovery. You can also find more more details on taking backups and restoring data from InfluxDB in the Backing up and restoring in InfluxDB Enterprise overview.
11. Where can I learn more about sampling, interpolation, time bucketing, aggregation, pivot and other key features of InfluxDB?
Features of InfluxDB can be found here: InfluxData Time Series Platform.
Implementation of InfluxDB features can be found here: Getting Started with InfluxDB.
12. What are all the different persistence providers supported with ThingWorx? When should I use InfluxDB?
Please refer to the model and data best practices section of the ThingWorx 9 Help Center for further information on options how to store your model and data with ThingWorx.
We have also updated the ThingWorx Platform 9.0 Sizing Guide to provide relevant information to estimate the amount of processing and memory that ThingWorx may need to meet your requirements. It also provides guidance on when to use InfluxDB for your scale needs.
13. When should I use InfluxDB over DataStax Enterprise (DSE)?
Here is a good blog post that benchmarks time series data performance of InfluxDB vs. Cassandra, which is the core of DataStax Enterprise (DSE). In specific use cases, InfluxData Enterprise may be more cost effective when compared to similar telemetry use cases with DSE.
14. How can I migrate my data from PostgreSQL to InfluxDB?
Migration from PostgreSQL or MSSQL is supported by the ThingWorx in-built data tools, which can export entities and data from PostgreSQL or MSSQL and then import them into InfluxDB.
Details on how to upgrade to ThingWorx 9.0 can be found in the Upgrading ThingWorx section of the ThingWorx 9 Help Center.
15. Should I use InfluxDB as a time series store rather than OSI PI, IP21, or others?
For ThingWorx 8.4 and later, InfluxDB is the recommended time series store. This can be implemented at the edge with ThingWorx (i.e. “front end”) using the open source edition and can also be implemented at the hub (i.e. “back end”) using either of the commercial editions designed for HA production workloads.
As always, ThingWorx can connect to most industrial software, including OSI PI, IP21, etc. with our integration toolset.
That’s a wrap—almost! We’ve added two extra questions for you.
16. What’s on the roadmap for ThingWorx with InfluxDB?
17. What should I do if I need technical support with InfluxDB?
If you’re as excited as I am about the ability to store your time series data with InfluxDB, let me know in the comments below!
Until next time, if you have any questions, just ask Kaya!
Great FAQ - thank you!
I really like the compression, interpolation, retention policies, and down-sampling features. These are really great ways to be able to capture and use time series data with less concern on having data bloat over the years and struggling to work with and backup such large volumes of data.
So how does a ThingWorx developer manage and configure these sorts of functionality; as I'm sure that it won't be from within ThingWorx?
I am also wondering how ThingWorx works with the InfluxDB for cases where InfluxDB is already in place and being used by a number of applications?
How is difference between influxdata and MSSQL or postgres ?
Hi @ssriuaipornchai ,
I just spoke with Ayush, our database expert. Here's what he had to say:
"ThingWorx includes several persistence provider implementations that enable you to choose a database option that best fits your use case. ThingWorx has the ability to leverage multiple data stores, which provides immense flexibility that best meets your IT preferences and investments. Here is our ThingWorx Model and Data Best Practices Guide to help you decide which persistence provider to choose: Microsoft SQL, AzureSQL, PostgreSQL or InfluxDB for your production IoT data storage needs."
Stay connected,
Kaya
Influx is built and optimised for high-speed, high-scale, long-term time-series data storage, retrieval, and management. This is not the case with any RDBMS.
Hi. This is great news. But is it not in all versions of ThingWorx? Doesn't seem to be in Foundation.
What version of ThingWorx is required to see the Influxdb persistency provider? It doesn't seem to be in the foundation version from https://developer.thingworx.com/en/resources/downloads
Also, the "ThingWorx Free Trial for Manufacturing" download seems broken.
InfluxDB is included with all Commercial Versions of ThingWorx. It could be that the Trial version has not this connector.
InfluxDB has two versions. One community free edition and a commercial edition. ThingWorx run with both editions. I have my ThingWorx setup on docker with a docker image of the community Edition of InfluxDB. The connector is included since Version 8.4. There are step-by-step instructions on the PTC support page on how to get InfluxDB connected. If your ThingWorx version support InfluxDB you can see under the PersistanceProviders. There should be an entry for InfluxDB.
@StefanBode wrote: MyPrepaidCenter
InfluxDB has two versions. One community free edition and a commercial edition. ThingWorx run with both editions. I have my ThingWorx setup on docker with a docker image of the community Edition of InfluxDB. The connector is included since Version 8.4. There are step-by-step instructions on the PTC support page on how to get InfluxDB connected. If your ThingWorx version support InfluxDB you can see under the PersistanceProviders. There should be an entry for InfluxDB.
ThingWorx includes several persistence provider implementations that enable you to choose a database option that best fits your use case. ThingWorx has the ability to leverage multiple data stores, which provides immense flexibility that best meets your IT preferences and investments.
I'm also looking for clarity on how Thingworx manages Influx downsampling techniques and property history retrieval for longer term retention policies.
Hi @Kaya ,
If I Choose the Influx DB from Azure Marketplace, and set it up, will the Time series data be stored in Azure Cloud or in he influx Cloud ?
Thanks,
Shashi.
ThingWorx connects to InfluxDB via the standard interface (line protocol over HTTP/S) so it can be either of the options you mentioned. In the case that it is the Azure Marketplace InfluxDB Enterprise, this is a self-hosted deployment so the customer would host/manage the environment. Note that this would run InfluxDB 1.X. In the InfluxDB Cloud hosted offering it is InfluxData who hosts and manages it, and it is InfluxDB 2.X.
Hi @geva ,
Thanks for the info, but the end Client requirement is "To Store data only in Azure" not again in any third party instances. So If I Use Influx from Marketplace of azure and setup InfluxDB, will the data store in the Azure or will it be with Influx ?
I know that, Influx only uses the Azure AKS and VM's to setup the Instance, but will the data be stored in Azure ?
Thanks,
Shashi.
If you manage your own instance and you have our own InfluxDB it is hosted where you installed your DB. This is 100% your decision.
If you are operating ThingWorx with PTC, then PTC uses InfluxDB 2.0 SaaS services. We do not host any system, but using the shared Instance of InfluxDB. As with all SaaS services there is no guarantee or information what cloud systems are used. Sometimes it is also a mixture of different systems because as a SaaS company you do not care about Azure/AWS and so on, but you care about the SaaS Services you get. Sometimes AWS is great and on another Service Azure. Even if InfluxDB now is telling you they operate on AWS this can change tomorrow.
Hi @StefanBode ,
Thanks for the reply, but More confused now, if we setup INFLUX with Azure Marketplace will data resides with Influx Cloud or with Azure ?
Thanks,
Shashi.
The Azure Marketplace is for Microsoft partners who provide applications or solutions which run on Azure; so yes the data would reside in Azure in either of the cases. You can read the details and follow the links on the InfluxDB Cloud for Azure page on the marketplace.
Hi @geva ,
Thanks a lot for the confirmation.
Thanks,
Shashi.
You are quite welcome!
Greg EVA (he/him)
Enterprise Architecture Community Lead, Solutions Consulting
E-mail: geva@ptc.com
Office: +33 4 72 52 59 50
Mobile: +33 6 67 50 35 77
ptc.com