Question: What should I know about using ThingWorx with InfluxDB to store my time series data?
Hi, ThingWorx users!
It’s here! Thanks for waiting patiently since my previous post announcing ThingWorx’ new support of InfluxDB as a time series persistence provider.
As of our 8.4 release, you can now use InfluxDB to store your ThingWorx time series data with incredible power and ease.
Want to learn more? Check out the following FAQs:
1. What is InfluxDB? Who is InfluxData?
InfluxDB is a time series database designed to handle high write and query loads. It is meant to be used as a backing store for any use case involving large amounts of timestamped data, like monitoring, application metrics, IoT sensor data, and real-time analytics that you’d find in ThingWorx.
InfluxDB is created by InfluxData, an awesome company that we are proud to call a PTC partner.
2. When would I want to use InfluxDB for IIoT?
While the ThingWorx IIoT platform supports multiple databases to persist IIoT data and is agnostic when it comes to the storage layer, InfluxDB is the ideal choice for time series. When the number of connected devices increases, along with the amount of streaming data, the need to have a high-scale telemetry database choice is obvious.
For very high scale data ingestion, InfluxDB should be used as a persistent provider with the ThingWorx platform for multiple reasons. Its flexibility and ease of use provides native support for standard time series functions, including: sampling, interpolation, time bucketing, aggregation, selector, transformation, predictor, etc. It does all of this while supporting a high compression of data (~45x) with the ability to handle thousands of writes per second and read thousands of rows in milliseconds.
3. What are the three different flavors of InfluxDB?
InfluxDB Open Source (TICK Stack), InfluxDB Enterprise & InfluxDB Cloud. Here’s more info on each:
4. What is the difference between InfluxDB opensource and enterprise?
5. Where can I find the pricing overview for buying enterprise licenses for InfluxDB?
The PTC product and go-to-market team have defined commercial pricing for InfluxDB Enterprise. For help with pricing, reach out to Andrei Parjol (firstname.lastname@example.org).
6. How do I configure InfluxDB with ThingWorx?
We’ve outlined the steps for you in the ThingWorx Help Center and created a quick video to instruct you on how to install InfluxDB with ThingWorx.
To see the current version of InfluxDB that we support, read our ThingWorx 8.4 System Requirements guide.
7. How do I configure InfluxDB and ThingWorx in a high availability scenario?
With the ability to leverage multiple data stores, we work to provide the flexibility to best meet the needs of your IT preferences and investments. InfluxDB helps us do that. To configure ThingWorx for High Availability, please refer to the updated ThingWorx 8 High Availability Administrator's Guide. To configure InfluxDB for High Availability at the database level, please refer to InfluxData’s documentation on how to Install and deploy InfluxDB Enterprise clusters.
8. Where can I learn more about how to monitor and manage InfluxDB?
Monitoring info for InfluxDB can be found here: Monitoring Tools for TICK Stack.
9. How can I tune and optimize InfluxDB with ThingWorx?
The best approach for running InfluxDB with PTC ThingWorx 8.4 (or later) is to treat the workload and configuration just as you would in a stand-alone deployment. We suggest to stick to the recommendations in the InfluxDB and TICK stack documentation.
10. How do I perform backup and recovery of ThingWorx with InfluxDB?
Please see the ThingWorx Platform Backup and Recovery Planning Technical Brief to plan for back and recovery. You can also find more more details on taking backups and restoring data from InfluxDB in the Backing up and restoring in InfluxDB Enterprise overview.
11. Where can I learn more about sampling, interpolation, time bucketing, aggregation, pivot and other key features of InfluxDB?
Features of InfluxDB can be found here: InfluxData Time Series Platform.
Implementation of InfluxDB features can be found here: Getting Started with InfluxDB.
12. What are all the different persistence providers supported with ThingWorx? When should I use InfluxDB?
Please refer to the ThingWorx Model and Data Best Practices guide for further information on options how to store your model and data with ThingWorx.
We have also updated the ThingWorx Platform 8.x Sizing Guide to provide relevant information to estimate the amount of processing and memory that ThingWorx may need to meet your requirements. It also provides guidance on when to use InfluxDB for your scale needs.
13. When should I use InfluxDB over DataStax Enterprise (DSE)?
Here is a good blog post that benchmarks time series data performance of InfluxDB vs. Cassandra, which is the core of DataStax Enterprise (DSE). In specific use cases, InfluxData Enterprise may be more cost effective when compared to similar telemetry use cases with DSE.
14. How can I migrate my data from PostgreSQL to InfluxDB?
Migration from PostgreSQL or MSSQL is supported by the ThingWorx in-built data tools, which can export entities and data from PostgreSQL or MSSQL and then import them into InfluxDB.
Details on how to upgrade to ThingWorx 8.4 can be found in our upgrade guide.
15. Should I use InfluxDB as a time series store rather than OSI PI, IP21, or others?
For ThingWorx 8.4 and later, InfluxDB is the recommended time series store. This can be implemented at the edge with ThingWorx (i.e. “front end”) using the open source edition and can also be implemented at the hub (i.e. “back end”) using either of the commercial editions designed for HA production workloads.
As always, ThingWorx can connect to most industrial software, including OSI PI, IP21, etc. with our integration toolset.
That’s a wrap—almost! We’ve added two extra questions for you.
16. What’s on the roadmap for ThingWorx with InfluxDB?
17. What should I do if I need technical support with InfluxDB?
If you’re as excited as I am about the ability to store your time series data with InfluxDB, let me know in the comments below!
Until next time, if you have any questions, just ask Kaya!
Great FAQ - thank you!
I really like the compression, interpolation, retention policies, and down-sampling features. These are really great ways to be able to capture and use time series data with less concern on having data bloat over the years and struggling to work with and backup such large volumes of data.
So how does a ThingWorx developer manage and configure these sorts of functionality; as I'm sure that it won't be from within ThingWorx?
I am also wondering how ThingWorx works with the InfluxDB for cases where InfluxDB is already in place and being used by a number of applications?
Hi @ssriuaipornchai ,
I just spoke with Ayush, our database expert. Here's what he had to say:
"ThingWorx includes several persistence provider implementations that enable you to choose a database option that best fits your use case. ThingWorx has the ability to leverage multiple data stores, which provides immense flexibility that best meets your IT preferences and investments. Here is our ThingWorx Model and Data Best Practices Guide to help you decide which persistence provider to choose: Microsoft SQL, AzureSQL, PostgreSQL or InfluxDB for your production IoT data storage needs."
Influx is built and optimised for high-speed, high-scale, long-term time-series data storage, retrieval, and management. This is not the case with any RDBMS.
Hi. This is great news. But is it not in all versions of ThingWorx? Doesn't seem to be in Foundation.
What version of ThingWorx is required to see the Influxdb persistency provider? It doesn't seem to be in the foundation version from https://developer.thingworx.com/en/resources/downloads
Also, the "ThingWorx Free Trial for Manufacturing" download seems broken.
InfluxDB is included with all Commercial Versions of ThingWorx. It could be that the Trial version has not this connector.