How to load large data set and display in the graph? Looking for options.
We need your input urgently.
In the context of our Predictive Analytics project with McKinsey, we are trying to figure what is the best technical approach with ThingWorx to solve the problem illustrated in the diagram below (and also slide attached):
1)We have a series of csv files which we call “Spectrum Image Files”. Each file contains a snapshot at a given time of a certain set of data. Think about it as a “frame” in a movie
2)Each file / frame is a .csv file which contains the frequency Spectrum Analysis of the acceleration/vibration signals along the 3 axis of the spindle of a CNC machine.
That means that the file has 4 columns: Frequency, Frequency Amplitude for X axis acceleration, Frequency Amplitude for Y axis acceleration, Frequency Amplitude for Z axis acceleration
Each row is a different frequency, from 1HZ to 50,000HZ (so 50,000 rows in the table)
3)Now, we want to graph in a mashup, each of the values of the AmplX, AmplY, AmplZ columns in the table. Each graph is basically a simple bar graph of the frequency amplitudes (vertical axis of the chart) by frequency (horizontal axis). The result is “frequency spectrum” of the acceleration signal for each axis of the CNC machine spindle.
4)Assuming that each Spectrum File is generated every 5 seconds (the series of files mentioned in 1), we want the mashup to update every 5 seconds with a new “frame”.
What’s the best way to solve this? (processing / loading / storing of the data, widget for the graph to use, …)