How to generate huge reports ?

[rbeck]

Hello,

I have to generate a report from Thingworx 7.3.

Data are coming from multiple databases.

I'm directly connected to these through jdbc.

The report needs to be downloadable and in CSV format. I have managed this part.

However, the report contains about 3 millions of lines and takes hours to be generated.

What would be the best way to do this?

Is there any possibility of background batching?

Is there a lock mechanism that I can use (if a report is being generated, the user should not ask a new generation)?

Or a simple property on a thing is enough?

Thank you

[PaiChung · Accepted solution]

If you just need to create a CSV download, I would create a service that writes that file into a FileRepository.

Don't send that data to the Client side browser through the data export widget.

So steps would be

1. call for the data

2. combine data as necessary

3. use the WriteCSV to a filerepository

You can do this all Server side and then do a mashup that shows files in the File Repository (FileListingWithLinks) to download the file client side.