cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - Visit the PTCooler (the community lounge) to get to know your fellow community members and check out some of Dale's Friday Humor posts! X

How to avoid data loss when adding multiple stream entries

TanmeyTWX
17-Peridot

How to avoid data loss when adding multiple stream entries

Hi @Willie 

Were you able to find any solution for this?

I have come across similar issue where multiple entries are being updated simultaneously and UpdateStreamEntry leaks few of the entries.

 

Note: My Stream datashape has String,Number and Infotable type columns.

1 ACCEPTED SOLUTION

Accepted Solutions

Hi Sathish,

 

I have segregated the nested infotables into separate infotables and using only AddStreamEntry and UpdateStreamEntry is not required anymore which actually was causing issue. Hence the issue is resolved.

View solution in original post

4 REPLIES 4

Hi @TanmeyTWX 

The use case was scrapped so I did not need the stream anymore.  According to @abhiramk , you can have the same timestamp in a stream if the source is different, but in my case, I had the same source so I was adding random milliseconds.  It might be possible to make the milliseconds less random if you have a property on the stream to act as a counter for milliseconds that resets to 0 after 999, but I haven't tried it so I don't know if it would work.  The threads might (probably) access and increment the property simultaneously.  As @nmilleson mentioned, maybe you could have a staging area for your data before adding them to a stream so you could ensure they are added.  Stage the data then change the timestamp one by one before adding.

Willie
16-Pearl
(To:Willie)

Hi @TanmeyTWX 

I asked ChatGPT.   Here's the response:

Thingworx - 2023-02-10 - ChatGPT response to duplicate stream entry issue3.PNG

I think a combination of #1 and #2/#4 could work.

  • How many records do you have in same timestamp? As you mentioned, I am passing a line of text as a string property and parsing the string using a service I wrote to create stream entries.  
  • What is the data format? How you are parsing it? Is it a JSON?
  • How frequently data change is happening?

Hi Sathish,

 

I have segregated the nested infotables into separate infotables and using only AddStreamEntry and UpdateStreamEntry is not required anymore which actually was causing issue. Hence the issue is resolved.

Top Tags