cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - You can change your system assigned username to something more personal in your community settings. X

Stream AddStreamEntry overwrites the data

gabitudor
10-Marble

Stream AddStreamEntry overwrites the data

hello, i am trying to cache some data, so: i have a service that has its output as an InfoTable and retrieves a couple of rows and 5 columns of data. i would like to cache this service as it takes some time to run, but the below code only caches the last row form the server even thought the loop is ok and reads all the rows. (that s why i aded the ii object, for debugging purposes) in monitoring i could not find anything. i also wanted to try the AddStreamEntries service but i got stuck with its implementation

 

can you please help?

 

var values;

var params = {
	infoTableName : "InfoTable",
	dataShapeName : "GabiDS"
};

// CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(AMTable_ModuleDDS_CRA_PG)
var table = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params);

    var tags = new Array();
    var timestamp = new Date();
		
	// result: INFOTABLE dataShape: AMTable_ModuleDDS_CRA_PG
	var actions = Things["GabiThing"].GetData();

    if (actions) {
        // loop through infotable
        if(Things["GabiStream"]){
            var params = {
                endDate: timestamp /* DATETIME */,
                immediate: true /* BOOLEAN */,
                startDate: undefined /* DATETIME */
            };
            // no return
            Things["GabiStream"].PurgeStreamEntries(params);
        }

        var tableLength = actions.rows.length;

        for (var x = 0; x < tableLength; x++) {
            var row = actions[x];
            values = Things["GabiStream"].CreateValues();

            try {
                values.x = row.x; //INTEGER
            } catch (err) {
                logger.error('x is null');
            }
            try {
                values.y = row.y; //INTEGER
            } catch (err) {
                logger.error('y is null');
            }
            try {
                values.z = row.z; //INTEGER
            } catch (err) {
                logger.error('z is null');
            }
            try {
                values.t = row.t; //INTEGER
            } catch (err) {
                logger.error('t is null');
            }

            var ii = new Object();
            ii.x = row.x;
            ii.y = row.y;
            ii.z = row.z;
            ii.t = row.t;
      
            table.AddRow(ii);
            
            var location = new Object();
            location.latitude = 0;
            location.longitude = 0;
            location.elevation = 0;
            location.units ="WGS84";
            
        
         var params = {
            tags : tags,
            timestamp : timestamp,
            source : me.name,
            values : values,
            location : location
        };    
        Things["GabiStream"].AddStreamEntry(params);
        }
    }

var result = table;
ACCEPTED SOLUTION

Accepted Solutions
jwasney
7-Bedrock
(To:ague)

I ran across this the other day.  I was only getting 1/20th of my data in the stream due to the timestamp.  To fix this I set timestamp = new Date(Date.now() + (i * 10));  That ensured there were no duplicate timestamps.

View solution in original post

10 REPLIES 10
ague
13-Aquamarine
(To:gabitudor)

Does the 'table' for which you are add a new row ('ii', in your example) get created properly?

 

I'm confused because I see you creating this 'values' variable, and then setting the values in your loop (ex: values.t = row.t), but then after you set them, I don't see you doing anything further with the 'values' variable. 

 

Am I missing something?

gabitudor
10-Marble
(To:ague)

That table is only created for the debugging purposes, to see if in for loop takes all the data from GetData service. But "values" is basically the row that is added in the stream, if you look closer
ague
13-Aquamarine
(To:gabitudor)

Okay, I see. Then the only other two things I can think of are this: 

 

1. I would make sure your timestamps aren't identical in any of the entries. I believe that if you and add entries to a steam that have identical time stamps and locations (and/or source  -  you'd have to look that up, not sure off the top of my head), I believe they'll begin to write over each other. 

 

2. I might check the way you're adding 'values' in the params - can you feed an object just like that? Does it have to be a certain datashape or something?

 

That's where I'd start looking into! 

 

gabitudor
10-Marble
(To:ague)

Thanks for the reply!

1. Unfortunately does not work with different location or timestamp.

2. The snippet is that way, it gives you values to fill based on your service's output (its datashape), location and parameters that you saw (tags, location, values, source and timestamp). Once i modify something the service AddStreamEntry is not going to work anymore

 

ague
13-Aquamarine
(To:gabitudor)

Okay, so regarding your point 1, in your response:

 

So you're saying that all your entries DO have identical timestamps? Then that is most likely why you're only ever able to add one entry. Try arbitrarily changing the location and/or source (just for testing purposes) and see if it then fixes the issue. 

gabitudor
10-Marble
(To:ague)

that's what i am saying, i tried. timestamp is basically the date when the entry is added to the stream, so i do not think it can be the same anyway as milliseconds are different. But, with different location, tags, timestamp etc. the stream looks the same, only one entry in it, the last one

ague
13-Aquamarine
(To:gabitudor)

But aren't you only creating your 'timestamp' variable once, outside of your loop? That won't change your time - it'll be the same and therefore the stream will keep overwriting. 

 

Try something like setting the seconds in the timestamp to be the value of 'x' in the loop - that way you ensure that your timestamp is unique in each entry.

jwasney
7-Bedrock
(To:ague)

I ran across this the other day.  I was only getting 1/20th of my data in the stream due to the timestamp.  To fix this I set timestamp = new Date(Date.now() + (i * 10));  That ensured there were no duplicate timestamps.

ague
13-Aquamarine
(To:jwasney)

Yup! I'm pretty confident that this is the Original Poster's issue :) 

gabitudor
10-Marble
(To:ague)

Thanks guys, indeed that solved the problem. It is pretty strange, because before i also tried using the pause snippet, but that did not work. Thank you so much to both of you!

Announcements


Top Tags