How to transfer incremented file edge to Thingworx ??

[jinkim2]

Is there any way to transfer file only incremented size ??

Below is the code that I used to transfer files from edge to Thingworx.

Really what I wanted is only transfer the incremented

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
var params = {
path: "/In" /* STRING */,
nameMask: undefined /* STRING */
};

// result: INFOTABLE dataShape: "FileSystemFile"
var deviceFileList = me.ListFiles(params);

for(var x = 0; x < deviceFileList.getRowCount(); x++ ){
    logger.error("deviceFileList :"+ deviceFileList.getRow(x).name + " " + deviceFileList.getRow(x).size);   
}

//2. Thingworx Server
var params = {
path: "/"+me.name /* STRING */,
nameMask: undefined /* STRING */
};

var twxFileList = Things["SystemRepository"].GetFileListing(params);

for(var x = 0; x < twxFileList.getRowCount(); x++ ){
    logger.error("twxFileList :"+ twxFileList.getRow(x).name + " " + twxFileList.getRow(x).size);
   
}

var params = {
infoTableName : "InfoTable",
dataShapeName : "FileSystemFile"
};

// CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(FileSystemFile)
var list = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params);
var listToRemove = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params);

//3. file check
for(var j = 0; j < twxFileList.getRowCount(); j++ ){
for(var k = 0; k < deviceFileList.getRowCount(); k++ ){
       
        var name1 = ""+twxFileList.getRow(j).name + "";
        var name2 =  ""+deviceFileList.getRow(k).name + "";
        if(name1 == name2){
            if(twxFileList.getRow(j).size != deviceFileList.getRow(k).size){
               
               logger.error("????twxFileList.getRow(j)$$$$$$$ : " + twxFileList.getRow(j).name+" : "+twxFileList.getRow(j).size + "  ///////  "
             + "????list.getRow(k)$$$$$$$ : " +deviceFileList.getRow(k).name+" : "+ deviceFileList.getRow(k).size);
                list.AddRow(deviceFileList.getRow(k));
            }else{
             listToRemove.AddRow(deviceFileList.getRow(k));
            }
        }
       
    }
}
for(var z = 0; z < listToRemove.getRowCount(); z++ ){
deviceFileList.Delete(listToRemove.getRow(z));
}

for(var x = 0; x < deviceFileList.getRowCount(); x++ ){
    logger.error("deviceFileList :"+ deviceFileList.getRow(x).name + " " + deviceFileList.getRow(x).size);
    list.AddRow(deviceFileList.getRow(x));
}

for(var x = 0; x < list.getRowCount(); x++ ){
    logger.error("list :"+ list.getRow(x).name + " " + list.getRow(x).size);
   
}
var fileList =  list;

//2. Copy To Thingworx
for(var i=0; i < fileList.length;i++){
   
   // logger.error(">>>>>>>> fileList.getRow(i).name : " + fileList.getRow(i).name);
   
    var params = {
        async: true /* BOOLEAN */,
        sourceRepo: me.name /* STRING */,
        metadata: undefined /* JSON */,
        targetRepo: "SystemRepository" /* STRING */,
        targetFile: fileList.getRow(i).name /* STRING */,
        targetPath: "/"+me.name /* STRING */,
        queueable: true /* BOOLEAN */,
        sourceFile: fileList.getRow(i).name /* STRING */,
        sourcePath: "/In" /* STRING */,
        timeout: undefined /* INTEGER */
    };

    // result: INFOTABLE dataShape: FileTransferJob
    var result = Subsystems["FileTransferSubsystem"].Copy(params);

}

[ttielebein]

Hello, so it looks like from your code, you want to update the file on the Platform with whatever changes were made on the remote device, but you don't want to override what is already in the file on the Platform, is that correct?

You would need to transfer the new edits under a different file name, load the content of both the old file and the new one from the repository, concatenate the data manually, and then delete the temp file from the repository. Something like this should do it:

var pathOnRepo = "/";

var tempFileName = "tmp_"+file_in;

var params = {

  async: false,

  sourceRepo: me.name,

  sourcePath: "/in/",

    sourceFile: file_in,

    metadata: undefined,

  targetRepo: "SystemRepository",

  targetPath: pathOnRepo,

    targetFile: tempFileName,

  queueable: false,

  timeout: 12000

};

var result = Subsystems["FileTransferSubsystem"].Copy(params);

var original_content = Things["SystemRepository"].LoadText({path: pathOnRepo+file_in});

var temp_content = Things["SystemRepository"].LoadText({path: pathOnRepo+tempFileName});

var new_content = original_content+"/n"+temp_content;

var params = {

  path: pathOnRepo+file_in,

  data: new_content,

    offset: 0

};

Things["SystemRepository"].WriteToTextFile(params);

Things["SystemRepository"].DeleteFile({path: pathOnRepo+tempFileName});

Hope this helps!

[YangJaeUng]

Hello,

It looks like very useful code for the upper problem.

Well, I have a quetion for this code.

Is it work correctly when the file size is big? (eg. 1 GByte over)

I'm sorry that I didn't test by myself yet.

I just want to know your experience.

It will be very useful for me.

Thanks in advance.

[ttielebein]

I have never tested this with a large file, but I assume it should work the same, so long as the server hosting the Platform and the one hosting the Edge device have enough memory to do the file transfer

[AnnaAn]

Hi,

a case is created for this issue and will discuss it offline.

Thanks,

Br,

Anna

[AnnaAn]

Hi Jae Ung,

Please check the link below as per we discussed as one alternative to implement your requirement

Upload big file to ThingWorx server periodically with lower bandwidth

Thanks,

Br,

Anna