How to extract model data of 3d models in Vuforia Studio (without external Tools)?

[#1]

1.) The first point  here is to clarify : is it possible to extract model data of 3d models in Vuforia Studio?  ( data could be extracted by Creo View Toolkit apps but here is considered only the Vuforia Studio environment)

Supposing , we have a model widget for an assembly model without explicit modelitem  widget definitions. The question is: Can we extract data for the components and if yes,  then what data we can extract?

In Vuforia Studio Environment Extracting of data is possible only in Preview mode, because we have in preview mode the method tml3dRenderer.GetObject() where we can access a model object (a component) example:

let comp_widget=tml3dRenderer.GetObject(selection).GetWidget()

where the selection is some thing like "<modelname>-<compPath>" e.g. : "model-1-/0/0/3/2"

Then from the widget we can extract data:

 var loc=tml3dRenderer.GetObject(selection).GetWidget().GetLocation()
 console.error("DEBUG getObj.GetWidget()")
 console.warn(tml3dRenderer.GetObject(selection).GetWidget())

When we   explore  the different methods in the crome debugging console,  we will find methods to  get or  to set  properties. To extract data, we can use the get... methods.

The methods of  tml3dRenderer.GetObject() seems currently not to work in Vuforia View on end devices (the tml3dRenderer object is a handle of the cordova vuforia plug in and it has a different implementation on the different end devices.In preview mode so far I know, the graphic is based on WebGL and Three.js) 

Therefore we will be not able for example to get the data of a component selection on the end device. So means we need a way to extract data in Preview mode and make it available in the Vuforia view on the end device.  Here I did not find a methods to extract the original component name but I was able to create a list (json) with the position data ( I did not add color but this is possible to access it - e.g. tml3dRenderer.GetObject(selection).GetWidget().GetColor())

We can create a json  e.g. of  following data:

{"model-1-/0/0/0":{"valid":false,"orientation":{"x":0,"y":0,"z":0},
"size":{"x":1,"y":1,"z":1},"scale":{"x":1,"y":1,"z":1},
"position":{"x":9.999999998199587e-24,"y":9.999999998199587e-24,"z":9.999999998199587e-24}},
"model-1-/0/0":{"valid":false,"orientation":{"x":0,"y":0,"z":0},"size":{"x":1,"y":1,"z":1},
"scale":{"x":1,"y":1,"z":1},"position":{"x":0,"y":0,"z":0}},
"model-1-/0/0/2":{"valid":false,"orientation":{"x":0,"y":90,"z":0},"size":{"x":1,"y":1,"z":1},
"scale":{"x":1,"y":1,"z":1},"position":{"x":0,"y":0.029500000178813934,"z":-5.51091050576101e-18}},
 ...}

we can  assign the json to a variable e.g. $scope.COMP_LOCs

So later we can read the current position data on end device:

var selection_location=$scope.COMP_LOCs[l_currentSelection]  
//read the location data from json varible
console.log("selection:"+l_currentSelection+"->X= "+ selection_location.position.x);
//print it to console
selection_location.position.x= round(parseFloat(selection_location.position.x) + 0.005,4)
//add 0.005 shift and round to 4 dec

2.)In point 1.)  we checked how to  extract the data of an compoent (a selection) .But Actually  we have a couple of methods to extract the data but what we do not have is a valid  selection of an assembly  component . This is required to obtain a valid modelitem widget (temporar) via tml3dRenderer.GetObject(). For the selection generation we have the model widget name e.g. “model-1” but   we do not have the component ID paths. To be able to construct a selection handle we need to construct the ID path of a component and then we need to check if it exist. This is some kind of graph search where we have an assembly with a components tree.  There the edges are the ids of the components.

e.g. /0/0/1/1 , /0/0/1/2, /0/0/1/4, … etc.

One possible algorithm is the deep first search:

To implement this I used the following javaScript code:

/////////////////////////////
var max_asm_depth=6; //this is the max depth in Creo Parametric
var max_numb_comp_asm=25;

///////////////////////////  ->deep first
 function check_comp_deep_first_recursively(target,path,arr) {
 //console.warn("called  check_comp_deep_first_recursively(target="+target+",path="+path+")");  
 
var selection     =  target+'-'+path
var path_array    =  path.split('/')
var depth         =  parseInt(path_array.length)     
var num           =  parseInt(path_array[depth -1])
var prev_num      =  parseInt(path_array[depth -2])
var prev_path     =  ''
for (var i=1;i < depth -1;i++)
{prev_path= prev_path +'/' + path_array[i]}

 if( check_for_valid_selection(selection) == 1)
 {
  arr[selection]=tml3dRenderer.GetObject(selection).GetWidget().GetLocation()
   if( (depth+1) < max_asm_depth)
   check_comp_deep_first_recursively(target,  path + '/0', arr)
   else  
   {   
   if(num +1 < max_numb_comp_asm)
      check_comp_deep_first_recursively(target, prev_path + '/'+(num +1), arr)}
     
 }
 else
 {
   var right_num = num +1
   if(right_num < max_numb_comp_asm)
      check_comp_deep_first_recursively(target, prev_path + '/'+right_num, arr)
   else
    if(!Number.isNaN(prev_num) )
   {//console.log("--2")
   	prev_path     =  ''
	for (var i=1;i < depth -2;i++)
	{prev_path = prev_path +'/' + path_array[i]}
   
     prev_path = prev_path +'/' +  (prev_num +1)
      check_comp_deep_first_recursively(target, prev_path , arr)
   }
 }
 }
//////////////////////////
///call of the function:
$scope.compJSON_loc_Data = new Object();
var target="model-1"
check_comp_deep_first_recursively(target,'/0',$scope.compJSON_loc_Data)
...

The code above has the following weak spot - I need to give the maximum depth (max_asm_depth) and the maximum possible branches (max_numb_comp_asm) 

The maximum depth currently in Creo assembly is 25 so that value which > 25 will not make a sense. 

The value of  max_numb_comp_asm  in a flat assembly (only one level of depth) corresponds to the number of the components - the maximum number of branches on particular level of depth

The another possible algorithm is the breadth first search:

To implement this  I used the following JavaScript code:

/////////////////////////////
var max_asm_depth=6; //this is the max depth in Creo Parametric
var max_numb_comp_asm=25;

/////////////////function check_comp_at_level(target,num,depth,arr)
//  ->breadth first
 function check_comp_at_level(selection,num,depth,arr) {
   var position ='';
     
  // console.log("call check_comp_at_level =>"+selection);
   try{
    // console.log("====== check here ===========");
     //console.warn(tml3dRenderer.GetObject(selection).GetWidget().GetLocation());
     var loc=tml3dRenderer.GetObject(selection).GetWidget().GetLocation()
     if( (loc.scale.x == 0) || (loc.scale.y == 0)  || (loc.scale.z == 0) )
        return 0; // the scale could not be zero
     //position= tml3dRenderer.GetObject(selection).GetWidget().GetLocation().position
     //console.warn(position);
     //arr[selection]=position
      arr[selection]=loc
     return arr[selection];
    }
  catch (e) {console.error("failsed with error="+e); return 0;}
 }

///////////////////////////
 function check_comp_at_level_recursively(selection,depth,arr) {
 //console.warn("called check_comp_at_level_recursively("+selection+",depth="+depth+")");  
 var num =0;
 if(depth >= max_asm_depth) {
 //console.log("maximum depth of max_asm_depth ="+max_asm_depth+" reached");
   return 0;}
 for (num=0;num < max_numb_comp_asm; num++) {
    var currentSelection =selection+'/'+num
 
 if(depth <0) return 0;
 var pos = check_comp_at_level(currentSelection,num,depth,arr)
 if(pos ==0 ) { continue;}
  else 
  {check_comp_at_level_recursively(currentSelection,(depth+1),arr)
  
  }
 } //end of for
   
 }
//////////////////////////

////////////////////////////////

 function check_for_valid_selection(selection) {   
   //console.log(" check_for_valid_selection =>"+selection);
   try{
 
     var loc=tml3dRenderer.GetObject(selection).GetWidget().GetLocation()

     if( (loc.scale.x == 0) || (loc.scale.y == 0)  || (loc.scale.z == 0) ) return 0;
     return 1;
    }
  catch (e) {console.error("failsed with error="+e); return 0;}
 }

///////////////////////////
///call of the function:
$scope.compJSON_loc_Data = new Object();
var target="model-1"
check_comp_at_level_recursively(target,'/0',$scope.compJSON_loc_Data)
...

The code for the breadth first search uses also the parameters for maximum depth (max_asm_depth) and the maximum possible branches (max_numb_comp_asm)  - so means it have the mentioned  restriction. If we set a value which is large this will increase the time until the search is completed so therefore depending of the particular assembly we need to set the both parameter properly ( we need to be able to scan the whole assembly but to minimize the search time)

For different assemblies the first deep or first breadth could lead to better results. For example, for flat assembly structures the better approach will be to use the first breadth algorithm 

But actually the performance is not so important here, because the search will be called one time and  then the json list should be saved. 

With the current functionality we can read a file (json file ) from the project  upload directory , but it seems that it is  not  possible to save the information to a e.g. json file there (upload folder).

To read a json file form the upload folder we can use some code like this:

target='model-1'
$http.get('app/resources/Uploaded/' + jsonFile).success(function(data, status, headers, config) {
    $scope.compJSON_mod=data;
// in this case the data is the received json object 
angular.forEach(data , function(color, path_id){ 
$scope.compJSON_Data[path_id] =position;
 console.log("target="+target+" --> $scope.compJSON_Data["+path_id+"] = "+$scope.compJSON_Data[path_id]);
});//end of the error function ////////// finish for each path_id
  })
  .error(function(data, status, headers, config) {console.log("problem in the http will create a new ");
                                                  

When we want to save data  (the generated json list) we need to use another workaround - we can use a thingworx repository. Following functions /events could be used to save and receive an json object to/from a twx repository:

// the methods SaveJSON and LoadJSON
// for the repository object should have 
//run permision for  es-public-access user
//////////////////////////////////////////////////////////////
$scope.SaveJsonToTwxRepository = function(path, content) {
 $scope.$applyAsync(function() {
    $rootScope.$broadcast('app.mdl.CAD-Files-Repository.svc.SaveJSON',
                          {"content": content, "path":path}

                         );}   ,500 );

};
//////////////////////////////////////////////////////////////
$scope.GetJsonFromTwxRepository = function(path) {
 $scope.$applyAsync(function() {
    
    $rootScope.$broadcast('app.mdl.CAD-Files-Repository.svc.LoadJSON',
      
                          {"path":path}

                         );}   ,500 );
   $scope.app.speak("after call of GetJsonFromTwxRepository")

//in the modelloaded listener register
// LoadJSON-complete event -> to laod the data into session
rootScope.$on('modelLoaded', function() {
////
 $scope.$root.$on('LoadJSON-complete', function(event, args) { 
console.log("LoadJSON-complete event");
$scope.COMP_LOCs=args.data
console.log(JSON.stringify( $scope.COMP_LOCs))
});
///
});

In  the code above I use the 'modelloaded' listener to register LoadJSON-complete event . Because the service is called asyncronously- we need this event to load the data into session when it is received from thingworx.

Here in this example the repository object is named "CAD-Files-Repository"

The Thingworx services should have run permission and it is required to be added in the external data panel :

So when we start the project in PREVIEW mode we can call the search for the assembly structure and save it then  to thingworx. In Vuforia View mode   then we can receive the previously saved json object from thingworx.

To check the current mode (if Preview or End Device)  we can use 

if(twx.app.isPreview() == true)
...

it will  check if the current mode is preview mode or Vuforia View on the end device - here an example of the workflow:

if(twx.app.isPreview() == true) 
{// preview mode  
//calling   breadth first - test
check_comp_at_level_recursively(target+'-',0,$scope.compJSON_POS_Data)  
//console.warn($scope.compJSON_POS_Data)
  //calling   deep first a second test and generating a data - locations
check_comp_deep_first_recursively(target,'/0',$scope.compJSON_loc_Data)
console.log("========================================")
console.log("$scope.compJSON_POS_Data ->breadth first")
console.log("========================================")
console.log(JSON.stringify($scope.compJSON_POS_Data))
console.log("========================================")
console.log("")    
console.log("")
console.log("========================================")
console.log("$scope.compJSON_loc_Data ->deep first")
console.log("========================================")
console.log(JSON.stringify($scope.compJSON_loc_Data))
$scope.SaveJsonToTwxRepository('/CADFiles/json_lists/compJSON_loc_Data.json',$scope.compJSON_loc_Data)
$scope.GetJsonFromTwxRepository('/CADFiles/json_lists/compJSON_loc_Data.json')
console.log("========================================")
console.log("")   
 }
 else { //here is the part on mobile device
$scope.GetJsonFromTwxRepository('/CADFiles/json_lists/compJSON_loc_Data.json')    
                } 

I tested all points of  the described techniques above in a  test project which I want to provide here as zip file for the HoloLens (hideComponetsHoloLens .zip):

So to be able to test it you need to create in Thingworx a repository thing - means a thing which uses  the thing template "FileRepositroy" with the name "CAD-Files-Repository" and create a folder there "/CADFiles/json_lists/" (if you use another name and another folder (e.g. "/" no folder - the root repository folder) you have to adapt the javaScript code:

...
/CADFiles/json_lists/compJSON_loc_Data.json
...
app.mdl.CAD-Files-Repository.svc.SaveJSON'
...
app.mdl.CAD-Files-Repository.svc.LoadJSON'