How to switch faster between sequences on the HoloLens.

[armink]

Dear Sir or Madam,
We are currently working on our diploma thesis in co-operation with
the school and a company. For our thesis, we are creating assembly
instructions in Creo Illustrate to train a worker. These instructions
are displayed to the worker via the HoloLens.

We control our assembly instructions in Vuforia Studio with parameters
that we receive from a Siemens controller via KEPServerEX.

To better explain our problem, you need to know that the yellow button
is a button for confirming a completed work step. This will play the
next work step.

Our current challenge is that the next assembly step should be played
on the HoloLens immediately after confirmation. In the preview, this
works perfectly in that the assembly step is played immediately after
confirmation. With the HoloLens, however, you often have to press the
confirm button twice so that the desired sequence is played. In
addition, the entire model is briefly displayed when he jumps from one
sequence to the next; there may also be a solution for this so that he
can switch better between sequences.

We have attached a video and our Vuforia Studio file to better
illustrate our problem.

In the Vuforia Studio file you will find our codes that are
responsible for calling the sequence. My guess is that there may be
other variants of the syntax so that the sequence is called "faster"
or "better". I hope you have a solution for us.

Code:

$scope.$watch("app.params['Zelleneinbau_4_VS']", function() {
if ($scope.app.params.Zelleneinbau_4_VS == 'true') {

$scope.view.wdg['model-1'].sequence = "app/resources/Uploaded/l-Creo 3D - Zelleneinbau_5F4.pvi";
$timeout(function() {
twx.app.fn.triggerWidgetService("model-1", 'playAll');
}, 500);

if (lang == 'de') {
$scope.app.view['Home'].wdg['3DLabel-Textanzeige']['text'] = "Akkuzelle an richtiger Position einbauen!";
} else {
$scope.app.view['Home'].wdg['3DLabel-Textanzeige']['text'] = "Install the battery cell in the correct position!";
}
var soundFilePath = "app/resources/Uploaded/11_Akkuzelle_Position_" + lang + ".mp3";
var audio = new Audio(soundFilePath);
$timeout(function() {
audio.play();
}, 500);

}
});

You can look up the complete code in the Vuforia Studio file!

With best regards!

[RolandRaytchev]

Hi @armink ,

the issue on HoloLens 2 is related possibly to the load time of the sequence . When you have sequence with  a large size it will possibly will not load until you try to play. Because each called action is called asynchronously and the next action will start without waiting that the previous action is completed. 

For this there are some possible actions

1.) trivial one is to call the next action with some delay large enought that you can be sure that the previous action will completed . This approach is not recommended becaue depend on size, and performance on device

2.) to use sequenceLoad. So when you load a new sequence then call the play first in the sequenceLoad event  as described in the topic  https://community.ptc.com/t5/Vuforia-Studio/Sequence-animations-on-hololens/m-p/702186#M9074 (but here is relevant the example attached there that shows how to use the sequenceLoad event in this context. Also there are discussed some points. The solution itself is not relevant in this case)

3.) try to use some construct form javascript to synchronize the next call - so this will be started first when the previous call was completed - something like javascript promise concept:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise

https://www.w3schools.com/js/js_promise.asp

4.) additional point is the usage the old techniques on HL2  to unzip the pvi file directly to the Upload folder from the pvz file. 

Please, let me know if this helps

[armink]

Hi Roland,

I would like to thank you for your quick reply. 

1. First of all, I tried to expand the delay so there is enough time for the sequence to start, which unfortunately did not work.

2. I have tried the following codes which did not work for me:

a. Sequence loaded:

$scope.view.wdg["model-1"].sequence = "app/resources/Uploaded/l-Creo 3D - UE_5FKiste_5FDeckel_5Fli.pvi";

}

$scope.$on('sequenceloaded', function(evt, arg1) {

    angular.element(document.getElementById('model-1')).scope().playAll();

});

b. Code from the other Community question:

$scope.$watch("app.params['UE_Kiste_Deckel_li_VS']", function() {     

  if ($scope.app.params.UE_Kiste_Deckel_li_VS == 'true') {

$timeout(function () {

  $scope.$applyAsync(()=>{$scope.setWidgetProp("app/resources/Uploaded/l-Creo 3D - UE_5FKiste_5FDeckel_5Fli.pvi");});     

  },50);

  $timeout(function () { 

        //$scope.$applyAsync(()=>{$scope.setWidgetProp(modelName, 'currentStep', parseInt(step_number));});

if(playAll)   //check if play should be applyed

  $timeout(function () {

  //replace this

  //angular.element(document.getElementById(modelName)).scope().play();  

  //to this one

  $scope.app.fn.triggerWidgetService("model-1", 'playAll');

  }, 100)

    //angular.element(document.getElementById(modelName)).scope().play(); }, 100);

                      }

            , 500);

      $scope.$applyAsync();

}

3. It would be really helpful if you could display me the JavaScript promise concept in an example file. We are active in mechanical engineering and are not the best programmers... so we hope that you can help us.

4. We are actually using the "old technique" you mentioned. What about the new technique? How does it work? Are there any examples? 

I still have the problem which can be seen above in the description and in the video, that I have to repeat the step so that the sequence is played and the problem that the whole model is sometimes visible between the steps.

I hope there are some solutions for this problem, maybe you can display it in an example. Thank you in advance!

Kind regards

[RolandRaytchev]

Hi @armink ,

I checked your points /questions - to this what to mention the following:

- the old techniques was only a workaround to extract the file ,because in old versions it does not worked - simple - it required to extract the pvi files outside the pvz to work. In the mean time acctually you do not need to extract because it is fixed- it should work (that is  the "new" techniques - means no bug and following the documentation - PTC Help for Vuforia Studio)

- to the point that you mentioned that you  are mechanical engineer (me too e.g. I study Werkzeugmaschinen u. Fertigungsmitteln)- and therefore is difficult- yes It could be, but should not be a problem to develop the skills. So far I undstood (you are student working on master/diploma and this is the best opportunity in this work to learn such  concepts  like - Javascript Angular and general CAD structure (componetns ids path etc) any othere concepts) so for example myself started learning Vuforia concept 6 years ago  started with javascript from scratch, therfore possibly there possibly I have some knowledge  gabs at some area 😉

So regarding your quesitos . I will provide here an example where I did made the concept what I mentioned and wanted to tested. I used your orginal project  data and some TWX names and service  described in the your another topic  reported.  This proejct  you set as resolved. Therefore I did not follow this further. But because some of the issues there appered   again I will provide here my test example with your data -just as study to see what possible -what I wanted originally to provide to the topic . So now I compled it and hope it could be helpful for you.

Because this eample was based on your data you should be able to addapt to your enviroment without to much changes. You need actualy only addapt the TWX external data add all enties there as shonw here. - Of course to work it requries the names of the TWX entietes  to be correct. Anyways I used the names what are already there so that the most of them should be correctly

And here  provide the modifed project the example attached.So hope this helps. Thanks

[RolandRaytchev]

also here copied the TWX entities what I used for test - exported as entities xml

[armink]

Dear Mr Raytchev,
First of all, thank you very much for your efforts. The example you sent is very interesting and cool. However, there must be easier solutions to solve this problem. Unfortunately, I don't have the time for now to analyse your file in detail and rebuild it, as I have to submit my diploma thesis soon. This is the code I currently use to access my pvi files: Are there any solutions to adapt this code to solve my problem?

//Überprüfung Kiste Deckel links
$scope.$watch("app.params['UE_Kiste_Deckel_li_VS']", function() {
  if ($scope.app.params.UE_Kiste_Deckel_li_VS == 'true') {
    
    console.log($scope.view.wdg['model-1'].sequence)
    $scope.view.wdg['model-1'].sequence = "app/resources/Uploaded/l-Creo%203D%20-%20UE_5FKiste_5FDeckel_5Fli.pvi";
	$timeout( function() {
      twx.app.fn.triggerWidgetService("model-1", 'playAll');
    }
             , 3000);
  }
})

What exactly could be adapted in this specific code here so that the animations are played immediately?
There must be a "better" or "simpler" syntax so that the timer does not have to be turned up to 3 seconds, for example, so that the sequences are all displayed on the HoloLens immediately after confirmation.

Another question, what does the  "$scope.$applyAsync();" actually do?

I have uploaded my latest file too.
Kind regards

[RolandRaytchev]

Hi @armink , ok understand,np.

So to $scope.$applyAsync() is something, what is called to ensure that setting of properties will have affect - and we could see the  e.g. the display change accoring to the new setting , because the most call are asynconously and thereofre when we  call this functio to be sure that the change are done - so something like this, at least  it have this effect. The concept there is much complexer - see angular $apply

But we do not need to know this for the work with JS in studio.

To the code - I think you can split you call in 2 things  - what I accutually did in the sample project , but ok simple

- set the sequence. set a global variable (e.g. $scope.app.myPlay=true) define a sequneceload event and play there the sequence:

$scope.app.play =false
//Überprüfung Kiste Deckel links
$scope.$watch("app.params['UE_Kiste_Deckel_li_VS']", function() {
  if ($scope.app.params.UE_Kiste_Deckel_li_VS == 'true') { 
    console.log($scope.view.wdg['model-1'].sequence)
   $scope.setWidgetProp('model-1','sequence','app/resources/Uploaded/l-Creo%203D%20-%20UE_5FKiste_5FDeckel_5Fli.pvi');
// this above will load the sequence I do not think that you
//need necessarily url encoding but it is still on the save side to use
$scope.$applyAsync();	$scope.app.play =true;
  }
})
//========================0
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
$scope.$on("sequenceloaded", function (evt, arg) {
 console.log("++++sequenceloaded++++")
if ($scope.app.play==true)  {
        $timeout(function () {  $scope.app.fn.triggerWidgetService( 'model-1',"playAll");}, 50)                   
  		$scope.$applyAsync();
        $scope.app.play =false } //set back play to false
});

I copied this from a test which is much complexer but this is simplest way to do this. I did not test it but should work if I do not have any typos. but let me know if this work then. Thanks

[armink]

Dear Mr. Raytchev,
Thank you for the quick reply, I have tried your code and it works conditionally. The sequence is played only once when I get a true from the ThingWorx platform. However, I need to ensure that the sequence is repeatable for the worker who is currently operating the system. This means that he can view the sequence as often as he wants by pressing the button..
I have tried this with your code but with a few additions:

$scope.app.play =false
$scope.$watch("app.params['Akkuzelleneinbau_1']", function() {
  if ($scope.app.params.Akkuzelleneinbau_1 == 'true') {
    $timeout(function () {
      $scope.app.fn.triggerWidgetService( 'model-1',"reset");//sollte eine Sequenz noch laufen und man drückt den Button, muss diese zuvor resettet werden
    }
             , 50)  	
    $scope.$applyAsync();
    console.log($scope.view.wdg['model-1'].sequence)
    $scope.setWidgetProp('model-1','sequence','app/resources/Uploaded/Versuchsaufbau/l-Creo%203D%20-%20Zelleneinbau1.pvi');
    $scope.$applyAsync(); $scope.app.play =true;
    $scope.$broadcast("sequenceloaded"); //dadurch wird das Ereignis "sequenceloaded" jedesmal ausgelöst - sonst kann die Sequenz nicht wiederholt werden
  }
}
             )
$scope.$on("sequenceloaded", function (evt, arg) {
  console.log("++++sequenceloaded++++")
  if ($scope.app.play==true)  {
    $timeout(function () {
      $scope.app.fn.triggerWidgetService( 'model-1',"playAll");
    }
             , 50)                   
    $scope.$applyAsync();
    $scope.app.play =false }
}
          );

I added the reset event so that whenever the button is pressed, the sequence is played from the beginning. In the preview this works perfectly.
The challenge I have now on the HoloLens2 is that when the true parameter arrives, the sequence loads for a short time but won't play itself. For example, when I get the true parameter a second time (I pressed the confirm button a second time), it works and I can see the sequence being played, which means the sequence got loaded. This could possibly be due to the sequence loaded event, I hope you can help me with this issue and tell me what exactly could be added or changed in the code so that the sequence is played the first time the true parameter arrives?

Hope to hear from you soon.

Kind regards

[RolandRaytchev]

the code I provided what splitted in two part to insure that when a sequence is set to modelwidget  then it will play so for the case that  i it requires a long time to load .

But one time sequence is set on Modelwidget it could be play many  time you start the play service. So when the code is called - /when I mean code it is only the part I provided h / and you have a 3D button which has a binding to the play service of the widget e.g. 'model-1' this always should play -so simple button binding to modelwidget  without any codes only  

[armink]

Dear Mr Raytchev,
I think there has been a misunderstanding, we do not use buttons to play our sequences. Our sequences are played depending on the parameters that deliver a true or a false from ThingWorx. As I mentioned, my issue is that the sequence is not played by the first time when it should have been loaded. To explain it in more detail: The PLC sends a true or false, depending on how the sensors switch, to the KepServerEX. This transfers the data to our ThingWorx platform. I then receive my true or false in Vuforia Studio via ThingWorx, which I use to play the sequence. However, I can trigger this true again with the yellow Confirm button so that the sequence is played again. And that's the challenge I have, how to get the sequence to play the first time a true is received.

Kind regards