Issue for starting 3D Video from JS

[mol22]

Hi all,

I am currently developing a step-by-step instruction for HoloLens 2 that should play a video when the next step is triggered. For this, I have written some JS code that switches the step (containing animations and switch in the step description). The issue that is comming up when I run the code below is that the status of the video is constantly switching from play to pause and backwards.

A minimal snippet of the code I use: 

getCurrentTitleLookUp = function (stepNumber) {
  //initialize variables
  switch (stepNumber) {
    case 1:
      //Changing labels and instructions
   //Setting the video source
  $scope.setWidgetProp("3DVideo-1", "src", "app/resources/Uploaded/Videos/Step-1.mp4");
    //Play the video 
   twx.app.fn.triggerWidgetService("3DVideo-1", "play");

      //play animations

      break;
//goes on for all steps. 
}


//That method is called on a button-click. 
$scope.app.nextStep = function () {
  stepNumber = stepNumber + 1;
 
  getCurrentTitleLookUp(stepNumber);
// here is some logic for assemby
 
}

The following output gives me the log constantly:

3DVideo-1 userpick play button video3D-ng.js:150

pause 3DVideo-1 video3D-ng.js:218

3DVideo-1 userpick play button video3D-ng.js:139

play 3DVideo-1 video3D-ng.js:206

3DVideo-1 userpick play buttonvideo3D-ng.js:150 

I used the following snipped for replaying the video, also on a button-click. There it works perfectly.

twx.app.fn.triggerWidgetService("3DVideo-1", "play");

Does anybody have an idea what I am doing wrong? I ran out of ideas, especially because it's working with the replay, but I am not sure what I am doing differently. 

Thanks in advance!

Marius 

[RolandRaytchev · Accepted solution]

Hello @mol22 ,

l

looking in  code of you post but unfortunately for me is not clear what  is the issue.   You mentioned it works perfectly but I could not see what is not working. Could you explain, please, more detailed, possibly the context where your code is called.

In generally when you set a property and then trigger a service which used this property – then this is possibly it is not working - you can try the subsequent call it after async or with some delay 

  //Setting the video source
  $scope.setWidgetProp("3DVideo-1", "src", "app/resources/Uploaded/Videos/Step-1.mp4");
// async call
$scope.$applyAsync();

//Play the video  with delay
 $timeout(() =>{ twx.app.fn.triggerWidgetService("3DVideo-1", "play");},1000);
//e.g. 1 second delay

you mentioned also "instruction for HoloLens 2 that should play a video when the next step is triggered" :

so how did you triggered the next step - do you use some listener e.g. new Step, step completed or step finished (as described in the post about Vuforia Studio Events) 

So far, I see you used the simples scenario via click on a  button to call the next step - which means here it should perform first a  setting of src property  then will play( widget play service)  the video file for the next step. I think this should so far work in the normal case

Another code version trying to play the code could be :

$scope.testPlay2= () =>{
var element = angular.element(document.querySelector('twx-widget[widget-id=\"3DVideo-1\"]'));
element.scope().play();
};

… Here is also a list of different attempt  which will have more or less success to call the 3DVideo widget play service – I found them in  some old projects but did not test if they all will work

/////////////////////////
 try{app.view["Home"].wdg["3DVideo-1"].svc.play;} catch(ex){console.warn("ex="+ex);}
/////////////////////////
try{twx.app.fn.triggerStudioEvent('3DVideo-1', 'play');} catch(ex){console.warn("ex="+ex);}
try{$scope.$root.$broadcast('app.view["Home"].wdg["3DVideo-1"] .svc.play');} catch(ex){console.warn("ex="+ex);}
 try{$scope.$root.$broadcast('app.view["Home"].wdg["3DVideo-1"] .svc.play');} catch(ex){console.warn("ex="+ex);}
    // this below is working when button 3DButton-2 has binding to the play event
try{twx.app.fn.triggerStudioEvent(document.querySelector('[widget-id="3DButton-2"]'), 'click');} catch(ex){console.warn("ex="+ex);}
/////////////////////////
try{twx.app.fn.triggerStudioEvent(document.querySelector('[widget-id="3DVideo-1-play"]'), 'click');} catch(ex){console.warn("ex="+ex);}
/////////////////////////
try{twx.app.fn.triggerStudioEvent(document.querySelector('[widget-id="3DVideo-1"]'), 'click');} catch(ex){console.warn("ex="+ex);}
try{twx.app.fn.triggerStudioEvent('3DVideo-1-play', 'click');} catch(ex){console.warn("ex="+ex);}
/////////////////////////
     try{
    $scope.app.fn.triggerWidgetService("3DVideo-1","play");
     } catch(ex){console.warn("ex="+ex);}

[mol22]

Hi @RolandRaytchev, 

thanks for your response! Your first suggestion did the trick. It appears that I had to wait till the Video is loaded. I even need to wait more than one second till I am able to play it. Otherwise it's not starting.

Thanks again!

Marius

[RolandRaytchev]

Hi @mol22 ,

thanks for the feedback and  I am hapy to hear that it was working in your case. But when we use a  solution like this:

/////////////////////////
$scope.testJsPlay= function() {
  $scope.setWidgetProp("3DVideo-1", "src", "app/resources/Uploaded/sonne1.mp4")
  $scope.$applyAsync(); 
  //Play the video with delay
  $timeout(() =>{
       try{
    $scope.app.fn.triggerWidgetService("3DVideo-1","play");
     } catch(ex){console.warn("ex="+ex);} 
               }
    ,6000); //e.g. 6 second delay !!! possibly to long
 
};

we need to ensure that we will delay enough long time  (here in the example 6 seconds - which is very long) so that the play service  call will work. If we use a video with large size and call it from some  location where the connection is slow ... possibly it will not work. So here is an attempt to have a workarounbd for this issue with fetch request (the url should be http or https which CORS conform) 

//in one button this function on click -local large file
app.testPlay('app/resources/Uploaded/20201103_173339_HoloLens.mp4','3DVideo-1');
//this def for second button to play test video form http url
 app.testPlay('http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerJoyrides.mp4','3DVideo-1');

Here I have the function defintion what I used. In my test it was working fine on HL2. It will not play in preview mode - work only on device (according to the Studio Help.- this widget does not work in Preview- error some message like "no shader defined") Therefore to  test also in preview mode if the data is received I have here some test printings which should be removed on the HL2 - because it is then to slow

$scope.app.testPlay= function(url,widgetName) {
  $scope.wdgName=widgetName
  console.log("---->widgetName="+widgetName)
  $scope.setWidgetProp($scope.wdgName, "src", url)
  $scope.$applyAsync();
  // here I will load the file to estimate the time
  console.log("before fetch("+url+") time ::" +$scope.getTime());
  fetch(url)
  .then(response => response.blob())
  .then(blob  => {
    var reader = new FileReader()
     reader.onload = function(){ console.log("onload time ::" +$scope.getTime());  
      	var dataURL = reader.result;
      	 $scope.app.view["Home"].wdg[ $scope.wdgName].src='data:video/mp4;base64,'+ dataURL;
         $scope.$applyAsync();
         // this print below will print the data -but is not efficient on device !
           console.warn( $scope.app.view["Home"].wdg[ $scope.wdgName].src)                    
                               };//end of onload funciton def
     //load end
     reader.onloadend = () => {
       console.log("onloadend time ::" +$scope.getTime());
        $timeout(() =>{
        $scope.app.fn.triggerWidgetService( $scope.wdgName,"play");  
        },1500) //1,5 sec delay to sync
                              } //onload end
     
    reader.readAsDataURL(blob)
  })

 }
//================ print time for check
//===================================================================
$scope.getTime = function() {
var date = new Date();
var h = date.getHours(); // 0 - 23
var m = date.getMinutes(); // 0 - 59
var s = date.getSeconds(); // 0 - 59
var session = "AM";
if(h == 0){h = 12;}
if(h > 12){h = h - 12;session = "PM";}
h = (h < 10) ? "0" + h : h;
m = (m < 10) ? "0" + m : m;
s = (s < 10) ? "0" + s : s;
var time = h + ":" + m + ":" + s + " " + session;
return time;
}
//===============================================================================================

[mol22]

Great @RolandRaytchev, thanks a lot! I'll try that code as well, especially the bad connection could happen I suppose. 

[RolandRaytchev]

wanted here to add an example. Not the best but showing some different tests using the 3D Video widget. Please pay attention, that it works 100% only on HL2 device and not in preview. Tested it with the previous version 9.5 and it was working . Not tested with 9.7 yet.