cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - Need to share some code when posting a question or reply? Make sure to use the "Insert code sample" menu option. Learn more! X

About HoloLens 2 voice command Alternatives

Sandy_Gunner
14-Alexandrite

About HoloLens 2 voice command Alternatives

Dear all, 

As of now Vuforia provides only the voice commands for the operation of the HoloLens. Using HoloLens experience inside the factory is very difficult because of the continuous loud noise within the factory premises. We are thinking to add buttons and perform the desired operation like scan, show library, show download etc.. from within the HoloLens experience.

Till now , we are familiar with the Only API which will switch the Vuforia view app to scan mode. app.fn.navigateToScanMode();

Do someone know the API to switch the Vuforia View to the Vuforia library as well as to the downloaded experiences ?  Thank you in advance.

4 REPLIES 4

Hi @Sandy_Gunner ,

I checked the reported issue to PTC R&D and see that related issue were reported and R&D is aware about this requirements but after deep check - there are currently some plans for improvement and  development - but they are not classified as urgent  because according to R&D team there are currently sufficient techniques to work around with loud enviroment. 

2 strategies' to improve the current behavior in the current release:

>> R&D statement about how to use better the voice command:

There is no specific improvements that can be made for the HL voice command recognition. Some further information:

  • The log shows low recognition quality on some of the basic key phrases, this could be an indicator of several things: Unusual pronunciation by the user, noisy environment that causes trouble for the microphone, or a faulty device.
  • Custom voice commands might have lower confidence than the standard key phrases so it might help to test out variants that fit the user pronunciation and are easier to recognize for the HL.
  • In general the voice recognition Microsoft provides on the HL is not on the same level as the current services from the Google Assistant or Apple’s Siri. This might lead to a higher expectation than the device can deliver.

>> Design ;

  • so when you start on HL2 , then the view app will go by default to scan mode - scan thingmark or QR code to open a project
  • the project have a start View where you have buttons to navigate -you can use the mentioned api:  
    • app.fn.navigateToScanMode()  to navigate back to scan mode
    • $scope.app.fn.navigate(View_name)  to navigate between views e.g. : 
      //navigate to Home view
      $scope.app.fn.navigate('Home');
      // navigate to View named CHAR2DVIEW
      $scope.app.fn.navigate('CHAR2DVIEW')​
    • window.open(url) e.g to open an experience:
      $scope.openUrl=function()
      {
      // let url= "https://www.bing.com/"
      let url="https://xxxxxxo.studio-trial.thingworx.io/ExperienceService/content/projects/bla/index-desktop.html?expIndex=0#/Home"
       window.open(url)
      // some related points
      // window.inf.allowIntent(url) 
      // window.location = url
      }​
  • Here also statement from the R&D team to the last issue reported this year "Discovery and Development of voiceless command alternatives for HoloLens"  - some point are already implemented further are under dev and possibly will come next.
  • Description:
    As a second step in offering alternative ways for navigating in the View app using HoloLens, the task is to couple the system navigation logic with buttons (until now it was solely voice commands). It is still a partial integration, meaning further improvement with long-range plan
  • Scope of dev
    Based on the developments from reported Ticket w: Investigate Microphone setup for Voiceless commands
    , the process should cover further steps:

        //I think this is planed improvement below I did not check if this is already part of the current 9.15 functionality /did not tested yet

           Attach customers  expected functions to the created buttons:

             Library - Opening Library and its content

             Downloads - Showing experiences that got downloaded to the device from the Library

  •          NOT Scope
    Within above sections, all Experiences should be presented as buttons, that upon being clicked, open the experience. When opening the experience, hide the selector menu Within the experience, show a button “Close” which leads back to the originator menu (Library / Downloads) serving as a “Back” button - not required
  • Implementing a floating panel - this is already down
  • Implementing a hand gesture as main trigger
  • Final UI design
  • Idea: Use a floating panel for the buttons 

 

Logic:
Step 0: Deactivate Microphone (or make sure that it is deactivated)

Step 1: Open View App

Step 2: Scan specific QR Code that triggers the functionality

Step 3: Once you close the app, functionality is automatically disabled

 

 

Hello @RolandRaytchev ,

 

Thank you for your prompt investigation. 

I forgot to mention about the offline use of the HoloLens inside the factory earlier. 

To use the API mentioned above, I think the network connection is mandatory which is not always guaranteed inside the factory.

$scope.openUrl=function()
{
let url="https://xxxxxxo.studio-trial.thingworx.io/ExperienceService/content/projects/bla/index-desktop.html?expIndex=0#/Home"
window.open(url);
}

 

Do the support team planning to provide the API to go to the library as well as in download similar as to go to the scan mode API or they will provide built-in UI to switch between scan mode, library and download for the Vuforia View application for HoloLens same as in Vuforia View for Ipad ? 

Hello @Sandy_Gunner , thanks for your feedback!

Yes calling the Vuforia Studio experience via link requires a network connection.

But... I further checked in the R&D reported tickets and found  further information about voiceless usage - menu. So I was not sure if this is already implemented yet or still in development therefore I tested it by myself. So remove Vuforia view app and installed it again on HL2- so my HL has an issue so that this is the way to have an update. Now having the latest version I could confirmed that the option suggested /developed by PTC R&D team is working in the current version (Hey View , version ) 99.15 build 7390

There is a command :  vuforiaview://ptc.com/command/toggle-feature-flag?feature=visual_ui

QR code

image-20220426-152530.png

This will switch on/off the voice less button UI -> example:20231129_094530_HoloLens.jpg4

So means you can navigate also without voice commands!

I hope this helps in your case. Thanks

 

 

 

Hi @RolandRaytchev ,

 

Thank you for the prompt investigation about the operation of HoloLens Vuforia View without the voice commands.

I have also tried this and could see the same screen as you posted above.

 

Switching back to downloaded experiences and library is possible from within the HL experience is possible by the method you explained above.
In the place with the good network connection, we can scan the QR code of the experience and able to access the library and download from within the experience. But what procedure can I follow if there is no internet connection to load the experience? Downloading the experience in advance and again using the voice command to go to the download for launching the experience is the only way as per my understanding. 
I am sorry for the additional request, what about the addition of the download and library switching button  in the initial load screen of the Vuforia view itself rather than switching from the loaded experience ?    

 

Announcements

Top Tags