Skip to main content
14-Alexandrite
November 27, 2023
Question

About HoloLens 2 voice command Alternatives

  • November 27, 2023
  • 1 reply
  • 1663 views

Dear all, 

As of now Vuforia provides only the voice commands for the operation of the HoloLens. Using HoloLens experience inside the factory is very difficult because of the continuous loud noise within the factory premises. We are thinking to add buttons and perform the desired operation like scan, show library, show download etc.. from within the HoloLens experience.

Till now , we are familiar with the Only API which will switch the Vuforia view app to scan mode. app.fn.navigateToScanMode();

Do someone know the API to switch the Vuforia View to the Vuforia library as well as to the downloaded experiences ?  Thank you in advance.

1 reply

21-Topaz I
November 28, 2023

Hi @Sandy_Gunner ,

I checked the reported issue to PTC R&D and see that related issue were reported and R&D is aware about this requirements but after deep check - there are currently some plans for improvement and  development - but they are not classified as urgent  because according to R&D team there are currently sufficient techniques to work around with loud enviroment. 

2 strategies' to improve the current behavior in the current release:

>> R&D statement about how to use better the voice command:

There is no specific improvements that can be made for the HL voice command recognition. Some further information:

  • The log shows low recognition quality on some of the basic key phrases, this could be an indicator of several things: Unusual pronunciation by the user, noisy environment that causes trouble for the microphone, or a faulty device.
  • Custom voice commands might have lower confidence than the standard key phrases so it might help to test out variants that fit the user pronunciation and are easier to recognize for the HL.
  • In general the voice recognition Microsoft provides on the HL is not on the same level as the current services from the Google Assistant or Apple’s Siri. This might lead to a higher expectation than the device can deliver.

>> Design ;

  • so when you start on HL2 , then the view app will go by default to scan mode - scan thingmark or QR code to open a project
  • the project have a start View where you have buttons to navigate -you can use the mentioned api:  
    • app.fn.navigateToScanMode()  to navigate back to scan mode
    • $scope.app.fn.navigate(View_name)  to navigate between views e.g. : 
      //navigate to Home view
      $scope.app.fn.navigate('Home');
      // navigate to View named CHAR2DVIEW
      $scope.app.fn.navigate('CHAR2DVIEW')​
    • window.open(url) e.g to open an experience:
      $scope.openUrl=function()
      {
      // let url= "https://www.bing.com/"
      let url="https://xxxxxxo.studio-trial.thingworx.io/ExperienceService/content/projects/bla/index-desktop.html?expIndex=0#/Home"
       window.open(url)
      // some related points
      // window.inf.allowIntent(url) 
      // window.location = url
      }​
  • Here also statement from the R&D team to the last issue reported this year "Discovery and Development of voiceless command alternatives for HoloLens"  - some point are already implemented further are under dev and possibly will come next.
  • Description:
    As a second step in offering alternative ways for navigating in the View app using HoloLens, the task is to couple the system navigation logic with buttons (until now it was solely voice commands). It is still a partial integration, meaning further improvement with long-range plan
  • Scope of dev
    Based on the developments from reported Ticket w: Investigate Microphone setup for Voiceless commands
    , the process should cover further steps:

        //I think this is planed improvement below I did not check if this is already part of the current 9.15 functionality /did not tested yet

           Attach customers  expected functions to the created buttons:

             Library - Opening Library and its content

             Downloads - Showing experiences that got downloaded to the device from the Library

  •          NOT Scope
    Within above sections, all Experiences should be presented as buttons, that upon being clicked, open the experience. When opening the experience, hide the selector menu Within the experience, show a button “Close” which leads back to the originator menu (Library / Downloads) serving as a “Back” button - not required
  • Implementing a floating panel - this is already down
  • Implementing a hand gesture as main trigger
  • Final UI design
  • Idea: Use a floating panel for the buttons 

 

Logic:
Step 0: Deactivate Microphone (or make sure that it is deactivated)

Step 1: Open View App

Step 2: Scan specific QR Code that triggers the functionality

Step 3: Once you close the app, functionality is automatically disabled

 

 

14-Alexandrite
November 29, 2023

Hello @RolandRaytchev ,

 

Thank you for your prompt investigation. 

I forgot to mention about the offline use of the HoloLens inside the factory earlier. 

To use the API mentioned above, I think the network connection is mandatory which is not always guaranteed inside the factory.

$scope.openUrl=function()
{
let url="https://xxxxxxo.studio-trial.thingworx.io/ExperienceService/content/projects/bla/index-desktop.html?expIndex=0#/Home"
window.open(url);
}

 

Do the support team planning to provide the API to go to the library as well as in download similar as to go to the scan mode API or they will provide built-in UI to switch between scan mode, library and download for the Vuforia View application for HoloLens same as in Vuforia View for Ipad ? 

21-Topaz I
November 29, 2023

Hello @Sandy_Gunner , thanks for your feedback!

Yes calling the Vuforia Studio experience via link requires a network connection.

But... I further checked in the R&D reported tickets and found  further information about voiceless usage - menu. So I was not sure if this is already implemented yet or still in development therefore I tested it by myself. So remove Vuforia view app and installed it again on HL2- so my HL has an issue so that this is the way to have an update. Now having the latest version I could confirmed that the option suggested /developed by PTC R&D team is working in the current version (Hey View , version ) 99.15 build 7390

There is a command :  vuforiaview://ptc.com/command/toggle-feature-flag?feature=visual_ui

QR code

image-20220426-152530.png

This will switch on/off the voice less button UI -> example:20231129_094530_HoloLens.jpg4

So means you can navigate also without voice commands!

I hope this helps in your case. Thanks