cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Enable microphone on mobile devices?

Alexandrite

Enable microphone on mobile devices?

Hi folks (and especially PTC tech folks)--

 

I have a question. I'm trying to implement voice recognition for mobile devices, similar to what is available on Vuzix or Realwear. I know this is not available out of the box. However, there is an HTML5 API for speech recognition, and I've been able to get it working successfully in the browser preview for an experience.

 

Unfortunately, when I publish, it doesn't work on mobile devices, giving an access error. As far as I can tell, this is because the Vuforia View app doesn't have permission to access the microphone on mobile devices.

 

Is there any way to assign microphone access to Vuforia View? I've tried fiddling with the settings on both my Android phone and iPad, and I can't find a way in settings to assign a permission that wasn't originally requested by the app.

 

Thanks if anyone can offer insight on this.

 

--Clay

5 REPLIES 5

Re: Enable microphone on mobile devices?

Hi @ClayHelberg ,

I also could confirm the described behavior.

I tried some code example (take some sample code from https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API/Using_the_Web_Speech_API#Demo.) and this was working in chrome preview mode but I could not get it to work on Android. I used something like:

///////////////////////////////////////
var SpeechRecognition = SpeechRecognition || webkitSpeechRecognition
var SpeechGrammarList = SpeechGrammarList || webkitSpeechGrammarList
var SpeechRecognitionEvent = SpeechRecognitionEvent || webkitSpeechRecognitionEvent

var colors = [ 'red' , 'green' , 'yellow'];
var grammar = '#JSGF V1.0; grammar colors; public <color> = ' + colors.join(' | ') + ' ;'

var recognition = new SpeechRecognition();
var speechRecognitionList = new SpeechGrammarList();
speechRecognitionList.addFromString(grammar, 1);
recognition.grammars = speechRecognitionList;
//recognition.continuous = false;
recognition.lang = 'en-US';
recognition.interimResults = false;
recognition.maxAlternatives = 1;


$scope.onclick123 = function() {
  recognition.start();
  console.log('Ready to receive a color command.');
}
recognition.onresult = function(event) {
  // The SpeechRecognitionEvent results property returns a SpeechRecognitionResultList object
  // The SpeechRecognitionResultList object contains SpeechRecognitionResult objects.
  // It has a getter so it can be accessed like an array
  // The [last] returns the SpeechRecognitionResult at the last position.
  // Each SpeechRecognitionResult object contains SpeechRecognitionAlternative objects that contain individual results.
  // These also have getters so they can be accessed like arrays.
  // The [0] returns the SpeechRecognitionAlternative at position 0.
  // We then return the transcript property of the SpeechRecognitionAlternative object

  var last = event.results.length - 1;
  var color = event.results[last][0].transcript;

  console.log( 'Result received: ' + color + '.')
 // bg.style.backgroundColor = color;
  console.log('Confidence: ' + event.results[0][0].confidence);
 // $scope.view.wdg['label-1']['text'] = color+";"
  
  $scope.setWidgetProp('label-1', 'text', color)
  $scope.$applyAsync();                   
                          
}

recognition.onspeechend = function() {
  recognition.stop();
}

recognition.onnomatch = function(event) {
  console.log("I didn't recognise that color.")
}

recognition.onerror = function(event) {
  console.log('Error occurred in recognition: ' + event.error)
}

Here I tried to set a label text with voice... (the code is not the best I used is quick by copy and paste - but it was working in preview:

2019-06-27_10-10-53.jpg

but as mention before -it was not working on android.

When I try to set the permission on Android device:
Settings>Apps  >click on the top right menu App-Permissions > Microphone - there is list of applicaitons where we can set the permissions to access the microphone but vuforia view is not contained by this list (e.g. there is Chalk)
It seems  that Vuforia View on android (and possibly on IOS )is not able to access the mirophone - because the applicaiton simple will not request these permissions. I asked yesterday in the internal PTC development group if the access for the microphone  could be achieve in the Vuforia View app and hope to have some feedback soon.

 

Re: Enable microphone on mobile devices?

Thanks for looking into it, @RolandRaytchev . I am eager to hear what you find out about whether this will be possible.

Re: Enable microphone on mobile devices?

App permissions (like camera, microphone, location, etc..) all need to be defined and requested by the developer in their binary. If that dev (PTC) has not requested these permissions for an app there is no way a user can add/allow them after the fact.

Re: Enable microphone on mobile devices?

Hi @ClayHelberg ,

there was internal request, where the develop only confirmed the current functionality as here reported.

So now I reported it as Jira Ticket , so,  hopefully , this will be handled as product idea and could  be implemented in future version.

You can track the status on this ticket by the PTC article link:  CS313303

Re: Enable microphone on mobile devices?

Excellent, thanks @RolandRaytchev !