Community Tip - When posting, your subject should be specific and summarize your question. Here are some additional tips on asking a great question. X
Hi folks (and especially PTC tech folks)--
I have a question. I'm trying to implement voice recognition for mobile devices, similar to what is available on Vuzix or Realwear. I know this is not available out of the box. However, there is an HTML5 API for speech recognition, and I've been able to get it working successfully in the browser preview for an experience.
Unfortunately, when I publish, it doesn't work on mobile devices, giving an access error. As far as I can tell, this is because the Vuforia View app doesn't have permission to access the microphone on mobile devices.
Is there any way to assign microphone access to Vuforia View? I've tried fiddling with the settings on both my Android phone and iPad, and I can't find a way in settings to assign a permission that wasn't originally requested by the app.
Thanks if anyone can offer insight on this.
--Clay
Hi @ClayHelberg ,
I also could confirm the described behavior.
I tried some code example (take some sample code from https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API/Using_the_Web_Speech_API#Demo.) and this was working in chrome preview mode but I could not get it to work on Android. I used something like:
/////////////////////////////////////// var SpeechRecognition = SpeechRecognition || webkitSpeechRecognition var SpeechGrammarList = SpeechGrammarList || webkitSpeechGrammarList var SpeechRecognitionEvent = SpeechRecognitionEvent || webkitSpeechRecognitionEvent var colors = [ 'red' , 'green' , 'yellow']; var grammar = '#JSGF V1.0; grammar colors; public <color> = ' + colors.join(' | ') + ' ;' var recognition = new SpeechRecognition(); var speechRecognitionList = new SpeechGrammarList(); speechRecognitionList.addFromString(grammar, 1); recognition.grammars = speechRecognitionList; //recognition.continuous = false; recognition.lang = 'en-US'; recognition.interimResults = false; recognition.maxAlternatives = 1; $scope.onclick123 = function() { recognition.start(); console.log('Ready to receive a color command.'); } recognition.onresult = function(event) { // The SpeechRecognitionEvent results property returns a SpeechRecognitionResultList object // The SpeechRecognitionResultList object contains SpeechRecognitionResult objects. // It has a getter so it can be accessed like an array // The [last] returns the SpeechRecognitionResult at the last position. // Each SpeechRecognitionResult object contains SpeechRecognitionAlternative objects that contain individual results. // These also have getters so they can be accessed like arrays. // The [0] returns the SpeechRecognitionAlternative at position 0. // We then return the transcript property of the SpeechRecognitionAlternative object var last = event.results.length - 1; var color = event.results[last][0].transcript; console.log( 'Result received: ' + color + '.') // bg.style.backgroundColor = color; console.log('Confidence: ' + event.results[0][0].confidence); // $scope.view.wdg['label-1']['text'] = color+";" $scope.setWidgetProp('label-1', 'text', color) $scope.$applyAsync(); } recognition.onspeechend = function() { recognition.stop(); } recognition.onnomatch = function(event) { console.log("I didn't recognise that color.") } recognition.onerror = function(event) { console.log('Error occurred in recognition: ' + event.error) }
Here I tried to set a label text with voice... (the code is not the best I used is quick by copy and paste - but it was working in preview:
but as mention before -it was not working on android.
When I try to set the permission on Android device:
Settings>Apps >click on the top right menu App-Permissions > Microphone - there is list of applicaitons where we can set the permissions to access the microphone but vuforia view is not contained by this list (e.g. there is Chalk)
It seems that Vuforia View on android (and possibly on IOS )is not able to access the mirophone - because the applicaiton simple will not request these permissions. I asked yesterday in the internal PTC development group if the access for the microphone could be achieve in the Vuforia View app and hope to have some feedback soon.
Thanks for looking into it, @RolandRaytchev . I am eager to hear what you find out about whether this will be possible.
App permissions (like camera, microphone, location, etc..) all need to be defined and requested by the developer in their binary. If that dev (PTC) has not requested these permissions for an app there is no way a user can add/allow them after the fact.
Hi @ClayHelberg ,
there was internal request, where the develop only confirmed the current functionality as here reported.
So now I reported it as Jira Ticket , so, hopefully , this will be handled as product idea and could be implemented in future version.
You can track the status on this ticket by the PTC article link: CS313303
Hi @RolandRaytchev,
i am trying something quite similar on a hololens 2.
From what i can tell there is no difference to the described behavior in the previous posts.
As seen in the code below i am trying to access the users mic via navigator.mediaDevices.getUserMedia, which returns a Promise which never gets resolved nor rejected. (The output label remains in its inital state)
Are there any news on this issue?
Is the article CS313303 still up to date?
Is there a supported solution to access the microphone or even a know workaround?
If not, are there intents to implement?
Thanks and best regards
$rootScope.$on('modelLoaded', () => {
try {
navigator.mediaDevices.getUserMedia({ audio: true })
.then(stream => {
$scope.view.wdg["lblOutput"].text = 'success';
$scope.$apply();
})
.catch(error => {
$scope.view.wdg["lblOutput"].text = JSON.stringify(error.toString());
$scope.$apply();
});
}
catch (e) {
$scope.view.wdg["lblOutput"].text = e.message;
}
});