How to add audio voice command during sequencing

[shuaib]

Hello,

I want to configure audio voice while playing sequencing of cad model in experience so that it will help to user in understanding of sequencing. Any suggestion or help will be appreciated.

[RolandRaytchev]

For which    device is it?  

Do you mean commands or only speech info?

So for the HoloLense the following function :

$scope.speak = function(spokenText){ 
var msg = new SpeechSynthesisUtterance(spokenText);
window.speechSynthesis.speak(msg);
}

will "speak" the spokenText argument string.

On the HoloLens the voice command could be defined easier on the UI.

For android or IOS devices there should be also functionality for example playing some audio files and voice recognition but I did not checked this functionality yet.

[shuaib]

Thanks Roland, but i am looking it for mobile devices not for hololens. There is widget in studio for audio voice  but i want to configure this audio voice with sequencing steps. For example when i click on play sequncing button, the sequencing steps (created in illustrate) should also generate audio voice like "in this step,unscrew the the bolt". Rather than showing this kind of message on text label i want to deleiver it in audio voice.. 

[RolandRaytchev]

I have no idea ( i did not checked how this should work - there could be a way to do this but need investigations) how to interpret the text to a speech.

One idea what I have is to record with audio recorder the adio files for the different step messages . Then upload the files and use some constuct like this below:

$scope.play_step1_audio = function(){
$scope.setWidgetProp('audio-1', 'audiosrc', "Uploaded/play_step2.mp3");
//to load the step 3 to the adio widget
angular.element(document.getElementById('audio-1')).scope().play(); }

And then you can use this on the same way as described in the post/Tech Tip but instead of setting a label you can call the play_stepXX_audio() function:

Display sequence step names from Creo Illustrate

In case that you want to recognize some voice spoken on microphone - I am not sure if this is possible....

[RolandRaytchev]

My observation is that it is working on HoloLense, IOS and in Crome preview (beieve is working also on windows devices)  but I did not get it working on the Android Galaxy S9.

Some code like : 

var msg = new SpeechSynthesisUtterance('Hello World');
window.speechSynthesis.speak(msg);

https://stackoverflow.com/questions/45178008/start-running-speechsynthesis-api-on-my-android-and-safari-devices

or

https://developers.google.com/web/updates/2014/01/Web-apps-that-talk-Introduction-to-the-Speech-Synthesis-API

But believe it is a security issue for my app or some other android setting... need to further check

[RolandRaytchev]

So now found what should be the correct call so that it will work also on Android device - It was working when tested on  Samsung Galaxy S9+

 
var speech = $injector.get('appSpeechService');
 speech.synthesizeSpeech( {text:"Hello World"}, 
                       (function(){console.log("success");}),
                       (function(){console.log("failed");}) ); 

If you use your code for different devices OS then you need to do some checks as described in:

https://stackoverflow.com/questions/11381673/detecting-a-mobile-browser

or you can also use  if(twx.app.isPreview()) ... to check if the code is called inside the preview