Community Tip - Stay updated on what is happening on the PTC Community by subscribing to PTC Community Announcements. X
Hello,
I am currently working on a HoloLens experience which will be used in large physical space and the users need access to some of the information no matter whether they are near the loaded experience or not.
The HoloLens Start menu and the Vuforia Studio Experience selector menu uses the concept, which Microsoft calls tag-along.
Is it possible to include/edit this property for a studio 3D Label?
Thanks in advance!
Solved! Go to Solution.
Hello @IstvanPolacsek ,
found that your question was staying for long time period without answer.
So, for me at first time was not quite clear what was your question , because I did not know the term "Microsoft calls tag-along " but I checked the Microsoft side like:
https://docs.microsoft.com/en-us/windows/mixed-reality/billboarding-and-tag-along
where this is more detailed explained.
So that I think , you want to have some kind of dynamically movement of elements to stay in same position in relation to the gaze vector and eye postion.
The answer is ... it could be possible
Let explain.
-currently in the HoloLens our AR world is the Vuforia View application. This means you need to use one of the possible tracking methods to have a position for all object in the space. So when you scan for example a thingmrk - in this case this is our world coordinate system and all 3d object are located in regards to model coordinate system. (actually the model coordinate system is an offset to the thin mark because it is seldom to the point 0.0,0) IF you move the location of the thingmark and scan it again , then the position all 3d objects will update regarding the new location of the thingmark.
But we have a functionality what may be helpful here. We can track the eye /camera position by having position vector, eye_direction vector and eye_up direction. In this case theoretical you can calculate every time when you eye /HoloLens device is moved or rotated - so that you will have the effect like some kind of billboard to your HoloLens.
The following code :
///////////////////////////////////// $rootScope.$on('modelLoaded', function() { $scope.setEYEtrack() // ... }) ////// //////////////////////////////////////////////////////////// $scope.setEYEtrack= function() { tml3dRenderer.setupTrackingEventsCommand (function(target,eyepos,eyedir,eyeup) { $scope.app.params['eyepos'] ="eyepos=("+eyepos[0].toFixed(2)+","+eyepos[1].toFixed(2)+","+eyepos[2].toFixed(2)+")"; $scope.app.params['eyedir'] ="eyedir=("+eyedir[0].toFixed(2)+","+eyedir[1].toFixed(2)+","+eyedir[2].toFixed(2)+")"; $scope.app.params['eyeup' ] ="eyeup =("+ eyeup[0].toFixed(2)+","+ eyeup[1].toFixed(2)+","+ eyeup[2].toFixed(2)+")"; var cam_mat = $scope.Get_mat4x4_camera(eyepos,eyedir,eyeup) .... }) //....
will stetup a tracking event and will provide to every change in the eye position the vectors for eyedir,eyeup and eyepos (eye position in respect to the model coordinates)
Regarding to the calculation of the correct values of the 3d element coordinates - it could be a little bit complicated but is still possible . Also, we need to have such calculation for each object we want to move.
The most trivial way is simple to set the billboard property of the element and to move only the position
var scale=5.0; $scope.setWidgetProp('3DImage-1', 'x', ( eyepos[0]+eyedir[0]*scale)); $scope.setWidgetProp('3DImage-1', 'y', ( eyepos[1]+eyedir[1]*scale)); $scope.setWidgetProp('3DImage-1', 'z', ( eyepos[2]+eyedir[2]*scale));
So it does not matter where we move the model ( move because I tested it here in preview) the 3dimage goes always to the screen center and with a distance offset to the screen plane in eye direction
Of course we could do a more precise positioning but need some additional mathematical consideration
So I tested it also on android device but the movement seem to be not updated on my first attempt fluently so may be will be better to update it on request - so means in some situation I need to update the position of my elements by command or by object click etc.
I
think my project have a lot of different thing I tested there, so that the result are ,may be, not so relevant according the performance and this should be further tested if better results could be achieved.
Hello @IstvanPolacsek ,
found that your question was staying for long time period without answer.
So, for me at first time was not quite clear what was your question , because I did not know the term "Microsoft calls tag-along " but I checked the Microsoft side like:
https://docs.microsoft.com/en-us/windows/mixed-reality/billboarding-and-tag-along
where this is more detailed explained.
So that I think , you want to have some kind of dynamically movement of elements to stay in same position in relation to the gaze vector and eye postion.
The answer is ... it could be possible
Let explain.
-currently in the HoloLens our AR world is the Vuforia View application. This means you need to use one of the possible tracking methods to have a position for all object in the space. So when you scan for example a thingmrk - in this case this is our world coordinate system and all 3d object are located in regards to model coordinate system. (actually the model coordinate system is an offset to the thin mark because it is seldom to the point 0.0,0) IF you move the location of the thingmark and scan it again , then the position all 3d objects will update regarding the new location of the thingmark.
But we have a functionality what may be helpful here. We can track the eye /camera position by having position vector, eye_direction vector and eye_up direction. In this case theoretical you can calculate every time when you eye /HoloLens device is moved or rotated - so that you will have the effect like some kind of billboard to your HoloLens.
The following code :
///////////////////////////////////// $rootScope.$on('modelLoaded', function() { $scope.setEYEtrack() // ... }) ////// //////////////////////////////////////////////////////////// $scope.setEYEtrack= function() { tml3dRenderer.setupTrackingEventsCommand (function(target,eyepos,eyedir,eyeup) { $scope.app.params['eyepos'] ="eyepos=("+eyepos[0].toFixed(2)+","+eyepos[1].toFixed(2)+","+eyepos[2].toFixed(2)+")"; $scope.app.params['eyedir'] ="eyedir=("+eyedir[0].toFixed(2)+","+eyedir[1].toFixed(2)+","+eyedir[2].toFixed(2)+")"; $scope.app.params['eyeup' ] ="eyeup =("+ eyeup[0].toFixed(2)+","+ eyeup[1].toFixed(2)+","+ eyeup[2].toFixed(2)+")"; var cam_mat = $scope.Get_mat4x4_camera(eyepos,eyedir,eyeup) .... }) //....
will stetup a tracking event and will provide to every change in the eye position the vectors for eyedir,eyeup and eyepos (eye position in respect to the model coordinates)
Regarding to the calculation of the correct values of the 3d element coordinates - it could be a little bit complicated but is still possible . Also, we need to have such calculation for each object we want to move.
The most trivial way is simple to set the billboard property of the element and to move only the position
var scale=5.0; $scope.setWidgetProp('3DImage-1', 'x', ( eyepos[0]+eyedir[0]*scale)); $scope.setWidgetProp('3DImage-1', 'y', ( eyepos[1]+eyedir[1]*scale)); $scope.setWidgetProp('3DImage-1', 'z', ( eyepos[2]+eyedir[2]*scale));
So it does not matter where we move the model ( move because I tested it here in preview) the 3dimage goes always to the screen center and with a distance offset to the screen plane in eye direction
Of course we could do a more precise positioning but need some additional mathematical consideration
So I tested it also on android device but the movement seem to be not updated on my first attempt fluently so may be will be better to update it on request - so means in some situation I need to update the position of my elements by command or by object click etc.
I
think my project have a lot of different thing I tested there, so that the result are ,may be, not so relevant according the performance and this should be further tested if better results could be achieved.
Hello @RolandRaytchev ,
Many thanks for the solution.
I was able to implement it using the following code:
$scope.setTagalong= function() { tml3dRenderer.setupTrackingEventsCommand (function(target,eyepos,eyedir,eyeup) { $scope.app.params['eyepos'] = [eyepos[0].toFixed(2),eyepos[1].toFixed(2),eyepos[2].toFixed(2)]; $scope.app.params['eyedir'] = [eyedir[0].toFixed(2),eyedir[1].toFixed(2),eyedir[2].toFixed(2)]; $scope.app.params['eyeup' ] = [eyeup[0].toFixed(2), eyeup[1].toFixed(2), eyeup[2].toFixed(2)]; }) }; $scope.showTagalongLabel = function() { $scope.setWidgetProp('tagalongLabel','visible',true); $scope.setWidgetProp('tagalongLabel','billboard',true); $scope.setTagalong(); $scope.setWidgetProp('tagalongLabel','x',(Number($scope.app.params['eyepos'][0])+Number($scope.app.params['eyedir'][0]))); $scope.setWidgetProp('tagalongLabel','y',(Number($scope.app.params['eyepos'][1])+Number($scope.app.params['eyedir'][1]))); $scope.setWidgetProp('tagalongLabel','z',(Number($scope.app.params['eyepos'][2])+Number($scope.app.params['eyedir'][2]))); };
I made a binding to a 2D button and now, each time I press the button, the position of the 3D label updates.
That's great!
What I am missing here is an event, which I can bind it to have the values updated constantly.
Any ideas?
Thanks,
Istvan
I was able to make the label follow the camera using setInterval method.
I made a test on an iPhone, will try to achieve the same thing using HoloLens.
@RolandRaytchev , Thank you very much again for the help!
Unfortunately I had no success reproducing it in HoloLens.
Seems like the tml3dRenderer function is not working on HoloLens.
Or maybe I am missing something, any ideas?
Thanks!
Hi @IstvanPolacsek ,
I tested it today on the HoloLens but it was not working with the current release. I am not 100% sure , but I believe it was working earler. I tested with 8.4.2 both preivew on studio and the vuforia view on the Hololens
In preview of HoloLense project work but not on the enddevice - HoloLens 1
The following code tested :
$rootScope.$on('modelLoaded', function() { $scope.setEYEtrack() console.warn(tml3dRenderer); }); //////////////////////////////////////////////////////////// $scope.setEYEtrack= function() { // OUTSIDE a Function when loading // but better to call in modelload //// define tracingEvent only on end device tml3dRenderer.setupTrackingEventsCommand (function(target,eyepos,eyedir,eyeup) { //3DImage-1 var scale=5.0; $scope.setWidgetProp('3DImage-1', 'x', ( eyepos[0]+eyedir[0]*scale)); console.log("3DImage-1 -> x ="+ ( eyepos[0]+eyedir[0]*scale)); $scope.$applyAsync(); $scope.setWidgetProp('3DImage-1', 'y', ( eyepos[1]+eyedir[1]*scale)); console.log("3DImage-1 -> y ="+ ( eyepos[1]+eyedir[1]*scale)); $scope.$applyAsync(); $scope.setWidgetProp('3DImage-1', 'z', ( eyepos[2]+eyedir[2]*scale)); console.log("3DImage-1 -> z ="+ ( eyepos[2]+eyedir[2]*scale)); $scope.$applyAsync(); $scope.app.params['eyepos'] ="eyepos=("+eyepos[0].toFixed(2)+","+eyepos[1].toFixed(2)+","+eyepos[2].toFixed(2)+")"; $scope.app.params['eyedir'] ="eyedir=("+eyedir[0].toFixed(2)+","+eyedir[1].toFixed(2)+","+eyedir[2].toFixed(2)+")"; $scope.app.params['eyeup' ] ="eyeup =("+ eyeup[0].toFixed(2)+","+ eyeup[1].toFixed(2)+","+ eyeup[2].toFixed(2)+")"; $scope.$applyAsync(); }) //////////////////////////////////////////////////// } ////// finish setEYEtrack
But this did not work on the HoloLens - so I also observed that this event seem not to fire
Hi @IstvanPolacsek,
so now I got it after a hint from PTC R&D and it works now on the HoloLens.
So you need to add/set in Vuforia Studio Installation the property "showTrackingEvents": true in the builder-settings.json.
Then when your restart studio and open the project and then you can see the property "Enable Tracking Events" for 3d Container Widget. You need to activate it.
Afterword’s you can track the event by calling :
$scope.$root.$on('tracking', function() { ...}
I
If you need the vectors for the camera you need to call the tml3dRenderer.setupTrackingEventsCommand - so then you do not need the definition of the $scope.$root.$on('tracking',... event
Example:
scope.setMyEYEtrack= function() { if(tml3dRenderer) { try { tml3dRenderer.setupTrackingEventsCommand (function(target,eyepos,eyedir,eyeup) { var scale=5.0; $scope.setWidgetProp('3DImage-1', 'x', ( eyepos[0]+eyedir[0]*scale)); $scope.setWidgetProp('3DImage-1', 'y', ( eyepos[1]+eyedir[1]*scale)); $scope.setWidgetProp('3DImage-1', 'z', ( eyepos[2]+eyedir[2]*scale)); $scope.$applyAsync(); },undefined) } catch (e) { $scope.setWidgetProp('3DLabel-2', 'text', "exception=");} }else $scope.setWidgetProp('3DLabel-2', 'text', "null tml3dRenderer object on HoloLens"); } ////// finish setEYEtrack
This will move a 3d Image widget (billboard = true) always in the middle of your view field with distance = scale - is here 5 m