cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - You can change your system assigned username to something more personal in your community settings. X

Moving multiple 3D models using hand gesture

Aditya1702
15-Moonstone

Moving multiple 3D models using hand gesture

Hi Everyone,

I want to pan, rotate & scale each model separately using hand gesture in a project having multiple models.

Can anyone help me out with this.

Thanks in Advance. 

3 REPLIES 3

Hi @Aditya1702 ,

I found this older post where was no answer , so I will try to answer here.

I do not know what mobile platform you want to use (HoloLens or Mobile device) but I think there is  regarding this issue no big difference .  So far I know the answer is that this is  currently not possible- we have no real way to track the touch coordinates for the specific rotation , pan or move of components- only one exception - I think there is only possible way of such effect  as touch for special target but  it does not work  for specific components. In this case you will move the position of the model world coordinate. Here related posts:

https://community.ptc.com/t5/Vuforia-Studio/support-touch-screen-thingworx-studio/m-p/549493

https://community.ptc.com/t5/Vuforia-Studio/Translate-or-rotate-object-by-touch-only-in-Vuforia-Studio/m-p/621743

https://community.ptc.com/t5/Vuforia-Studio/How-to-zoom-and-rotate-an-3D-object-by-touch-in-AR-developed-in/m-p/505899

 

Regarding to a way  how to implement some handling of components - the only way what you can  use  is the tracking event and the coordinate of device location and the eye vector which are passed as arguments to the tracking event callback. I found also similar suggestion in one of the R&D post:

Trick is to use the pos,gaze info to give you an xyz location that is relative to the device.
The navigator widget is an example. Because gaze is a vector, you can add this to pos to give you a point in front of the camera. So for example you can use some pointer (object e.g. a modewidget which is updated dynamicaly)
e.g.  techniques 
1.	tracking the user gaze on hl2
2.	attaching a ‘pointer’ a fixed distance in front
3.	when user ‘doupletap’ (double airtap) the pointer is left in a fixed position in space
4.	 ‘tapping’ to record that location.
a marker could be placed to show this  

 

 

I think via similar techniques you can find some points in the current augmentation and then interpret them as modification of selected component e.g. 2 points in space could be interpreted as vector for movement. Or  they could define a rotation axis and one additional point should define the  rotation along this axis. But all this requires that you will calculate the transformation matrix of the specific geometrical change ,of the component position and then translate it new values of the model/modelItem properties x,y,z, rx,ry,ry  and this w is not trivial.

Here some post which described related techniques , which could be helpful:

https://community.ptc.com/t5/Vuforia-Studio/Translating-a-model-when-clicked-on-it/m-p/636185#M7112

https://community.ptc.com/t5/Vuforia-Studio/Rotate-model-with-button-click-along-Z-rotation-Axis/m-p/633018

https://community.ptc.com/t5/Vuforia-Studio/Simultaniously-rotate-model-and-image-Vuforia-Studio/m-p/653234#M7709

https://community.ptc.com/t5/Vuforia-Studio/Possibility-to-change-set-the-spincenter-csys-of-models-and/m-p/796282#M10843

https://community.ptc.com/t5/Vuforia-Studio/I-want-to-change-position-of-modelitem-to-move-when-i-click-on/m-p/637163#M7161

https://community.ptc.com/t5/Vuforia-Studio/Model-rotation-please-share-sample-project/m-p/854323#M11534

 

and these handling a tracking of the eye vector and position

https://community.ptc.com/t5/Vuforia-Studio-and-Chalk-Tech/How-can-we-make-a-3D-Widget-on-HoloLens-visible-in-front-of-me/ta-p/658541

https://community.ptc.com/t5/Vuforia-Studio-and-Chalk-Tech/How-to-create-a-custom-button-pressable-on-HoloLens-device/ta-p/654984

 

 

 

 

 

 

 

I implemented a very simple version of this (using vertical swipes to "toss" a virtual object), by leveraging HTML5 touch events. The process goes something like this:

 

  1. Add a Card widget to the center 2D panel, and use CSS to make it fixed and fill the screen
  2. Attach event listeners for touch events to that panel ("panel-6" in the code snippet), like so:
    $scope.init = function() {
      try {
        var container = document.querySelector("*[widget-id='panel-6'] div");
          container.addEventListener("touchstart",$scope.graphicTouchStart);
          container.addEventListener("touchmove",$scope.graphicTouchMove);
          container.addEventListener("touchend",$scope.graphicTouchEnd);
        $scope.originZ = $scope.view.wdg['model-1'].z;
        $scope.originY = $scope.view.wdg['model-1'].y;
      }
      catch (ex) {
        $scope.setWidgetProp("label-1","text","Couldn't set touch listeners: " + ex.message);
      }
    }
    
    angular.element(document).ready($scope.init);
    
    ​
  3. Define the graphic event handlers. For your use case, "touchmove" is probably the most important one. You can determine the movement by storing the start location on "touchstart", and calculating the difference. Once you know the direction, you can apply the appropriate rotation to the desired model.
  4. Keep in mind that this approach means that your "touchscreen" panel will intercept most touch events, so Vuforia View may not see certain things like tapping on a button or a 3D model any more. You can manage the visibility of the panel to control when the user is in "normal Vuforia touch mode" and your custom "rotate my model" mode.

For details on how HTML5 touch event handling works, see this excellent article from Mozilla: https://developer.mozilla.org/en-US/docs/Web/API/Touch_events/Using_Touch_Events

 

This will be a bit of work for you, but hopefully the result will be worth the effort!

Hi @ClayHelberg ,

thanks for pointing to this possible solution. So I did test the touch event  in the past but seems that it is not supported by each widget- so with card work but e.g.  was not working with the panel- at least the event did not fire when I tested it , but possibly was something  wrong in my project. 

Anyways with card widget seems to work fine. But the calculation of the correct movement -required to be considered more detailed - , how to interpret the measure in windows coordinate to model coordinate is something what need a concept- e.g. in CAD systems e.g. Creo there are some transformations (by transformation matrix - or euler x,y,z,rx,ry,rz) which did the transformation between model, screen and windows coordinates and also component coordinates - e.g. when we want to move a component. So means also it is important that the movement should be not zoom sensitive

Based on your suggestion I want to provide small  example which do only movements of an model with simple delta calc, no real transformations - 

Also want to mention that this approach  works only for mobile project but not for Eyewear -because there is no 2D element - possibly could work in tmlText 

here also a good link for touch event : https://w3c.github.io/touch-events/#dom-touchevent-touches

 

Announcements

Top Tags