Manipulating Scene to Add New AR Content

[BenAtLmco]

Hello!  I'm wondering if there is any way to directly manipulate the AR scene being displayed to be able to add additional content.  I know that there have been several questions on this board about being able to dynamically add 3D widgets, with the general response being that it is not supported/not possible.  I'm wondering if it's possible to simply add a new model to the scene without having to use a widget.

From what I have seen from poking around in my files, I'm under the impression that Vuforia is using threejs to create and maintain the AR scene displayed to the user.  I know that threejs provides a variety of tools to control and manipulate a scene and that I have access to that library via the variable THREE in Home.js.  However, I've been struggling to identify the variable which contains the scene in which the AR content is rendered.

I thought maybe it could be the scene contained in PTC.GLOBAL_SCENE which can be obtained via PTC.GLOBAL_SCENE.getRoot(), but this doesn't seem to be the case.  The variable returned by getRoot() doesn't seem to contain any of my other models as children and is an Object3D and not a scene.

Am I right in my assumption that I should be able to add new models by directly manipulating the 3D scene?  And if so, where do I access the scene to be able to add new content?

Any help would be greatly appreciated!

[RolandRaytchev]

Hi @BenAtLmco ,

I think that you mentioned is generally possible but is strongly not recommended.

Why not: 

- so in Studio we have a different levels - example. three.js (on specific platform! -e.g. chrome) , cordova , angular.js etc.

Hacking some solution on low level could cause 2 things:

- your application will be not portable on all platforms (e.g. it will work only on preview mode but not on mobile)

-possibly after some changes during the Studio development will cause that your application will not work anymore. So, support is not guaranteed for later releases

Therefore, I think the best approach here to try to solve you problem with the available supported Studio APIs

Of course, such solution depends on the specific goals in your application.

For example: we can specify some “scen” in one model widget.

Then all other object which are possible to be added could be contained by second assembly  which is display in additional model widget. Also, the both model widget e.g. used the same cood system or there are some defined relations. So you can display then the additional items depending on your use case etc.

But if your provide more details about what you want to achieve then we can check what Studio functionality could be used – case based.

Thanks

[BenAtLmco]

Hi @RolandRaytchev!

I wanted to give you some more context for the specific outcomes I was trying to achieve.  From your feedback, it seems the way I'm currently trying to achieve my goals is headed in the wrong direction and I wanted to see if you might have some guidance on how to accomplish them using existing studio APIs.

The first use case is to display 3D annotations in the AR environment.  The placement and text/images in the annotations would be controlled via Thingworx.  The quantity of annotations is unknown beforehand, which is why I was exploring adding models dynamically.  I've thought of a few workarounds:

- Having a single 3DLabel/3DImage which is updated while iterating through the list of annotations

- Having a finite number of 3D Labels/3D Images and hide/showing them based on how many annotations need to be displayed.

The second use case is loading in models where it is impractical to load in the entire assembly.  If I had a situation where my assembly consists of 1,000's - 10,000's of individual components, I'm not sure that hide/showing them is the best approach.  I would like to potentially be able to have the user query for the components they would like to load, retrieve them from Thingworx, and then dynamically load them into the AR experience.  Obviously similar workarounds as listed above might work, but both come with restrictions.  Either only a single model can be displayed at a time or I am forced to have a preset limit on the number of models which can be displayed.

I'm sure there are good technical reasons behind the scenes for why it is not possible to dynamically add/remove widgets, but it's a limitation I've run up against several times.  It might not necessarily be an impediment to getting most of the functionality I want out of the applications I'm building, but it seems like an odd/unfortunate restriction (at least to someone inexperienced like me!).  I was hoping to see if there was a hacky/janky way to bypass those restrictions, even if just to experiment with to better understand how Vuforia is working behind the scenes.

[RolandRaytchev]

Hi @BenAtLmco ,

thanks for your feedback! Ok I see. 

As first point want to mention here an addtional reference to a related post : "Best practice of 3D Connecting Line/Arrow ?"

OK the idea to have the possibility to use infinite numbers of elements sound good but we have always a world with finite number of elements. Having 10000 components - means very large models could cause some performance issues on mobile devices - so always we need to try to reduce the number of displayed data entities as much as possible. For example in the Creo Parametric world we can try to simplify the assembly display showing only the data which is relevant to the current view – one example is usage  of simplified representation  with some rules ( exclude inner level parts or small parts … etc.)

Let suppose you really can display an assembly with infinite number of parts. In the current view you can try to give some advice. Your advice should provide specific information about an specific components / or group of components. In this case for each step you need to display limited numbers of annotation, so that your advise could be simple ,  clear and concise. I do not think that the infinitive numbers of annotation in a specific view will be very helpful.

So, I think ,the approach could be to use a finite numbers of annotation for all possible components. Ok in this case they should be kind of dynamic. If you have a data about the position of each annotation you can use e.g. some 3D widgets which you can move according the current scene. Also the implementation of such approach  could require a lot of work, I  believe.   

[SteveGhee]

on the topic of managing content dynamically, it may not be immediately apparent that all 3d DOM content within View is specified as a URI e.g. it can be a pointer to a local file (in the experience 'uploads' folder, or a response from some remote service call e.g. to Thingworx.  Content does not have to be static or even declared at edit time - it can be pulled into the view dynamically.  Thus if you have a usecase where you want content to be loaded based on some state or condition, this is very easy to do.  This technique can be used both for content not known at edit time, but also for lists of content unknown e.g. N annotations to be shown.   N can be defined at runtime.  

[SteveGhee · Accepted solution]

It is important to state first that View does not use three.js at all, so you cannot follow that path.

View does, however, provide a scene graph to which elements can be added and controlled - its an extension to the html DOM and there are various 3d scene elements that you will no doubt recognise e.g. 3d models, labels, images etc., all that can be tracked within a physical 3d space.  

The APIs to manage the content of the 3D DOM have not, until now, been documented, but that is being addressed now, and we will soo be publishing more detailed examples and documentation that should address many of the questions asked here.