Hi @aletenti ,
so far I know there is no direct way to do this - via standard widget on HoloLens 2 or general in 3D space. Possibly there are some development customized extension but I do not know such extension. Possibly somebody else could share here if he/she has information about other techniques.
For me -there should be 2 possible ways to try to implement it - what I consider there is only the display of the data. Ok the data should be received by e.g. Thingworx service first and then need to be displayed . And I consider here only the display of e.g. Table but there we need to call the service and in the service -complete event we need then to call the function to display the data on e.g. 3D panel - so to update the data. Then possibly we need a timer or event setting e.g. call every 5 minutes to update the data this is also a think what we need to implement if we want to have an repeater functionality there in the 3D area. So the options I see are:
1.) use tmlText widget to define a shader and then to display the data as shader on image panel. The data need to be passed to the shader as shaders parameter something like this https://stackoverflow.com/questions/8847899/how-to-draw-text-using-only-opengl-methods for GLSL ... But ! ... we need for HoloLens2 to use HLSL and I could not find a relevant example ... I did not search very detailed because I think the point 2 is that I prefer as option
2.) use the data to generate a dynamic table in SVG format and display it on e.g. Image widget. E.g. the following table from stack overflow... forum I used as example : https://stackoverflow.com/questions/6987005/create-a-table-in-svg
$scope.sampleSVG1 = function(text) { console.log("$scope.sampleSVG1")
// Name of image widget to edit
let sampleImgWidget = '3DImage-2';
let svgData = `<?xml version='1.0' standalone='no'?>
<!DOCTYPE svg PUBLIC '-//W3C//DTD SVG 1.1//EN'
'http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd'>
<svg width='80%' height='80%' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink'>
<title>SVG Table</title>
<g id='columnGroup'>
<rect x='65' y='10' width='75' height='110' fill='gainsboro'/>
<rect x='265' y='10' width='75' height='110' fill='gainsboro'/>
<text x='30' y='30' font-size='18px' font-weight='bold' fill='crimson'>
<tspan x='30' dy='1.5em'>Q1</tspan>
<tspan x='30' dy='1em'>Q2</tspan>
<tspan x='30' dy='1em'>Q3</tspan>
<tspan x='30' dy='1em'>Q4</tspan>
</text>
<text x='100' y='30' font-size='18px' text-anchor='middle'>
<tspan x='100' font-weight='bold' fill='crimson'>Sales</tspan>
<tspan x='100' dy='1.5em'>$ 223</tspan>
<tspan x='100' dy='1em'>$ 183</tspan>
<tspan x='100' dy='1em'>$ 277</tspan>
<tspan x='100' dy='1em'>$ 402</tspan>
</text>
<text x='200' y='30' font-size='18px' text-anchor='middle'>
<tspan x='200' font-weight='bold' fill='crimson'>Expenses</tspan>
<tspan x='200' dy='1.5em'>$ 195</tspan>
<tspan x='200' dy='1em'>$ 70</tspan>
<tspan x='200' dy='1em'>$ 88</tspan>
<tspan x='200' dy='1em'>$ 133</tspan>
</text>
<text x='300' y='30' font-size='18px' text-anchor='middle'>
<tspan x='300' font-weight='bold' fill='crimson'>Net</tspan>
<tspan x='300' dy='1.5em'>$ 28</tspan>
<tspan x='300' dy='1em'>$ 113</tspan>
<tspan x='300' dy='1em'>$ 189</tspan>
<tspan x='300' dy='1em'>$ 269</tspan>
</text>
</g>
</svg>`;
let svgURL = "data:image/svg+xml;base64," + btoa(svgData);
$scope.setWidgetProp(sampleImgWidget,'src',svgURL)
}
Here I consider it as static table but you can of course generate it dynamical using the data sets. So the mentioned code example in preview:
I attached the test project , possibly could be helpful for you. Thanks
Hi @aletenti ,
unfortunately I did not have idea how to do this with the image widget (SVG) - because , I do not know how to interpret the click with the 3D coordinates on the 3DImage. So if somebody know how to do this, so in this case this could work and we can check in which ratio the click was on the 3DWidget - in this case we can interpret the click and find out which row/column was indented to be selected by the click. I believe that this should be possible but have no idea how to do this.
What I did in similar case - I did have a panel with specific number of widgets as rows and columns which I used to display a list of sequences (12 where I have a list of 5 rows) and tried to use some fake "scroll" where I change already existing element depending what they should display. So possibly this approach could be a solution but it requires more intensive program coding:
I attached a test project. there is watch construct which is looking for app parameter "CurrentStep" and when there is an change it will change the display of the list accordingly. It check the step number and will calculate dynamically the widget display
$scope.$watch('app.params.CurrentStep', (step)=>{
let tS=step
let numberOfSteps=$scope.app.NumbersOfSteps
if(step <=2) return;
else if(step >= numberOfSteps-1) return;
else{
$scope.app.prevStep2=tS-2
$scope.app.prevStep=tS-1
$scope.app.nextStep=tS+1
$scope.app.nextStep2=tS+2
}
console.warn("changed::")
console.log("$scope.app.params.CurrentStep="+$scope.app.params.CurrentStep)
console.log("$scope.app.prevStep="+$scope.app.prevStep)
console.log("$scope.app.nextStep="+$scope.app.nextStep)
console.log(" $scope.app.NumbersOfSteps="+ $scope.app.NumbersOfSteps)
$scope.app.updateProps($scope.app.prevStep,$scope.app.params.CurrentStep,$scope.app.nextStep, $scope.myFile)
})
In your case you can watch for another parameter where for example a TWX service will write there a json list
When there is a change you need to set the widgets names - and also the action what the widget will do according to the list content. Such solution could be implemented , no problem, but I think it will require significant more programming
the question here is , when I understand correctly the request , then when we click on a 3D element like this:
it is not how to identify which widget we did click as element , let say 3D Widget Image (we can get this easely by the userpick event) but what is required is the information what we clicked. As here shown in the picture - the click means selected is the second row and second column selected. And possibly we want to have the value used there. At least, I think, this was the original question in the topic -> required some functionality like a repeat element in 3D space. So the picture will show that to display a list is no an issue e.g. as SVG in Image but possibly as it was mentioned in the last post here as html.
The question what I see, is how to handle the selection - to find what was selected on the canvas.
The only idea I have is to use in 3D Space the eye vector of the device
tml3dRenderer.setupTrackingEventsCommand (function(target,eyepos,eyedir,eyeup) {
// handle here the eyepos and calculate the intersection point with the plane
// of the panel, use x,y,z, rx, ry,rz to calcualte the plane eq.
})
and then to calculate the point where the eyevector will intersect the plane of the 3dWidget , let say 3DImage. Then we will have a 3D point in space which we could interpret as ratio in relation to the position and the size of the 3Dimage (we need to know how many items , rows or column are displayed there) and so possibly we will then have idea what we did select.
I hope somebody has better idea. Thanks
I wasn't sure if the idea what I mentioned in my last post would work. That's why I was curious to test it. I tested it and it was working. I didn't want to invest so much time in it, so the calculation is not really precise, but I wanted to test the principle - and it seem to works. It's not very performant but it works fluidly on the HOloLens 2 device with the current Vuforia View version - so that it was no issue with the display , no performance concern. OK such workflow should not be active the whole time because it did consider every tracking event and will perform some no trivial calculations.Therefore should be activated only temporary when a selection is required. So here in preview the test but I tested on HL and the resutls are better then in preview mode:
The calculation is a vector calculation with moderate compexity:
// eq
// calculate tranform of the widget from x,y,z,rx,ry,rz calling $scope.euler2trfMatR
//calculate then the inverse matrix
// calculate from tranform then the vectors plane_X_Vec and plane_Y_Vec
// planeX_Vec = inversMat * [1,0,0]T
// planeY_Vec = inversMat * [0,1,0]T
// Norm_plane = norm(plane_X_vec X plane_Y_vec) here is X for VectorProduct von X and Y plane
// d=(p-p0)*n/(l*n) //according Wiki
// in javascript looks like this below
// let dist= dot_product( ( new Vector(planeX) ).minus(DEVICE_Location).retArray(),Norm_plane)/dot_product( eyeVecNorm,Norm_plane)
point_intersect = (norm(EYE_VEC)*dist +DEVICE_LOCATION)
so I used as "cursor" a model which I set to the position of the intersection point with the plane where the Image is located.
This all was embeded in the $scope.app.IntersectionEYEvectorTo3DWidget callback. so in the example:
var arrArg=[]
let propList=['x','y','z','rx','ry','rz']
// let argStr=""
propList.forEach((si)=>{ arrArg.push($scope.view.wdg[widgetId][si]);})
// evalStr= "arrArg.push($scope.view.wdg['"+widgetId+"']."+si+");" ; eval(evalStr) })
let transfMat = euler2trfMatRarr(arrArg);
let transfMatInv = matrix_invert(transfMat)
let X_plane = new Vector( [1.0,0.0,0.0]).norm()
let Y_plane = new Vector( [0.0,1.0,0.0]).norm()
let N_plane = X_plane.prod(Y_plane).norm()
let dist = X_plane.minus(new Vector(locVec)).dot(N_plane) / (new Vector(eyeVec)).norm().dot(N_plane)
console.warn("dist="+dist)
let vec_intersect = (( (new Vector( eyeVec)).norm().scale(dist) ).add( new Vector(locVec) ))
$scope.app.pointInts=vec_intersect.retArray()
let point_intersect_projection= vec_intersect.matTr( transfMatInv).retArray();
console.warn(" point_intersect_projection==> ", point_intersect_projection)
$scope.app.pointIntsProj=point_intersect_projection
let count=0
let val=0
propList.forEach((el)=>{
if( el.indexOf("r") == -1) val =vec_intersect.retArray()[count]; // x,y,z
else val = $scope.view.wdg[widgetId][el]; //rotaiton rx,ry,rz
$scope.setWidgetProp("csys",el,val) ; count++;
})
$scope.$applyAsync();
and then the projection of the intersction point x,y is used for rundementery/not exact decsion abou the selected row column - something like this:
$scope.sampleSVG1 = function(text) { console.log("$scope.sampleSVG1")
let sampleImgWidget = '3DImage-2';
let table_Xmin = -0.34579918
let table_Xmax = -0.1663985
let table_Ymin = 0.0411993
let table_Ymax = 0.10022511
let deltaX= (table_Xmax-table_Xmin)/2.3
let deltaY= (table_Ymax-table_Ymin)/3
let boderXmax = -0.1263985
let boderXmin = - 0.38579918
let boderYmax = 0.14022511
let boderYmin = 0.0011993
let cX = $scope.app.pointIntsProj[0]
let cY = $scope.app.pointIntsProj[1]
let pnt_inside =false
console.log("cX=",cX," <> cY=",cY)
if (cX> boderXmin)
if (cX< boderXmax)
if(cY> boderYmin)
if(cY< boderYmax)
pnt_inside = true;
console.log("pnt_inside="+pnt_inside)
let color1= "gainsboro"
if(pnt_inside) color1="#AB7C94";
console.log("color1="+color1)
let column=0
let row=0
if(pnt_inside) {
row= Math.floor( (boderYmax-cY) /deltaY)-1
column= 4-Math.floor( (boderXmax-cX) /deltaX)
console.warn("row="+row+"<> column="+column)
$scope.app.msg="selected : column="+column+" || row ="+row;
}
else
$scope.app.msg="NOTING SELECTED!";
$scope.setWidgetProp("3DLabel-1","text", $scope.app.msg);
for(c=0; c<4;c++)
for(r=0; r<4;r++)
{ let ev_str="$scope.app.r"+r+"c"+c+"= \" \"";
eval(ev_str)
}
if(column<=0) column=0;
else
column--;
if(column > 3) column=3;
if(row<=0) row=0;
else
row--;
if(row > 3) row=3;
if(pnt_inside)
{ let ev_str="$scope.app.r"+row+"c"+column+"= \"fill = \'magenta\'\" ";
eval(ev_str)
console.log(ev_str)
}
let svgData = `<?xml version='1.0' standalone='no'?>
<!DOCTYPE svg PUBLIC '-//W3C//DTD SVG 1.1//EN'
'http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd'>
<svg width='900' height='300' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink'>
<title>SVG Table</title>
<g id='columnGroup'>
<rect x='65' y='10' width='75' height='110' fill='${color1}'/>
<rect x='265' y='10' width='75' height='110' fill='${color1}'/>
<text x='30' y='30' font-size='18px' font-weight='bold' fill='crimson'>
<tspan x='30' dy='1.5em' ${$scope.app.r0c0} >Q1</tspan>
<tspan x='30' dy='1em' ${$scope.app.r1c0} >Q2</tspan>
<tspan x='30' dy='1em' ${$scope.app.r2c0} >Q3</tspan>
<tspan x='30' dy='1em' ${$scope.app.r3c0} >Q4</tspan>
</text>
<text x='100' y='30' font-size='18px' text-anchor='middle'>
<tspan x='100' font-weight='bold' fill='crimson'>Sales</tspan>
<tspan x='100' dy='1.5em' ${$scope.app.r0c1} >$ 223</tspan>
<tspan x='100' dy='1em' ${$scope.app.r1c1}>$ 183</tspan>
<tspan x='100' dy='1em' ${$scope.app.r2c1}>$ 277</tspan>
<tspan x='100' dy='1em' ${$scope.app.r3c1}>$ 402</tspan>
</text>
<text x='200' y='30' font-size='18px' text-anchor='middle'>
<tspan x='200' font-weight='bold' fill='crimson'>Expenses</tspan>
<tspan x='200' dy='1.5em' ${$scope.app.r0c2}>$ 195</tspan>
<tspan x='200' dy='1em' ${$scope.app.r1c2}>$ 70</tspan>
<tspan x='200' dy='1em' ${$scope.app.r2c2}>$ 88</tspan>
<tspan x='200' dy='1em' ${$scope.app.r3c2}>$ 133</tspan>
</text>
<text x='300' y='30' font-size='18px' text-anchor='middle'>
<tspan x='300' font-weight='bold' fill='crimson'>Net</tspan>
<tspan x='300' dy='1.5em' ${$scope.app.r0c3}>$ 28</tspan>
<tspan x='300' dy='1em' ${$scope.app.r1c3}>$ 113</tspan>
<tspan x='300' dy='1em' ${$scope.app.r2c3}>$ 189</tspan>
<tspan x='300' dy='1em' ${$scope.app.r3c3}>$ 269</tspan>
</text>
</g>
</svg>`;
//console.log(svgData)
let svgURL = "data:image/svg+xml;base64," + btoa(svgData);
$scope.setWidgetProp(sampleImgWidget,'src',svgURL)
}
//'''''''''''''''''''''''''''''''''''''''''''''''''''''''''
so that the effect /not very accurate but for the validation test is OK/ that the row/column is displayed in magenta - when there is e.g. double tap then the e.g. specific action is done for row-X column-Y
