Skip to main content
ptc-51469131-Visitor
1-Visitor
July 7, 2013
Solved

How to call values from a table and then do the minerr() calculation?

  • July 7, 2013
  • 87 replies
  • 29459 views

Hi:

This is the reply I got from created by VladimirN. in Creo.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

This is a built-in Mathcad function. Minerr(var1, var2, ...) - returns the values of var1, var2, ..., coming closest to satisfying a system of equations and constraints in a Solve Block. Returns a scalar if only one argument, otherwise returns a vector of answers. If Minerr cannot converge, it returns the results of the last iteration.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

I have two questions:

1) As I read the minerr() instruction, var1, var2, .. should be either integer or complex number.

Can var1, var2 be vectors?

2) Is there any way to call var1, var2 from a separated table and then put them into minerr()?

Best answer by Werner_E

Richard Jackson wrote:

speed up the calculation for a larger set of parameters

The time taken for non-linear least squares scales as the square of the number of residuals and as the square of the number of fitted parameters.

As I understand it Chih-Yu Jen will have 25 to be fitted parameters und a residual vector of approx. 1500 elements (or 3000 if real and imaginary parts are stacked as per your suggestion). Every try with a set of different parameters TT will have to call a function make_Theory(TT) which returns a vector of 1500 complex numbers. The method to gain those values is rather complex and would involve creating and multiplying 25 2x2 matrices for every single value of that vector. Some methods how are in this post http://communities.ptc.com/message/211040#211040 and the following (which is more compact, maybe a bit faster and has two errors (m-->M, n-->TT)).

So I think that speeding up make_Theory(TT) would help in speeding up the overall calculation.

Back in the days of mainframe computers that occupied an entire room and had 1/10000th of the processing power of my current smart phone, I ran some least squares optimizations that took two days to converge.

You are showing your age

But two days would be fine, in a prior post in this thread, when Chih-Yu Jen sent some information about calc time with fewer parameters and smaller residual vector, I made an estimation that the calculation with the real values would take up three weeks or 5 and ahalf days with a minor improvement of mine. Not sure if that would have been true - think he didn't bother trying. Guess he will let us know how speedy we got in the meantime.

87 replies

Werner_E
Werner_E25-Diamond I
25-Diamond I
July 7, 2013

For questions about Mathcad its always a good idea to state which version you are using and its always helpful to attach a worksheet which shows what you have done so far.

Unless otherwise stated Mathcad 15 is assumed

1) As I read the minerr() instruction, var1, var2, .. should be either integer or complex number.

Can var1, var2 be vectors?

Yes. The help calls it "array variables".

2) Is there any way to call var1, var2 from a separated table and then put them into minerr()?

Not sure what you mean. You can "call" a function, but not a variable. But the variables can be from any place any other MC variables can ste from - a matrix ("table"?) in the same sheet, from an extern Mathcad sheer or even read in from a CSV or an Excel file.

EDIT: I just had a glance at the thread you are coming from and it seems to me that you misunderstand what "Minerr" would do. You can provide guess values for the variables which should be varied, you may add some constraints but you cannot force Mathcad to consider only values for these variables from a given set of data. Maybe you should give the help a second look and also the quicksheet it referrs to.

What I understood from that post of yours is, that you have a set/vector of X-values and you want to chose any five of them to put them as parameters in function you have defined and a looking for that set of five values for which the function result is closest to a given value.

It looks to me that it needs a self written function to accomplish that.

But then I may be wrong as its not really fully clear to me what you are really after. Think you should post a worksheet and show by example what you are trying to accomplish.

ptc-51469131-VisitorAuthor
1-Visitor
July 8, 2013

Hi:

I hope you can see the below Excel table I posted (also imported to MathCad) and I use MathCad 15.

As you can see, X axis is 1E15, 2E15... and Y axis is frequency (0.3, 0.35....).

1,2,3... is the data.

For the question 1, the var1, var2...I paln to use inside minerr() are the data in 1E15, 2E15.....columns (data: 1,2,3..)

For example, if I define 1E15, 2E15, 3E15 and 4E15 columns as variables, data (1,1,1,1 in frequency 0.3), data (2,2,2,2 in frequency 0.35) will do the calculation separately and then come out a data column like the "t" columnat right side of the table. The calculated "t" column will compare with the measured "t" column in minerr() to find the minimun error.

So you say it can work by using array variables, right?

For the question 2 (I guess it is similar to question 1), my plan is to choose data from this table in each loop (for example, select 1E15==> data in 1E15 column will be used) and then use minerr() to find the variables (say 4 variables) with a minimun error.

In all examples I read, the initial guess value (integer or complex number) is defined first and then LMA in minerr() will determine proper values to find minimum error.

To me, is there any way to select data ( data column) from this table in each loop (this is what I mean "call") and then put them into minerr() to determine proper variable (say 1E15, 3E15, 5E15 and 8E15)?

Since the variables (var2, var2,...) I plan to use are data column, should I use vectors as guess values?

Maybe the table I import shouldn't contain X/Y axis (I am new to the programming).

Frequency1E+152E+153E+154E+155E+156E+157E+158E+159E+15Frequencyt
0.301111111110.301
0.352222222220.352
0.403333333330.403
0.454444444440.454
0.505555555550.505
0.556666666660.556
0.607777777770.607
0.658888888880.658
0.709999999990.709
0.751010101010101010100.7510
0.801111111111111111110.8011
0.851212121212121212120.8512
0.901313131313131313130.9013
0.951414141414141414140.9514
1.001515151515151515151.0015
Werner_E
Werner_E25-Diamond I
25-Diamond I
July 10, 2013

If you still need help post the worksheet as described here http://communities.ptc.com/message/208839#208839

(chose "Use advanced editor" at the upper right).

Especially how the five matrices M1..M5 depend on the five parameters and where these parameters should be taken from. Also info if these parameters have to be all different and in ascending order.

ptc-51469131-VisitorAuthor
1-Visitor
August 1, 2013

Hi:

I have worked in the recent days to come out this initial code.

In this code:

1) Import the file Measure.XLSX which contains 9x1325 database.

2) Use linear interpolation to expand the database and use f_look to choose arbitrary columns in between.

3) Setup 26 initial variables (TTs)

4) These 26 TTs will look up the database to form 26 n's

5) Unitilize these 26 n's to obtain Theory_t and compare with Meaure_t

Now I have proved that the right TTs can give a well overlapped Theory_t and Meaure_t.

My question is: If I choose arbitrary TTs, how can I use minerr() to obtain the right TTs?

My thinking is: minerr(TTs) should be able to find the cloest Theory_t to Meaure_t. The obtained TTs will be close to measured TTs.

During the process, step 4/5 will be repeated again and again until the cloest Theory_t to Meaure_t is found.

I have setup err2G as the sum of the difference bew Theory_t and Meaure_t and then use minerr() to determine Fitted_N (the calculated vector of TTs) based on CN (the vector of initial TTs).

Now the result shows that Fitted_N is always equal to CN (calculated TTs = initial TTs).

But my goal is that Fitted_N is equal to measured TTs.

I know there is are problems in this code but don't know how to fix them.

Can anyone provide a hint?

Werner_E
Werner_E25-Diamond I
25-Diamond I
August 1, 2013

I have no clue what you are doing in this file.

What I can spot without the EXCEL data is, that you have not defined the function M.0() to M.25(), so I wonder where the pictures of the plots stem from.

Some additional ideas:

- Wouldn't it be easier to write T:=stack(1.49*10^19, 1.93*10^19,.....) ?

- Why don't you make n a 2-dimensional array? You could then create a range var j:=0..25 and write n[i,j:=f(TT[j,Y[i)..... and instead of n.4 you would use n<4> where <4> is the column selector.

Werner_E
Werner_E25-Diamond I
25-Diamond I
August 6, 2013

Space is getting so small as of intendation cause the thread is rapidly growing, so I answered to the first post.

I have verified that minerr and minimize should work with vector arguments. At least minimize would require a function returning a scalar rather than e vector. This is best achieved by defining Q(TT):=|maks_Theory(TT)-Measure|, that way also getting rid of any imaginary elements.

Don't know if MinErr would have problems with imaginary elements.

I created a sheet (using a real signal only but you may expand if you like) to show that all methods (Minerr, Find, minimize) will work with a similar setup you have. I generate a signal depending on selectable number of parameters (amplitudes, corresponding to your TT), sample a selectable number of datapoints (Measure), add some noise and try to find the best fit. Find() will only return a result if no noise is added so a perfect fit is possible.

MinErr and Minimize basically return similar results. Thats interesting as MinErr will use LM algorithm by default and Minimize defaults to CG (LM not available here). If I force MinErr tu use CG the results are inferiour. Maybe thats because of the limited number of iterations minErr will do - you wrote its 500 - I was not able to find a number for that limit!? With higher values for number of amps and data points it shows that Minimize will take MUCH longer than MinErr.

Anyway, here is the sheet to play with. Add complex parts if you like but at least its a template how it could/should and does work 😉

And even with 25 amplitudes and 2000 data points its very quick.

I added at the end a time to see how much time a selectable number of calls to make_Theory take. You may do so likewise in your sheet to find out where optimization could help most. Of course the time will vary depending on number of sample points and number of amplitudes.

My machine is very lame and with 25 amplitudes and 2000 points it took about a minute for 1000 calls to make_Theory.

You may also add timing for the solve block to see whats the influence of an increase of parameters and an increase of points.

Of course I would expect longer calculation times in your sheet as make_Theory involves calls to two interpolation routines and a series of matrix operations. But I am not sure if the long times you report are justified by this.

The main problem still is that in your sheet MinErr seems not to change your guess values while it looks like it tries hard to do so (because of the long calc time). Did you try to call your SSE function with the guess values and then again with slightly modified guess values? Did the SSE change? Still thinking it could be some kind of scaling problem - just a vague idea. I remember that Alan Stevens had helped on solve blocks here in the past and used some sort of scaling in his solutions once or twice.

ptc-51469131-VisitorAuthor
1-Visitor
August 6, 2013

Hi:

Thanks a lot. I will read through the code in detail.

You inspired me the question whether minerr() can handle the complex number.

Now I think the answer is: No.

The code didn't work while using Q(TT):=|maks_Theory(TT)-Measure|.

By the use of Q(TT):=maks_Theory(TT)-Measure, the result came out in 30s and was the same as initial TTs (we already know).

When I use Q(TT):=Re(maks_Theory(TT))-Re(Measure), the result came out in 5 mins and gave me a different TTs set (close to my expectation).

When I use below code, the result came out also in 5 mins and gave me a different TTs set (also close to my expectation).

Therefore I think minerr() cannot handle the complex number.

Untitled.png

Now I go back to try 25 TTs and 200 elements and check how long will it take.

I have another question:

Is it possible to setup below restriction to minerr()?

All calculated TTs cannot be larger than the first TTs (TT0) and cannot be smaller tha the last TTs (TT13).

As I know, we cannot guide TTs determined from minerr().

I am asking for comfirmation.

RichardJ19-Tanzanite
19-Tanzanite
August 6, 2013

I've been on an extended business trip and only just got back to looking at this again.

minerr can handle complex numbers, but it does then require complex guesses. Were your guess values complex?

The code in your image is not correct. You do not want to add the residuals for the real and imaginary parts because they could cancel. You should create a vector of residuals that contains both the real and imaginary residuals: stack(Re(make_Theory)-Re(Measure, Im(make_Theory)-Im(Measure))=0

You can use constraints with minerr, but a constraint is not treated as a hard constraint. The error in the constraint is just minimized along with the residuals. To get a hard constraint weight it very heavily. For example, if you want A to be less that B, write the constraint as (A-B)*10^15<0.

Regarding scaling issues, CTOL is an absolute number, not a fractional change in the residuals. If your residuals are very large, for example on the order of 10^12, and CTOL is the default value of 10^-3, you are asking for a relative error of 1 part in 10^15, which is on the order of numeric roundoff. It will either never converge, or it will take a very long time to do so. Make CTOL something realistic compared to the magnitude of your residuals.

Terms & ConditionsAccessibility statement