Solved: Optimization problem using minimize

DaveWooff · ‎Apr 19, 2012

Hello all,

I'm trying to minimize the error in a function and have started with what I know to be fairly optimal guess values (achieved by using known good values and help from the Excel solver although it seems fairly limited in what it can do). It doesn't seem to work very well though. If I change one of my initial guess values it doesn't optimize back to the original optimal value.

Is it because I've set the problem up incorrectly or because the type of problem is not easily solved without further constraints etc. or something else I don't appreciate?

The mathcad doc attached is set up in the initial optimized state. If you alter the first element of vector "TCa" or "TCb" slightly, you will see the error curve in the first set of plots worsen. However, I would expect the optimization to have a good go at getting back to the optimal state, but it doesn't seem to.

I've tried to lay things out as neatly as possible but if there is any ambiguity please let me know,

Any advice gratefully received.

D

RichardJ · ‎Apr 20, 2012

You only think you solved it

You have a couple of problems

First, in the minerr version you are trying to minimize the maximum error. That's a very unusual error metric, and is likely to lead to strange behavior. The maximum error could jump around as it switches from one data point to another, and iterative solvers do not like objective functions that jump around. You should minimize the sums of the squares (Chi_Sq in the attached worksheet), to get the least square solution. That is what genfit does.

Second, minimize is lacking the best of the optimization algorithms, Levenberg Marquardt (which is what genfit uses). So even if you do minimize the sums of the squares, it's not very reliable. You should use minerr instead, in which case you simply pass it the vector of residuals (do not sum and square the residuals yourself!). You can add constraints to minerr, but they are treated as soft constraints, not hard constraints, and the errors in the constraints are just minimized along with the residuals. To make them hard constraints weight them very heavily.

Now you have a solution complete with constraints

View solution in original post

DaveWooff · ‎Apr 20, 2012

It's OK, I solved it using genfit (probably best function to use for this kind of problem).

Thanks to anyone who was thinking of replying anyway!

MikeArmstrong · ‎Apr 20, 2012

David,

If you don't mind, would you post the worksheet with the solution in for completes.

Mike

RichardJ · ‎Apr 20, 2012

You only think you solved it

You have a couple of problems

First, in the minerr version you are trying to minimize the maximum error. That's a very unusual error metric, and is likely to lead to strange behavior. The maximum error could jump around as it switches from one data point to another, and iterative solvers do not like objective functions that jump around. You should minimize the sums of the squares (Chi_Sq in the attached worksheet), to get the least square solution. That is what genfit does.

Second, minimize is lacking the best of the optimization algorithms, Levenberg Marquardt (which is what genfit uses). So even if you do minimize the sums of the squares, it's not very reliable. You should use minerr instead, in which case you simply pass it the vector of residuals (do not sum and square the residuals yourself!). You can add constraints to minerr, but they are treated as soft constraints, not hard constraints, and the errors in the constraints are just minimized along with the residuals. To make them hard constraints weight them very heavily.

Now you have a solution complete with constraints

DaveWooff · ‎Apr 20, 2012

Superb, thank you Richard.

Yes, that works a treat and also it handles larger values of N whereas my attempt didn't and was also very dependent on initial values as I found when I started to experiment a little.

David.

DaveWooff · ‎Apr 20, 2012

>First, in the minerr version you are trying to minimize the maximum error. That's a very unusual error metric

I meant to add that I was originally trying to obtain a minimax approximation which is why I took that approach.

DaveWooff · ‎Apr 20, 2012

I have attached the worksheet solution.

genfit returns signed "optimal" coefficients so I have taken the magnitude of the coefficients in the original function to deal with this. I don't see a way to constrain the results of genfit.

DaveWooff · ‎Apr 24, 2012

I tried this solution in Mathcad 2001i as well as Mathcad 15 and it doesn't optimise anywhere near as well using the former. Does anyone know if the MinErr function been improved significantly between these two versions? From the help it seems that they both use the same method.

RichardJ · ‎Apr 24, 2012

In 2001i right click on the "minerr" and force the algorithm to Levenberg-Marquardt. This is not necessary in MC14 or MC15 but is essential in some earlier versions (MC12 and earlier for sure; I'm not sure about MC13).

DaveWooff · ‎Apr 24, 2012

Got it. Thaks very much