Minerr: Quasi-Newton, Levenberg-Marquardt, Conjugate Gradient?

[mnori]

How to choose one of the  Minerr Options? How initial values and Objective functions are affected by the selected Method? Are there Mathcad alternative solutions for non-linear regression of parameters?

[ValeryOchkov]

Only in Mathcad 15

See please

Решение алгебраических уравнений и систем

[RichardJ]

For non-linear regression (i.e. fitting a curve to data) you should use the L-M algorithm. The objective function should generate a vector of residuals (do NOT sum and square them!).

Mathcad really offers nothing else for this. The genfit function also uses the L-M algorithm, and is not as flexible as minerr. Simulated annealing was available at one point, with an extension pack, but that's unfortunately long gone. Is there a reason you want something other than the L-M algorithm? It's old, but it's very good.

[mnori]

I am using Minerr for the non-linear regression of the parameters of a Volumetric Equation of State. I read that Quasi-Newton could be a good option for time consuming solution.

[RichardJ]

I have a paper (although a very old one) that shows that the LM algorithm significantly out performs other downhill algorithms for fitting of data. The algorithm doesn't know what it's minimizing though, so the only difference between fitting data and any other minimization problem would be the number of residuals (when fitting data there are many: one for each data point). One reason LM is fast is that it adapts the step size as it iterates to the solution. It takes big steps to begin, and avoids overshoot by reducing them as it approaches the minimum.

[ValeryOchkov]

See please Why minimize not work in this Mathcad Prime 3.0 sheet?

[RichardJ]

Yes, that's another desirable property of the LM algorithm. It inherently re-scales the parameter space to avoid the numerical problems Brent mentions. It seems the Knitro solvers probably don't, and that's why minerr works when minimize doesn't. I added a comment to that effect in the thread.

[mnori]

Thank you Richard,

In case of multiple object functions in a solve block, how the Minerr convergence tolerances work ( TOLL and CTOL )? Are the same criteria applied with LM or Quasi-Newton?