Alternative convergence criteria may not work reliably in all cases; misleading output of the minimiser log

[marq]

In the current reverse calling implementation of the minimiser loop, the minimiser returns to the forward model occasionally only to perform a line search. In some cases, this may actually result in an apparent increase of the cost function, although all that the call to the forward model actually does is some task during an ordinary minimisation sequence. M1QN3 calls this a "simulation" in contrast to an "iteration".

The current implementation does not distinguish between these two different modes, and the logging does not reflect the correct interpretation of the cost function evaluation in this case. Additional convergence criteria - which rely on the development of the cost function and the increments of the last few iterations - may also cause misleading results, and in particular may terminate the minimisation loop although trui convergence has not been achieved. The problem occurs if the minimisation problem become more nonlinear, i.e. in particular for bending angle retrievals; refractivity retrievals are most often ok.

The proper solution is to understand the reverse communication between minimiser loop and minimiser so that the calling routine interpretes calls to the forward model / cost function evaluation correctly.

Until that has been implemented, a workaround is to use the minimiser's convergence criteria instead of the alternative criteria, even at the cost af a few additinal iterations.

[Huw Lewis]

M1QN3 minimiser is replaced in v1.2 by ROPP-specific minimiser minROPP. The 'alternative' convergence criteria are considered only in this case - check change in J and change in state vector between iterations.

[Huw Lewis]

New ROPP-specific minimiser minROPP committed to trunk and tested successfully. Also providing Levenberg-Marquardt algorithm (using same 'alternative' convergence criteria). [1595]

Ticket closed.