Two new direct minimum search procedures for functions of several variables
Bruno F. W. Witte, William R. Holst · 1964
The method can somewhat vaguely be classified as a modified "steepest descent." It is, of course, an iterative procedure. Each cycle, to be iterated on, consists essentially of two parts: in Part I a "best" line is found, in Part II an attempt is made to minimize the given function along this line. Thus, each cycle resembles the corresponding cycle in the method of steepest descent. The method of steepest descent differs from our method in the manner in which in Part I the "best" line is found. Steepest descent, in fact, implies that this line (let us call it the "baseline" from now on) be defined by the starting point for the cycle and the gradient of the function at this starting point, where the starting point, in turn, is the minimum point of the preceding cycle. Well known modifications of the steepest descent are concerned with, for example, baselines restricted to a subspace normal to the preceding baseline, or with different ways of minimizing along a given baseline, or perhaps with the question as to how feasible it is to seek a minimum at all along a given baseline during each cycle before switching to the next baseline.