6. Implementation and Software
Society for Industrial and Applied Mathematics eBooks · 2008
The techniques explained in Chapters 3 and 5 reveal a sequence of operations that the transformed program must perform in order to calculate the correct derivative values. The insight that the AD framework gives us is valuable even if we propose to carry out the program transformation by modifying the source code manually. However, manual transformation is time consuming and prone to error. It is also hard to manage; if the underlying program is subject to continual change, then keeping track of these changes, in order to maintain the integrity of the transformed version, is also time consuming. Therefore, it is usually desirable to automate at least partially the process of transformation. Regardless of how automatically, i.e., with what level of automation, program transformation is done there is a trade-off between the sophistication of the transformation process and the efficiency with respect to time and space bounds of the transformed program. As a rule, a general-purpose AD tool will not produce transformed code as efficient as that produced by a special-purpose translator designed to work only with underlying code of a particular structure, since the latter can make assumptions, often with far-reaching consequences, whereas the former can only guess. In many cases, an unsophisticated approach suffices to produce AD code that is within a constant factor of the optimal performance bounds. In this situation, subsequent effort is devoted just to reducing the value of the constant. In other cases, a careful analysis of the structure of the code will reveal that several orders of magnitude can be gained by the use of more sophisticated techniques, such as preaccumulation (see section 10.2), or by the careful exploitation of by-products of the underlying computation, such as LU decompositions, in the derivative process (see Exercise 4.6). Such transformation processes may involve a substantial amount of human intervention, including some modification of the code, or a much greater degree of sophistication in the design of the automated tools. For example, our primary motivation might be the wish to explore new models or algorithms to see whether they merit further investigation.