|
Download the Fast Forward
Automatic Differentiation Library
Download
the User Manual for the Fast Forward Automatic Differentiation
Library
This Fast Forward Automatic Differentiation
Library provides a toolkit for computing values of functions
and their derivatives up to specified order at a given point.
The library routines restrict neither the number of independent
variables nor the order of the derivatives. The library supports
several configurations optimized for speed, memory, and debugging.
Automatic differentiation is a broad field
which includes methods of computing derivatives of functions
represented by computer programs. The automatic differentiation
in contrast with symbolic differentiation propagates numerical
values of the derivatives rather symbolic expressions.
Our automatic differentiation approach [6]
is based on the notion of a differential tuple (the
notion of differential tuple was introduced by Shevchenko
in [1], however Neidinger calls a similar
data structure a "PYRAMID" [2]) -- a specialized
data structure that holds values of the partial derivatives
up to the specified order, binomial coefficients, and index
arrays. Differentiation of a composite function is performed
in the forward mode: first differential tuples for independent
variables are generated, then they are propagated through
the computational graph of the function. At each node of the
graph,a chain rule is applied to the inputted tuples and the
resulting tuple is sent to the next node.
The automatic differentiation algorithms are
implemented through overloaded elementary functions (exp,
pow, sin, cos, etc.), arithmetic operations(+, -, *, /), various
utility functions including differentiation operators, and
a function to generate differential tuples corresponding to
independent variables [6]. Overloading,
in conjunction with our automatic differentiation technique,
allows us to make differentiation easy; the partial derivatives
of a function are generated as a tuple by simply writing the
function in the usual notation (ex. f = pow(cos(x),n)). Moreover,
we do not need to explicitly analyze the computational graph
of the function being differentiated: it is constructed by
the compiler and the intermediate differential tuple contains
the "history" of the previous computations.
Our automatic differentiation technique utilizes
the generalized Leibnitz rules, which were derived for many
independent variables by Manko [3,4],
Shevchenko [1, 5].
The same formulas were obtained independently by Neidinger
in [2]. The beauty of the generalized
Leibnitz chain rules is the fact that these rules give the
exact representation of derivatives combining derivatives
of the arguments with binomial coefficients.
For more information see references [6,
7]. |
