delicatessen.derivative.approx_differentiation

approx_differentiation(xk, f, epsilon=1e-09, method='capprox')

Numerical approximation to compute the gradient. This function implements numerical approximation methods for derivatives (i.e., it provides the first-order forward, backward, and central difference approximations). This function is an alternative to SciPy’s approx_fprime (which only offers forward difference).

Note

This functionality is only intended for use behind the scenes in delicatessen.

The forward difference approximation is

\[\frac{f(x + \epsilon) - f(x)}{\epsilon}\]

the backward difference approximation is

\[\frac{f(x) - f(x - \epsilon)}{\epsilon}\]

and the central difference approximation is

\[\frac{f(x + \epsilon) - f(x - \epsilon)}{2\epsilon}\]

Here, the numerical approximation is implemented by generating matrices for output from a function evaluated under minor perturbations (determined by epsilon) of each input argument. These matrices are then subtracted from each other and then scaled by epsilon.

Parameters

xk (ndarray, list, shape (n, )) – Point(s) or coordinate vector to evaluate the gradient at.
f (callable) – Function of which to estimate the gradient of.
epsilon (float, optional) – Increment to perturb the points by to compute the gradient. This should be a small value
method (str, optional) – Approximation to use to compute the gradient. Default is capprox which uses the central difference method. One can also specify the forward difference (fapprox) or backward difference (bapprox) methods.

Returns

Corresponding array of the pairwise derivatives for all different input x values.

Return type

numpy.array

Examples

>>> import numpy as np
>>> from delicatessen.derivative import approx_differentiation

To illustrate use, we will compute the derivative of the following function

\[f(x) = x^2 - x^1 + sin(x + \sqrt{x})\]

>>> def f(x):
>>>     return x**2 - x + np.sin(x + np.sqrt(x))

If you work out the deriative by-hand, you will end up with the following

\[2x - 1 + \left( \frac{1}{2 \sqrt{x}} + 1 \right) \cos(x + \sqrt{x})\]

Instead, we can use the central difference approximation to evaluate the derivative at a specific point. Here, we will evaluate the derivative at \(x=1\)

>>> dy = approx_differentiation(xk=[1, ], f=f)

which returns 0.37578, which is close to plugging in \(x=1\) into the previous equation.

The derivative of a function with multiple inputs and multiple outputs can also be evaluated. Consider the following example with three inputs and two outputs

>>> def f(x):
>>>     return [x[0]**2 - x[1], np.sin(np.sqrt(x[1]) + x[2]) + x[2]*(x[1]**2)]

>>> approx_differentiation(xk=[0.7, 1.2, -0.9], f=f, method='fapprox')
>>> approx_differentiation(xk=[0.7, 1.2, -0.9], f=f, method='bapprox')
>>> approx_differentiation(xk=[0.7, 1.2, -0.9], f=f, method='capprox')

which will each return a 2-by-3 array of all the x-y pair derivatives at the given values. Here, rows correspond to the output and the columns correspond to the inputs.