# Differentiation

## The derivative of a function from a single variable

The derivative of a function is the amount of change in the function at a given point. f(x) shows the change of f at infinitesimal x.

Given a certain function f: U → R defined on the open interval U ⊂ R. Given a point a ∈ U. The derivative of the function f at point a is equal to the limit:

$$f'(a)=\lim_{h \to 0} \left(f(a+h)-f(a)\over h\right)$$

## Derivative in the direction

Such a derivative shows the increment of the function along a given direction.

Given a set U ⊂ R^{2} and a function f : U → R. Given a certain point (a,b)∈U
and the unit vector **u** = (c,d)∈R^{2}. The derivative of the function f in the direction of the vector
**u** at the point (a,b) is equal to:

$$\vec{u} = cos\theta\vec{i} + sin\theta\vec{j} $$

$$D_u f(a,b) = \lim_{h \to 0} \left( f(a + h \cdot cos\theta, b + h \cdot sin\theta) - f(a,b) \over h \right) $$

## Partial derivatives

For the open interval U⊂ R^{2} and the function f: U → R is the partial derivative of x -
this is the derivative in the direction of the vector (1,0), i.e.:

$$ \frac{df}{dx} (a, b) = \lim_{h \to 0} \frac{f(a+h,b) - f(a,b)}{h} $$

Similarly, the derivative in y is the derivative in the direction (0,1):

$$ \frac{df}{dy}(a,b)=\lim_{h \to 0}\frac{f(a,b+h)-f(a,b)}{h} $$

In general, for an n-dimensional space, the partial derivative with respect to the i-th variable, df/dx_{i},
is the limit:

$$ \frac{df}{dx_i}=\lim_{h\to0}\frac{f(x_1,x_2,...,x_i+h,...,x_n)-f(x_1,...,x_n)}{h} = \lim_{h\to0}\frac{f(x+he_i)-f(x)}{h} $$ where 1≤i≤n and the vector e_{1}is the vector of the standard basis: e_{i}=(0,...,1,...,0) with a unit at position i.

## Tangent plane to the surface

Let be given a function f: U ⊂ R^{2} → R and a point (a,b) ∈ U. If there is a tangent plane
at the point (a,b,f(a,b)), then it is defined by the equation:

$$ z = P_{(a,b)}f(x,y)=f(a,b)+\frac{df}{dx}|_{(a,b)}(x-a)+\frac{df}{dy}|_{(a,b)}(y-b) $$

Let the function f be given: U ⊂ R^{2} → R, the differential of the function at point (a,b) will be the expression:

$$D_{(a,b})f(x,y) = \frac{df}{dx}|_{(a,b)}(x-a) + \frac{df}{dy}|_{(a,b)}(y-b)$$

## Differentiability of the function

The function f(x,y) is differentiable in (a,b) if there are partial derivatives and the following limit:

$$ \lim_{(x,y)\to(0,0)}\frac{f(x,y)-(f(a,b)+D_{(a,b)}f(x,y))}{||(x,y)-(a,b)||} = 0 $$

## Gradient of the function

The gradient vector of the function f:U⊂R^{n}→R at point a:

$$ \nabla f(a) = (\frac{df}{dx_1}(a), ..., \frac{df}{dx_n}(a))$$

**The theorem.** If the function f:U⊂R^{n}→R is differentiable, then there are
partial derivatives in any direction and are defined as D_{v}f(a)=∇f(a)⋅**v**

## Higher order partial derivatives *mixed derivatives*

The differentiable function f is given:U⊂R^{n}→R, df/dx_{i}: U⊂ R^{n}→R,
the second-order derivative is defined as: d/dx_{j}(df/dx_{i})(x_{i}, ..., x_{n}) =
d^{2}f/dx_{j}dx_{i}(x_{i}, ..., x_{n}). The same
thus, higher-order partial derivatives are formed.

If a function f is differentiable up to the power of n and all partial derivatives up to the power of n are continuous, then
the function f belongs to the class n: f∈C^{n}

**Schwarz's theorem**: mixed partial derivatives of the same function, differing only in the order of differentiation,
are equal to each other, provided they are continuous.

## Jacobi matrix

Let be given a vector function f: U⊂ R^{n} → R^{m}, f=(f_{1},...,f_{m})
and there are quotients derivatives f_{1},...,f_{m} at point a∈U, Jacobi matrix
for f at point a will be defined as:

$$ J_{f}(a) = \left[ {\begin{array}{ccc} \frac{df_1}{dx_1}(a) & ... & \frac{df_1}{dx_n}(a) \\ ... & ... & ... \\ \frac{df_n}{dx_1}(a) & ... & \frac{df_n}{dx_n}(a) \\ \end{array} } \right] $$

A function f is differentiable at point a if there exists a Jacobi matrix at point a and the limit is:

$$ \lim_{x\to a}\frac{||f(x)-f(a)-J_{f}(a)(x-a)||}{||x-a||} = 0 $$