Differentiation
The derivative of a function from a single variable
The derivative of a function is the amount of change in the function at a given point. f(x) shows the change of f at infinitesimal x.
Given a certain function f: U → R defined on the open interval U ⊂ R. Given a point a ∈ U. The derivative of the function f at point a is equal to the limit:
$$f'(a)=\lim_{h \to 0} \left(f(a+h)-f(a)\over h\right)$$
Derivative in the direction
Such a derivative shows the increment of the function along a given direction.
Given a set U ⊂ R2 and a function f : U → R. Given a certain point (a,b)∈U and the unit vector u = (c,d)∈R2. The derivative of the function f in the direction of the vector u at the point (a,b) is equal to:
$$\vec{u} = cos\theta\vec{i} + sin\theta\vec{j} $$
$$D_u f(a,b) = \lim_{h \to 0} \left( f(a + h \cdot cos\theta, b + h \cdot sin\theta) - f(a,b) \over h \right) $$
Partial derivatives
For the open interval U⊂ R2 and the function f: U → R is the partial derivative of x - this is the derivative in the direction of the vector (1,0), i.e.:
$$ \frac{df}{dx} (a, b) = \lim_{h \to 0} \frac{f(a+h,b) - f(a,b)}{h} $$
Similarly, the derivative in y is the derivative in the direction (0,1):
$$ \frac{df}{dy}(a,b)=\lim_{h \to 0}\frac{f(a,b+h)-f(a,b)}{h} $$
In general, for an n-dimensional space, the partial derivative with respect to the i-th variable, df/dxi, is the limit:
$$ \frac{df}{dx_i}=\lim_{h\to0}\frac{f(x_1,x_2,...,x_i+h,...,x_n)-f(x_1,...,x_n)}{h} = \lim_{h\to0}\frac{f(x+he_i)-f(x)}{h} $$ where 1≤i≤n and the vector e1 is the vector of the standard basis: ei=(0,...,1,...,0) with a unit at position i.
Tangent plane to the surface
Let be given a function f: U ⊂ R2 → R and a point (a,b) ∈ U. If there is a tangent plane at the point (a,b,f(a,b)), then it is defined by the equation:
$$ z = P_{(a,b)}f(x,y)=f(a,b)+\frac{df}{dx}|_{(a,b)}(x-a)+\frac{df}{dy}|_{(a,b)}(y-b) $$
Let the function f be given: U ⊂ R2 → R, the differential of the function at point (a,b) will be the expression:
$$D_{(a,b})f(x,y) = \frac{df}{dx}|_{(a,b)}(x-a) + \frac{df}{dy}|_{(a,b)}(y-b)$$
Differentiability of the function
The function f(x,y) is differentiable in (a,b) if there are partial derivatives and the following limit:
$$ \lim_{(x,y)\to(0,0)}\frac{f(x,y)-(f(a,b)+D_{(a,b)}f(x,y))}{||(x,y)-(a,b)||} = 0 $$
Gradient of the function
The gradient vector of the function f:U⊂Rn→R at point a:
$$ \nabla f(a) = (\frac{df}{dx_1}(a), ..., \frac{df}{dx_n}(a))$$
The theorem. If the function f:U⊂Rn→R is differentiable, then there are partial derivatives in any direction and are defined as Dvf(a)=∇f(a)⋅v
Higher order partial derivatives mixed derivatives
The differentiable function f is given:U⊂Rn→R, df/dxi: U⊂ Rn→R, the second-order derivative is defined as: d/dxj(df/dxi)(xi, ..., xn) = d2f/dxjdxi(xi, ..., xn). The same thus, higher-order partial derivatives are formed.
If a function f is differentiable up to the power of n and all partial derivatives up to the power of n are continuous, then the function f belongs to the class n: f∈Cn
Schwarz's theorem: mixed partial derivatives of the same function, differing only in the order of differentiation, are equal to each other, provided they are continuous.
Jacobi matrix
Let be given a vector function f: U⊂ Rn → Rm, f=(f1,...,fm) and there are quotients derivatives f1,...,fm at point a∈U, Jacobi matrix for f at point a will be defined as:
$$ J_{f}(a) = \left[ {\begin{array}{ccc} \frac{df_1}{dx_1}(a) & ... & \frac{df_1}{dx_n}(a) \\ ... & ... & ... \\ \frac{df_n}{dx_1}(a) & ... & \frac{df_n}{dx_n}(a) \\ \end{array} } \right] $$
A function f is differentiable at point a if there exists a Jacobi matrix at point a and the limit is:
$$ \lim_{x\to a}\frac{||f(x)-f(a)-J_{f}(a)(x-a)||}{||x-a||} = 0 $$