Jacobians
At its core, the Jacobian matrix encapsulates the rate of change of a vector-valued function concerning its input variables. Represented as a matrix of partial derivatives, the Jacobian elegantly captures the intricate relationship between the input and output of a multivariate function. Mathematically, for a function [Tex]f: \mathbb{R}^n \rightarrow \mathbb{R}^m [/Tex] J is defined as
[Tex]\begin{bmatrix} \frac{\partial f_1}{\partial x_1} & \frac{\partial f_1}{\partial x_2} & \cdots & \frac{\partial f_1}{\partial x_n} \\ \frac{\partial f_2}{\partial x_1} & \frac{\partial f_2}{\partial x_2} & \cdots & \frac{\partial f_2}{\partial x_n} \\ \vdots & \vdots & \vdots & \vdots \\ \frac{\partial f_m}{\partial x_1} & \frac{\partial f_m}{\partial x_2} & \cdots & \frac{\partial f_m}{\partial x_n} \end{bmatrix} [/Tex]
As you can see, each element in the matrix is the partial derivative of the corresponding output function [Tex]f_i [/Tex]with respect to each input variable [Tex]x_j [/Tex]. Therefore, the Jacobian briefly encloses the local sensitivity of each output with respect to changes in the inputs. We can clearly see that with the help of the following example.
Consider the function having the given vector values[Tex] f(x, y) = [2x, 3y][/Tex] . Here, [Tex] f [/Tex] takes in a two-dimensional input [Tex](x, y)[/Tex] and produces a two-dimensional output. In order to compute the Jacobian matrix [Tex]J [/Tex] for this function, we calculate the partial derivatives as follows:
[Tex]J = \begin{bmatrix} \frac{\partial (2x)}{\partial x} & \frac{\partial (2x)}{\partial y} \\ \frac{\partial (3y)}{\partial x} & \frac{\partial (3y)}{\partial y} \end{bmatrix} [/Tex]
After evaluating each element we get,
[Tex]J = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix} [/Tex]
From the results, we understand that a unit change in [Tex]x [/Tex] leads to a 2-unit change in the first output. Similarly, a unit change in [Tex]y[/Tex] corresponds to a 3-unit change in the second output. This might seem like a very basic interpretation but it forms the basis for more complex applications in machine learning, optimization, and sensitivity analysis.
Jacobians in TensorFlow
In the field of machine learning and numerical computation, it is extremely important to understand mathematical concepts. One of such fundamental concepts is Jacobian matrix. It is important part of calculus and has extensive applications in diverse fields. In this article, we will discuss Jacobians and how we can compute Jacobians using TensorFlow.