线性代数基础

Matrix derivatives

For a function $f:{\mathbb{R}}^{n\times d}\mapsto \mathbb{R}$ mapping from $n$-by-d matrices to the real numbers, we define the derivative of $f$ with respect to $A$ to be:

$${\nabla }_{A}f(A)=[\begin{array}{ccc}\frac{\partial f}{\partial A_{11}}& \cdots & \frac{\partial f}{\partial A_{1d}}\\ ⋮& \ddots & ⋮\\ \frac{\partial f}{\partial A_{n1}}& \cdots & \frac{\partial f}{\partial A_{nd}}\end{array}]$$

Thus, the gradient ${\nabla }_{A}f(A)$ is itself an $n$-by-$d$ matrix, whose $(i,j)$-element is $\partial f/\partial A_{ij}$. For example, suppose $A=\left[\begin{array}{cc}{A}_{11}& A_{12}\\ A_{21}& A_{22}\end{array}\right]$ is a 2-by-2 matrix, and the function $f:{\mathbb{R}}^{2\times 2}\mapsto \mathbb{R}$ is given by

$$f(A)=\frac{3}{2}{A}_{11}+5A_{12}^2+A_{21}{A}_{22}.$$

Here, $A_{ij}$ denotes the $(i,j)$ entry of the matrix $A$. We then have

$${\nabla }_{A}f(A)=[\begin{array}{cc}\frac{3}{2}& 10A_{12}\\ A_{22}& A_{21}\end{array}].$$