Matrix derivatives cheat sheet, Cheat Sheet of Mathematics

Download Matrix derivatives cheat sheet and more Cheat Sheet Mathematics in PDF only on Docsity! Matrix derivatives cheat sheet Kirsty McNaught October 2017 1 Matrix/vector manipulation You should be comfortable with these rules. They will come in handy when you want to simplify an expression before differentiating. All bold capitals are matrices, bold lowercase are vectors. Rule Comments (AB)T = BTAT order is reversed, everything is transposed (aTBc)T = cTBTa as above aTb = bTa (the result is a scalar, and the transpose of a scalar is itself) (A + B)C = AC + BC multiplication is distributive (a + b)TC = aTC + bTC as above, with vectors AB 6= BA multiplication is not commutative 2 Common vector derivatives You should know these by heart. They are presented alongside similar-looking scalar derivatives to help memory. This doesn’t mean matrix derivatives always look just like scalar ones. In these examples, b is a constant scalar, and B is a constant matrix. Scalar derivative Vector derivative f(x) → dfdx f(x) → df dx bx → b xTB → B bx → b xTb → b x2 → 2x xTx → 2x bx2 → 2bx xTBx → 2Bx For a more comprehensive reference, see https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook. pdf 1