Spark's Work

KL Divergence of Normal Distribution and Matrix Algebra

When working on a research problem, I have had to calculate the Kullback-Leibler Divergence between two multivariate normal distributions.

This should be a straightforward problem of calculation from first principle, but I have always had some trouble with matrix algebra, so it is better to cross-check with others' answers.

Luckily, the question has already been answered on StackExchange (multivariate is here; a special case of univariate is here).

While reading the answer, I also came across this amazing resource called The Matrix Cookbook, which contains a lot of useful results matrix albegra. I am certain that I will come back to this book for reference sometime in the future.