1
$\begingroup$

There is a notation used in many sources (e.g. Wikipedia: http://en.wikipedia.org/wiki/Exponential_family) for the natural parameters of exponential family distributions which I do not understand, and I cannot find a description of.

With vector parameters and variables, the exponential family form has the dot product between the vector natural parameter, ${\boldsymbol\eta}({\boldsymbol\theta})$ and the vector sufficient statistic, ${\mathbf{T}}({\mathbf{x}})$, in the exponent. i.e. $e^{{\boldsymbol\eta}({\boldsymbol\theta})^{\top}{\mathbf{T}}({\mathbf{x}})}$.

However, many examples of these parameters for different distributions are vectors composed of matrices & vectors. E.g. the multivariate Normal distribution has parameter $[\Sigma^{-1}\mu\space\space-\frac{1}{2}\Sigma^{-1}]$ and sufficient statistic $[\mathbf{x}\space\space\mathbf{xx^{\top}}]$.

So what are these "vectors" and moreover, how is the dot product between them defined? Does this notation have a name?

$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I believe you're supposed to "vectorize" the matrix, i.e. rearrange into a $n^2 \times 1$ vector.

Equivalently, you can take $A\cdot B = \textrm{tr}(A^TB)$ as the definition.

EDIT - Example:

If $A = \left(\begin{array}{cc}a & b \\ c & d \end{array}\right)$, $B = \left(\begin{array}{cc}e & f \\ g & h \end{array}\right)$, then

$$ A^TB = \left(\begin{array}{cc}ae + cg & af + ch \\ be + dg & bf + dh \end{array}\right) $$

and the trace is $ae + bf + cg + dh$. Likewise, if we first vectorize the matrices $$ \widetilde{A} = \left(\begin{array}{c}a & b & c & d \end{array}\right)^T\\ \widetilde{B} = \left(\begin{array}{c}e & f & g & h \end{array}\right)^T\\ $$

it's straightforward to see $\widetilde{A}\cdot\widetilde{B} = \textrm{tr}(A^TB)$.

$\endgroup$
4
  • $\begingroup$ I see: vectorizing each element makes sense. However, I don't see how using the trace definition helps since the first element of each vector is a vector, while the second element of each is a matrix. $\endgroup$
    – Chris Burgess
    Commented Oct 10, 2013 at 14:33
  • $\begingroup$ The result in either case is a scalar though. I'll add a quick example to my answer. $\endgroup$
    – BaronVT
    Commented Oct 10, 2013 at 15:06
  • $\begingroup$ Oh, I see what you're saying. If the first entry of the outer matrix is a vector (n x 1), and the second entry is an inner matrix (n x n), then the outer matrix is (n x (n + 1)) and this will still work. (after transposing the first matrix, you'll be multiplying an ((n+1) x n) matrix by an (n x (n+1)) one, the result is ((n+1) x (n+1)), and then you take the trace.) $\endgroup$
    – BaronVT
    Commented Oct 10, 2013 at 15:20
  • $\begingroup$ Ah I see. So I can think of the two outer vectors as essentially defining a pair of block matrices (in each, a column plus a matrix). Then I can use the trace definition and it works equivalently to vectorization. Thanks $\endgroup$
    – Chris Burgess
    Commented Oct 10, 2013 at 16:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .