Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Assume that $V_i,V_j,D$ are all dependent random variables and real-valued, and let the matrix $H$ be defined by

$$ H_{ij} = \mathrm E(V_i V_j) - \mathrm E(\mathrm E(V_i\mid D)\mathrm E(V_j\mid D)) $$

My goal is to determine whether $H$ is positive semi-definite, it is the Hessian of a log-likelihood function that I would like to know whether it is convex.

If I can show that $\langle V_i, V_j \rangle = H_{ij}$ is an inner product then $H$ is Gramian and so it is positive semi-definite.

The first two axioms for an inner-product follows directly

$$ \langle V_i, V_j \rangle = \langle V_j, V_i \rangle $$ $$ \langle aV_i, V_j \rangle = a \langle V_j, V_i \rangle $$

Now, to determine whether the third axiom holds

$$ \langle V_i, V_i \rangle \geq 0 $$

I need to determine whether it is true that

$$ \mathrm E(V_i^2) - \mathrm E(\mathrm E(V_i\mid D)^2) \geq 0 $$

$$ \mathrm E(V_i^2) - \mathrm E(V_i)^2 \geq 0 $$

$$ Var(V_i) \geq 0 $$

From Schwarz' inequality $\mathrm E(V_i)^2 \leq \mathrm E(V_i^2)$, so $\langle V_i, V_i \rangle \geq 0$ is true.

I am quite far from my comfort zone. Is my reasoning OK?

Update: Would also be interesting to hear about other ways that one can prove $H_{ij}$ is positive-semi-definite.

share|improve this question
1  
For SEMI-definite, you don't need to show that $Var(V_i)=0$ off $V_i=0$. –  Fabian Jan 24 '12 at 23:22
    
@Fabian, thanks. –  j-a Jan 25 '12 at 7:56

1 Answer 1

up vote 0 down vote accepted

I still can't see anything wrong with the approach in the question.

I figured out an alternative way to prove it:

$$ H_{ij} = \mathrm E(V_i V_j) - \mathrm E(\mathrm E(V_i\mid D)\mathrm E(V_j\mid D)) $$

Noting that $$ \begin{align} Cov(X,Y|Z) & = E[(X-E[X|Z])(Y-E[Y|Z])] \\ & = E[XY] - E[X|Z]E[Y|Z] \end{align} $$

We get

$$ H_{ij} = E[Cov(V_i,V_j|Z)] $$

Let

$$ A_{ij} = Cov(V_i,V_j|Z) $$

Which is positive semidefinite

$$ x^TAx \geq 0 \quad \forall x $$

also $$ E[x^TAx] =x^TE[A]x = x^THx \geq 0 \quad \forall x $$

So $H$ is positive semidefinite.

share|improve this answer
    
If anyone else answers the original question, I will accept that as the answer instead. –  j-a Jan 28 '12 at 10:01

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.