# Partial derivative respect to random variable - How does one compute this?

CLARIFICATION:

If someone could please help me understand the following: When examining the expected value in this specific situation, how is the distribution of $\theta$ relevant? What difference would it make whether $\theta$ is $10^{10}$ or $10^{-10}$, $-3$, or normal or not?

How does one take a partial derivative with respect to a random variable?

For an assignment, I need to find $E_{Y\mid\theta} \left[- \dfrac{\partial^2}{\partial^2\theta} \ln[P(y\mid\theta)] \right]$, where

$Y \sim N(\theta, 1)$ and $\theta\sim N(0, \delta^2)$, $\delta$ is known

And I realized I do not know how to compute $\dfrac{\partial^2}{\partial^2\theta} \ln[P(y\mid\theta)]$

If I treat $\theta$ as a standard variable and take the second partial derivative of $\ln[P(y\mid\theta)]$ I get a value of $-1$. This does not make sense to me as a correct solution as I would get the same result regardless of $\theta$'s distribution.

I am bit confused and any clarification or assistance would be appreciated.

Thank you.

-
Note that the derivate that you are taking is when $\theta$ is given. Therefore you dont take the derivative of something random. –  Seyhmus Güngören Feb 11 '13 at 17:04
Thanks @SeyhmusGüngören. Does that mean I then cannot compute the expected value above? –  user20721 Feb 11 '13 at 17:16
I think there is a problem here. small $y$ can not be a random variable and at the same time the parameter of a probability as you described. It is like asking what is the probability of a random variable? what is that? –  Seyhmus Güngören Feb 11 '13 at 17:24
@SeyhmusGüngören Typo on my part, thank you –  user20721 Feb 11 '13 at 17:31

I dislike using the same notation for both a random variable and the argument to its density function or cumulative distribution function. Thus $F_X(x)=\Pr(X\le x)$, and the meanings of $X$ and $x$ are different. The lower-case $x$ can be any number, so for example $F_X(3)=\Pr(X\le3)$, etc. I think a good case can be made that certain notational usages ultimately make clear thinking easier. In particular, $f_Y(y)$ is not a random variable, but $f_Y(Y)$ is, and it looks as if that's what we've got here. So let's say $$Y\mid\theta \sim N(\theta,1)\text{ and }\theta\sim N(0,\delta^2).$$

I take it $P(y\mid\theta)$ is supposed to be the conditional density of $Y$ given $\theta$ evaluated at $y$. I'll write $f_{Y\mid\theta}(y)$.

Everything you're asking about is conditional on the value of $\theta$ (or did I miss something?) so the distribution of $\theta$ doesn't matter at all; for the purposes of this problem it's not even a random variable. I don't know what a partial derivative with respect to a random variable would mean (except maybe a Radon--Nikodym derivative, but that has no relevance here).

Now we have $$\mathbb E\left[ -\frac{\partial^2}{\partial\theta^2} \ln f_{Y\mid \theta}(Y) \mid\theta \right].$$ $$= \mathbb E \left[ -\frac{\partial^2}{\partial\theta^2} \ln\left( \frac{1}{\sqrt{2\pi}} \exp\left( \frac{-(Y-\theta)^2}{2} \right) \right) \right] =\mathbb E\left[ -\frac{\partial^2}{\partial\theta^2}\left( \frac{-(Y-\theta)^2}{2} \right) \right].$$ Then it appears you have the expected value of a constant.

-
Thank you very much for your clear explanation. On the topic of notation, are there any good resources you can point me to regarding notation in statistics that perhaps is accessible to someone with only aan intermediate math background? –  user20721 Feb 12 '13 at 5:08