# Partial derivative respect to random variable - How does one compute this?

CLARIFICATION:

If someone could please help me understand the following: When examining the expected value in this specific situation, how is the distribution of $\theta$ relevant? What difference would it make whether $\theta$ is $10^{10}$ or $10^{-10}$, $-3$, or normal or not?

How does one take a partial derivative with respect to a random variable?

For an assignment, I need to find $E_{Y\mid\theta} \left[- \dfrac{\partial^2}{\partial^2\theta} \ln[P(y\mid\theta)] \right]$, where

$Y \sim N(\theta, 1)$ and $\theta\sim N(0, \delta^2)$, $\delta$ is known

And I realized I do not know how to compute $\dfrac{\partial^2}{\partial^2\theta} \ln[P(y\mid\theta)]$

If I treat $\theta$ as a standard variable and take the second partial derivative of $\ln[P(y\mid\theta)]$ I get a value of $-1$. This does not make sense to me as a correct solution as I would get the same result regardless of $\theta$'s distribution.

I am bit confused and any clarification or assistance would be appreciated.

Thank you.

-
Note that the derivate that you are taking is when $\theta$ is given. Therefore you dont take the derivative of something random. – Seyhmus Güngören Feb 11 '13 at 17:04
Thanks @SeyhmusGüngören. Does that mean I then cannot compute the expected value above? – user20721 Feb 11 '13 at 17:16
I think there is a problem here. small $y$ can not be a random variable and at the same time the parameter of a probability as you described. It is like asking what is the probability of a random variable? what is that? – Seyhmus Güngören Feb 11 '13 at 17:24
@SeyhmusGüngören Typo on my part, thank you – user20721 Feb 11 '13 at 17:31

I dislike using the same notation for both a random variable and the argument to its density function or cumulative distribution function. Thus $F_X(x)=\Pr(X\le x)$, and the meanings of $X$ and $x$ are different. The lower-case $x$ can be any number, so for example $F_X(3)=\Pr(X\le3)$, etc. I think a good case can be made that certain notational usages ultimately make clear thinking easier. In particular, $f_Y(y)$ is not a random variable, but $f_Y(Y)$ is, and it looks as if that's what we've got here. So let's say $$Y\mid\theta \sim N(\theta,1)\text{ and }\theta\sim N(0,\delta^2).$$
I take it $P(y\mid\theta)$ is supposed to be the conditional density of $Y$ given $\theta$ evaluated at $y$. I'll write $f_{Y\mid\theta}(y)$.
Everything you're asking about is conditional on the value of $\theta$ (or did I miss something?) so the distribution of $\theta$ doesn't matter at all; for the purposes of this problem it's not even a random variable. I don't know what a partial derivative with respect to a random variable would mean (except maybe a Radon--Nikodym derivative, but that has no relevance here).
Now we have $$\mathbb E\left[ -\frac{\partial^2}{\partial\theta^2} \ln f_{Y\mid \theta}(Y) \mid\theta \right].$$ $$= \mathbb E \left[ -\frac{\partial^2}{\partial\theta^2} \ln\left( \frac{1}{\sqrt{2\pi}} \exp\left( \frac{-(Y-\theta)^2}{2} \right) \right) \right] =\mathbb E\left[ -\frac{\partial^2}{\partial\theta^2}\left( \frac{-(Y-\theta)^2}{2} \right) \right].$$ Then it appears you have the expected value of a constant.