Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I'm reading about statistical decision theory and on one point in my book the author defines the expected squared prediction error by:

$$EPE = E(Y-g(X))^2 = \int(y -g(x))^2Pr(dx, dy)$$

I like to write this with the density function so that it stays more precise:

$$EPE = \int\int(y-g(x))^2f(x,y)\;dx\;dy$$

Now on the other part the author says that by conditioning on $X$, $EPE$ can be written as:

$$EPE = E_XE_{Y|X}([Y-g(X)]^2\;|\;X)$$

For some reason this notation confuses me...could someone write this conditional notation of $EPE$ more precisely, i.e. so that it would include the joint density function of random variables $X$ and $Y$ etc.?

Just to be sure: $X$ is the variable we use to predict $Y$ and $g(X)$ is the function we are trying to solve, which minimizes $EPE$.

Thank you for any help :)

share|improve this question

1 Answer 1

up vote 4 down vote accepted

$EPE = \int\int {(y-g(x))^2f(x,y)dxdy}$

By Bayes' Theorem $f(x,y)=f(y\,|\,x)\,f(x)$ we have:

$EPE = \int\int {(y-g(x))^2f(y\,|\,x)\,f(x)dxdy}$

Rearranging gives:

$EPE = \int f(x)\;\left(\,\int (y-g(x))^2f(y\,|\,x)dy\,\right)\;dx$

Using definition of $E_x$ we get:

$EPE = E_x(\;\int (y-g(x))^2f(y\,|\,x)dy\;) $

Using definition of $E_{Y\,|\,X}$ we get:

$EPE = E_x(\;\;\;E_{Y\,|\,X}(\,(Y-g(X))^2\,|\,X\,)\;\;\;) $

Or an even shorter notation:

$EPE = E_xE_{Y\,|\,X}(\,[Y-g(X)]^2\,|\,X\,) $

P.S.: I'm reading this book too and posting Python code to reproduce the figures in the book at https://github.com/mattsgithub/stat-learn. It's a work-in-progress.

share|improve this answer
    
+1 Thank you for your answer! Sorry for accepting it so late =) –  jjepsuomi Feb 14 at 8:42

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.