2
$\begingroup$

Let $X_1$ and $X_2$ are independent $N(0, \sigma^2)$ which means (mean = 0, variance = $\sigma^2$) random variables. What is the distribution of $X_1^2 + X_2^2$?

My approach is that $X_1\sim N(0, \sigma^2)$ and $X_2\sim N(0, \sigma^2)$.

Transforming $X_1$ and $X_2$ into standard normal, $X_1/\sigma\sim N(0, 1)$ and $X_2/\sigma\sim N(0, 1)$.

Then $X_1^2/\sigma$ and $X_2^2/\sigma$ have chi-squared distribution with 1 degree of freedom.

Then I found the moment-generating function for $X_1^2$ and $X_2^2$;$$m_{X_1^2} = (1-2t)^{-1/2}$$ and $$m_{X_2^2} = (1-2t)^{-1/2}$$

So the moment generating function for $X_1^2 + X_2^2$ is $$m_{X_1^2}(t) m_{X_2^2}(t) = (1-2t)^{-2/2}$$

So $X_1^2 + X_2^2$ has a chi-squared distribution with 2 degrees of freedom. My question can I treat $X_1^2/\sigma$ + $X_2^2/\sigma$ as $X_1^2$ + $X_2^2$ like I did above?

$\endgroup$

marked as duplicate by Did, heropup, user61527, Claude Leibovici, user91500 May 23 '14 at 5:52

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ You mean $\frac{X_1^2}{\sigma^2}$ and $\frac{X_2^2}{\sigma^2}$ have chi-squared distributions with 1 degree of freedom. $\endgroup$ – afedder May 14 '14 at 21:11
  • $\begingroup$ yes, it is what I mean $\endgroup$ – user111548 May 14 '14 at 21:12
  • 7
    $\begingroup$ This question is a duplicate of this one by a different user or possibly the same user posting under a pseudonym because the work shown is exactly the same. $\endgroup$ – Dilip Sarwate May 14 '14 at 21:13
  • $\begingroup$ So try deriving the moment-generating function with these scaled random variables (by the factor $\frac{1}{\sigma^2}$)...shouldn't be too hard if you already know how to derive the moment-generating function normally. $\endgroup$ – afedder May 14 '14 at 21:14
  • $\begingroup$ @afedder: Do you think it should be $X_1/\sigma$~$N(0, 1)$ and $X_2/\sigma$~$N(0, 1)$ or $X_1/\sigma^2$~$N(0, 1)$ and $X_2/\sigma^2$~$N(0, 1)$.? $\endgroup$ – user111548 May 14 '14 at 21:16
3
$\begingroup$

Hint: Recall that if $X_1,X_2,...,X_n$ are independent and identically distributed as $\chi_{1}^{2}\,$, then $$\sum_{i=1}^{n} X_{i} \sim \chi_{n}^{2}$$ and that if $Z \sim N(0,1)\,$, then $$Z^2 \sim \chi_{1}^{2} \,\,.$$ Additional hint/spoiler: By the above, for independent $X,Y \sim \chi_{1}^{2}$ , it follows that $X+Y \sim \chi_2^2 \,.$ It can be shown that $X+Y \equiv W$ for $W \sim \text{Exp}(\frac{1}{2})$. You should verify this and then you are basically finished. To do this, prove that $X \sim \chi_n^2$ has density given by $$f(x \mid n) = \frac{1}{2^{n/2}\Gamma(n/2)}x^{n/2-1}e^{-x/2} \,\,\,\,\text{for $x>0$}\,.$$ Then, see that the density of $X \sim \chi_2^2$ is $$f(x \mid 2) = \frac{1}{2}e^{-\frac{1}{2}x} \,\,\,\,\text{for $x>0$}\,,$$ and this is the density of a random variable from an exponential distribution with parameter $\frac{1}{2}$. $$$$ Another relevant derivation: Suppose $X$ is a random variable and $Z = aX$ for some $a \in \mathbb{R} \backslash \{0\}$. Then the cumulative distribution function of $Z$ is given by $$F_Z(z) =\mathbb{P}(Z \leq z)=\mathbb{P}(aX \leq z)=\mathbb{P}\left(X\leq \frac{z}{a}\right)=F_X\left(\frac{z}{a}\right)\,\,,$$ where $F_X$ is the cumulative distribution function of $X$. Now, we derive the density function of $Z$, which we will denote $f_Z$, in the case that $a>0$ (this is the only case that applies here since $\sigma^2>0$): $$f_Z(z)=\frac{\text{d}}{\text{d}z}F_Z(z)=\frac{\text{d}}{\text{d}z}F_X\left(\frac{z}{a}\right)=\frac{1}{a}f_X\left(\frac{z}{a}\right) \,,$$ where $f_X$ is the density function of $X$. Final hint: The above hints are in the order of usage. $$$$ The following is a solution based on the above hints for future readers' benefit. Notice that we can standardize $X_1$ and $X_2$, so that $$\frac{X_i - 0}{\sigma} = \frac{X_i}{\sigma} \sim N(0,1) \,\,\,\,\,\text{for} \,\,i=1,2\,\,.$$ It follows that $$\left(\frac{X_i}{\sigma}\right)^2 \sim \chi_1^2 \,\,\,\,\,\text{for}\,\,i=1,2\,\,,$$ so that $$\left(\frac{X_1}{\sigma}\right)^2 + \left(\frac{X_2}{\sigma}\right)^2=\frac{1}{\sigma^2}(X_1^2 + X_2^2) \sim \chi_2^2\,\,.$$ Also, it is not difficult to verify that a $\chi_2^2$ random variable is equivalent in distribution to an $\text{Exp}(\frac{1}{2})$ random variable, so $$\frac{1}{\sigma^2}(X_1^2 + X_2^2) \sim \text{Exp}\left(\frac{1}{2}\right)\,.$$ Now, let $X = \frac{1}{\sigma^2}(X_1^2 + X_2^2)$ and $a=\sigma^2>0$, so that $Z=aX=X_1^2 + X_2^2$, and apply the last hint. We know that the density function of $X$ is given by $$f_X(x)=\frac{1}{2}e^{-\frac{1}{2}x},$$ and it follows that the density function of $Z$ is $$f_Z(z)= \frac{1}{\sigma^2}f_X\left(\frac{z}{\sigma^2}\right)=\frac{1}{2\sigma^2}e^{-\frac{1}{2\sigma^2}z} \,\,.$$ We recognize this as the density for a random variable from an exponential distribution with parameter $\frac{1}{2\sigma^2}$. In other words, $$X_1^2 + X_2^2 \sim \text{Exp}\left(\frac{1}{2\sigma^2}\right)\,\,.$$ @DilipSarwate @FelixMarin @user3001408, you might be interested in this derivation.

$\endgroup$
  • $\begingroup$ @DilipSarwate: That's why I said to the OP to check what will happen to the distribution if you scale by $1/\sigma^{2}$ using the MGF derivation $\endgroup$ – afedder May 14 '14 at 21:44
  • $\begingroup$ From the above, we know that $$\frac{1}{\sigma^{2}}(X_1^2 + X_2^2) \sim \chi_{2}^{2}.$$ This is certainly useful information, although it is not the entire solution. $\endgroup$ – afedder May 14 '14 at 21:51
  • $\begingroup$ So $cU \sim \chi_{2}^{2}$ for a constant $c>0$ and a random variable $U$. Can we figure out the distribution of $U$? The answer is, of course, at least in this case. $\endgroup$ – afedder May 14 '14 at 21:54
  • 1
    $\begingroup$ He pretty much spoon fed you the answer...did the entire calculation, all you have to do is conclude the distribution. He tells you the cumulative distribution for a general exponential random variable with parameter $\lambda$...all you have to do is compare the constants in his answer to $\lambda$ in the general function and you will figure out the parameter of this exponential distribution (he has already confirmed for you that it is exponential, I wouldn't have even done this much after going through the calculation). Do some work on your own homework, man. @user111548 $\endgroup$ – afedder May 17 '14 at 9:05
  • 1
    $\begingroup$ The second equality is using the fact that the random variables are independent and the density function of a normal distribution with mean $0$ and variance $\sigma^{2}$...the third equality is a shift from rectangular to polar coordinates - this type of substitution is learned in a multivariable calculus course (usually required for an intro probability theory course). Recall that $\text{d}x\text{d}y$ is substituted with $r\text{d}r\text{d}\theta$ and that $r^2 = x^2+y^2$. $\endgroup$ – afedder May 17 '14 at 9:25
1
$\begingroup$

You have $$ X_{i}^{2}=\sigma^{2}(Z^{2})=\sigma^{2}\Gamma(\frac{1}{2},2)=\Gamma(\frac{1}{2},2\sigma^{2}) $$

Therefore we have $$ X_{1}^{2}+X_{2}^{2}=\Gamma(1,2\sigma^{2}) $$

where we used property of $\Gamma$-distribution.

$\endgroup$
  • $\begingroup$ Remember, $\Gamma(\alpha,\lambda)$ is special for $\alpha=1$. $\endgroup$ – afedder May 17 '14 at 12:43
  • $\begingroup$ Also, this isn't a formal proof whatsoever. $\endgroup$ – afedder May 17 '14 at 12:46
  • $\begingroup$ No, but I believe it should be readable to OP and he/she should be able to do it himself. I have no intention to fill in detail for his/her homework. $\endgroup$ – Bombyx mori May 17 '14 at 12:59
  • 1
    $\begingroup$ Also, most intro probability classes don't teach the properties of the gamma distribution to the extent that you are using here. - the most important property for this is the constant multiple of a chi-squared random variable of $n$ degrees of freedom (or $\Gamma(\frac{n}{2},\frac{1}{2})$ random variable), which you seem to use as if it is a trivial leap. $\endgroup$ – afedder May 17 '14 at 13:14
  • $\begingroup$ 2nd and 3rd equalities (and even 1st in terms of notation) would be very confusing to me at the level of the OP $\endgroup$ – afedder May 17 '14 at 13:19
0
$\begingroup$

Not really - recall that the mgf is $$ m_X(t) = \mathbb{E}\left[e^{tX}\right] $$ and if you rescale $X$ by a constant $\sigma$, what happens to the result?

$\endgroup$
  • $\begingroup$ Based on what you said, you think $X_1/\sigma$~$N(0, 1)$ and $X_2/\sigma$~$N(0, 1)$ is wrong? $\endgroup$ – user111548 May 14 '14 at 21:17
  • $\begingroup$ @user111548 no. I meant that the mgf of $X$ and mgf of $X/\sigma$ are not the same thing. $\endgroup$ – gt6989b May 14 '14 at 21:18
  • $\begingroup$ ok...Before I got into the mgf. Do you agree that $X_1 /σ$ ~$N(0,1)$ and $X_2 /σ$ ~$N(0,1)$ and $X_1^2/\sigma^2 $ has a chi-squared distribution with 1 degree of freedom? $\endgroup$ – user111548 May 14 '14 at 21:21
  • $\begingroup$ @user111548 Yes. $\endgroup$ – gt6989b May 14 '14 at 21:22
0
$\begingroup$

$\newcommand{\+}{^{\dagger}} \newcommand{\angles}[1]{\left\langle\, #1 \,\right\rangle} \newcommand{\braces}[1]{\left\lbrace\, #1 \,\right\rbrace} \newcommand{\bracks}[1]{\left\lbrack\, #1 \,\right\rbrack} \newcommand{\ceil}[1]{\,\left\lceil\, #1 \,\right\rceil\,} \newcommand{\dd}{{\rm d}} \newcommand{\down}{\downarrow} \newcommand{\ds}[1]{\displaystyle{#1}} \newcommand{\expo}[1]{\,{\rm e}^{#1}\,} \newcommand{\fermi}{\,{\rm f}} \newcommand{\floor}[1]{\,\left\lfloor #1 \right\rfloor\,} \newcommand{\half}{{1 \over 2}} \newcommand{\ic}{{\rm i}} \newcommand{\iff}{\Longleftrightarrow} \newcommand{\imp}{\Longrightarrow} \newcommand{\isdiv}{\,\left.\right\vert\,} \newcommand{\ket}[1]{\left\vert #1\right\rangle} \newcommand{\ol}[1]{\overline{#1}} \newcommand{\pars}[1]{\left(\, #1 \,\right)} \newcommand{\partiald}[3][]{\frac{\partial^{#1} #2}{\partial #3^{#1}}} \newcommand{\pp}{{\cal P}} \newcommand{\root}[2][]{\,\sqrt[#1]{\vphantom{\large A}\,#2\,}\,} \newcommand{\sech}{\,{\rm sech}} \newcommand{\sgn}{\,{\rm sgn}} \newcommand{\totald}[3][]{\frac{{\rm d}^{#1} #2}{{\rm d} #3^{#1}}} \newcommand{\ul}[1]{\underline{#1}} \newcommand{\verts}[1]{\left\vert\, #1 \,\right\vert} \newcommand{\wt}[1]{\widetilde{#1}}$ Let's $\ds{X \equiv X_{1}^{2} + X_{2}^{2}}$:

\begin{align} &\color{#00f}{\large\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} {\expo{-X_{1}^{2}/\pars{2\sigma^{2}}} \over \root{2\pi}\sigma} \,{\expo{-X_{2}^{2}/\pars{2\sigma^{2}}} \over \root{2\pi}\sigma} \delta\pars{X - X_{1}^{2} - X_{2}^{2}}\,\dd X_{1}\,\dd X_{2}} \\[3mm]&={1 \over 2\pi\sigma^{2}}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} \expo{-\pars{X_{1}^{2} + X_{2}^{2}}/\pars{2\sigma^{2}}} \delta\pars{X - X_{1}^{2} - X_{2}^{2}}\,\dd X_{1}\,\dd X_{2} \\[3mm]&=\Theta\pars{X}\,{\expo{-X/\pars{2\sigma^{2}}} \over 2\pi\sigma^{2}}\times \\[3mm]&\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} \!\!\!\!\!\Theta\pars{\root{X} - \verts{X_{1}}}\bracks{% {\delta\pars{X_{2} + \root{X - X_{1}^{2}}} \over 2\verts{X_{2}}} +{\delta\pars{X_{2} - \root{X - X_{1}^{2}}} \over 2\verts{X_{2}}}} \,\dd X_{1}\,\dd X_{2} \\[3mm]&=\Theta\pars{X}\,{\expo{-X/\pars{2\sigma^{2}}} \over 2\pi\sigma^{2}} \int_{-\root{X}}^{\root{X}}{\dd X_{1} \over \root{X - X_{1}^{2}}} =\Theta\pars{X}\, {\expo{-X/\pars{2\sigma^{2}}} \over 2\pi\sigma^{2}}\bracks{2\arcsin\pars{1}} \\[3mm]&=\color{#00f}{\large\Theta\pars{X}\, {\expo{-X/\pars{2\sigma^{2}}} \over 2\sigma^{2}}} \end{align}

$\ds{\Theta\pars{x}}$ is the Heaviside Step Function. $\ds{\delta\pars{x}}$ is the Dirac Delta Function.

$\endgroup$
  • $\begingroup$ What is $\Theta(X)$? $\endgroup$ – Dilip Sarwate May 15 '14 at 3:32
  • $\begingroup$ @DilipSarwate I added two links at the answer end. I guess we are not allowed to put more than one link in a comment. Thanks a lot. $\endgroup$ – Felix Marin May 15 '14 at 3:56
  • $\begingroup$ Thanks for clarifying the notation. You might want to take a look at this answer of mine to a question which is a duplicate of this question. $\endgroup$ – Dilip Sarwate May 15 '14 at 4:14
  • 1
    $\begingroup$ This is like killing a kitten with a bazooka, in my opinion. $\endgroup$ – afedder May 15 '14 at 6:26
  • 1
    $\begingroup$ @afedder It's likely true but it's a nice technique in more complicated situations. It avoids to think about limits since those functions take care of that. For me it's quite useful since I'm pretty bad with drawing. It can save your life when the real messy shows up. I saw other solutions somewhere with inequalities and a derivative to the end. The $\Theta$ can handle the inequalities and the derivative yields the $\delta$ such that we avoid one step and "the drawing". We have a saying in spanish for your idea ( translated ): "To kill a cockroach with a machine gun". Thanks a lot. $\endgroup$ – Felix Marin May 15 '14 at 6:51
0
$\begingroup$

You can approach it like this:

1) Calculate the distribution of $X_1^2$ and $X_2^2$, individually. Call them $f(X_1)$, and $g(X_2)$.

2) Now the second step. Now you calculate the distribution of the sum of $f(X_1)$ and $g(X_2)$ via convolution integral.

This can be one way of calculating what you are asking!

Note:

To calculate the distribution of $X_1^2$ (e.g.), you can use the CDF method. For example $F_{X_1^2}(x)=P(X_1^2 \le x)$. Now you express it in terms of $X_1$, and then differentiate to get the PDF.

$\endgroup$
  • $\begingroup$ What a perfectly dreadful way of finding the answer where no advantage is being taken of the fact that $X_1$ and $X_2$ are zero-mean independent normal random variables with the same variance. This approach would "work" in general for arbitrary independent random variables but why bother trying to touch your ear after wrapping your arm around your head two times? $\endgroup$ – Dilip Sarwate May 14 '14 at 23:04
  • $\begingroup$ I would be interested to learn if you have something better to add. $\endgroup$ – user3001408 May 14 '14 at 23:09
  • $\begingroup$ Sure. Look at this answer to the question which I have already pointed (see my comment on the main question) as a duplicate of this one. $\endgroup$ – Dilip Sarwate May 14 '14 at 23:22
  • $\begingroup$ @dilip Yes, it is good! I agree convolution integral is terrible when you have to add, say 5, or 50 independent random variables. But convolution integral is somewhat intuitive to start with and understand what is going on. In addition, it has a nice application if you can connect with Fourier transform!!! $\endgroup$ – user3001408 May 15 '14 at 16:02

Not the answer you're looking for? Browse other questions tagged or ask your own question.