Let $Y_i$ be independent $N(\beta_0 + \beta_1x_i, \sigma^2$ for $1\leq i\leq n$, where $\{x_i\}_i=1^n, \beta_0, \beta_1, \sigma^2 >0$ are constants. Let $\hat{\beta_0}, \hat{\beta_1}$ be the solution to \begin{align} \min_{\beta_0, \beta_1} \sum_{i = 1}^n (Y_i - \beta_0 - \beta_1x_i)^2. \end{align}

Note that $\hat{\beta_0}, \hat{\beta_1}$ are of the form \begin{align} \hat{\beta_0} &= \sum_{i = 1}^n a_i Y_i = \mathbf{a}^T\mathbf{Y} \\ \hat{\beta_1} &= \sum_{i = 1}^n b_i Y_i = \mathbf{b}^T\mathbf{Y} \\ \end{align}

The questions are:

(a) Find $c$ such that $\mathbf{d} = \mathbf{a} + c\mathbf{b}$ satisfies $\mathbf{b}^T\mathbf{d} = 0$, and consider the following orthogonal linear transformation

\begin{align} \left[ \begin{array}{c} Z_1 \\ \vdots \\ Z_n \end{array} \right] = \sigma^{-1} \mathbf{K} \left[ \begin{array}{c} Y_1 - \beta_0 - \beta_1 x_1 \\ \vdots\\ Y_n - \beta_0 - \beta_1 x_n \end{array} \right], \end{align}

where the first and second rows are \begin{align} \mathbf{k}_1 \triangleq \mathbf{d}^T / \lVert \mathbf{d} \rVert_2, \\ \mathbf{k}_2 \triangleq \mathbf{b}^T / \lVert \mathbf{b} \rVert_2, \end{align}

and the rows $\mathbf{k}_3, \cdots, \mathbf{k}_n$ are orthonormal vectors. In other words, $\mathbf{K}^T\mathbf{K} = \mathbf{I}$, i.e., $\mathbf{K}^{-1} = \mathbf{K}^T$, so that $\det(\mathbf{K}) = \pm 1$ and $\sum_{i = 1}^n Z_i^2 = \sum_{i = 1}^n (Y_i - \beta_0 - \beta_1 x_i)^2/\sigma^2$.

(b) Define SSE (sum of squared errors) $\triangleq \sum_{i = 1}^n (Y_i - \hat{\beta_0} - \hat{\beta_1} x_i)^2$. Show that for any orthogonal matrix as in Part (a),

\begin{align} SSE/\sigma^2 = \sum_{i = 1}^n \frac{(Y_i - \beta_0 - \beta_1 x_i)^2}{\sigma^2} - Z_1^2 - Z_2^2 = Z_3^2 + \cdots + Z_n^2. \end{align}

Find the joint distribution of $(\hat{\beta_0}, \hat{\beta_1}, SSE/\sigma^2)$.


1 Answer 1


Part (a) To find $c$. we let \begin{align} &\mathbf{b}^T(\mathbf{a} + c\mathbf{b}) = \mathbf{b}^T\mathbf{a} + c\lVert \mathbf{b} \rVert_2^2 = 0 \nonumber\\ & \Leftrightarrow c = -\frac{\mathbf{b}^T\mathbf{a}}{\lVert \mathbf{b} \rVert_2^2}. \end{align}

Now we need help for Part (b). Do we need to derive the closed form for $\hat{\beta_0}$ and $\hat{\beta_1}$ in order to proceed?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.