1
$\begingroup$

My regression exam is a month away and i am trying to learn Matrix regression however and struggling with the questions as a whole they are:

  1. (a) Consider two independent random variables ξ1 and ξ2, such that ξ1 ∼ N(0,1) and ξ2 ∼ N(0,2). Let $η1 =(ξ1+ξ2, ξ2)^{T}$ , $η2 =(ξ1, ξ1−ξ2)^{T}$. Find the covariance matrix between η1 and η2. [ 5 marks ]

(b) Consider the matrix representation of the p parameter linear regression model $$(p > 0) Y = Xθ + e,$$ where Y is the response vector for the n observations, X is the n × p matrix of explanatory variables (of full column rank), θ is the parameter vector and e is a vector of independent error terms, each with the $N~(0,σ^{2})$ distribution.

Find the least squares estimator $\hat{\theta}$ of θ, prove that $\hat{\theta}$ is unbiased, and derive the covariance matrix of $\hat{\theta}$. [ 15 marks ]

(c) Following part (b), let $\hat{Y} = X\hat{\theta}$ and $r = Y − \hat{Y}$. Find the expectation of $r^{T}r$. Is r correlated with $\hat{Y}$ ? Why? [ 10 marks ]

The only part I've managed is part of b) $S=(y-X \theta )^{T}(y-X \theta )$ then; $$\frac{\delta S}{\delta \theta_{j}} = \frac{-\delta X \theta^{T}}{\delta \theta_{j}}(y-X \theta )-(y-X \theta )^{T}\frac{\delta X \theta^{T}}{\delta \theta_{j}}=-2\frac{\delta X \theta^{T}}{\delta \theta_{j}}(y-X \theta )$$

Minimum is given by $$ X^{T}X \theta = X^{T}y $$

Premultiplying by $$(X^{T}X)^{-1}X^{T}X \theta = (X^{T}X)^{-1}X^{T}y$$

Simplifying to: $$\hat{\theta} = (X^{T}X)^{-1}X^{T}y $$

This is unbiased as: $$E(\hat{\theta})=E((X^{T}X)^{-1}X^{T}y)$$ $$=(X^{T}X)^{-1}X^{T}E(y)$$ $$=(X^{T}X)^{-1}X^{T}X \theta$$ $$= \theta $$

Therefore is unbiased. So i am unable to do a, c and the part deriving the covariance matrix.

Any help would be greatly appreciated thank you

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.