My regression exam is a month away and i am trying to learn Matrix regression however and struggling with the questions as a whole they are:

  1. (a) Consider two independent random variables ξ1 and ξ2, such that ξ1 ∼ N(0,1) and ξ2 ∼ N(0,2). Let $η1 =(ξ1+ξ2, ξ2)^{T}$ , $η2 =(ξ1, ξ1−ξ2)^{T}$. Find the covariance matrix between η1 and η2. [ 5 marks ]

(b) Consider the matrix representation of the p parameter linear regression model $$(p > 0) Y = Xθ + e,$$ where Y is the response vector for the n observations, X is the n × p matrix of explanatory variables (of full column rank), θ is the parameter vector and e is a vector of independent error terms, each with the $N~(0,σ^{2})$ distribution.

Find the least squares estimator $\hat{\theta}$ of θ, prove that $\hat{\theta}$ is unbiased, and derive the covariance matrix of $\hat{\theta}$. [ 15 marks ]

(c) Following part (b), let $\hat{Y} = X\hat{\theta}$ and $r = Y − \hat{Y}$. Find the expectation of $r^{T}r$. Is r correlated with $\hat{Y}$ ? Why? [ 10 marks ]

The only part I've managed is part of b) $S=(y-X \theta )^{T}(y-X \theta )$ then; $$\frac{\delta S}{\delta \theta_{j}} = \frac{-\delta X \theta^{T}}{\delta \theta_{j}}(y-X \theta )-(y-X \theta )^{T}\frac{\delta X \theta^{T}}{\delta \theta_{j}}=-2\frac{\delta X \theta^{T}}{\delta \theta_{j}}(y-X \theta )$$

Minimum is given by $$ X^{T}X \theta = X^{T}y $$

Premultiplying by $$(X^{T}X)^{-1}X^{T}X \theta = (X^{T}X)^{-1}X^{T}y$$

Simplifying to: $$\hat{\theta} = (X^{T}X)^{-1}X^{T}y $$

This is unbiased as: $$E(\hat{\theta})=E((X^{T}X)^{-1}X^{T}y)$$ $$=(X^{T}X)^{-1}X^{T}E(y)$$ $$=(X^{T}X)^{-1}X^{T}X \theta$$ $$= \theta $$

Therefore is unbiased. So i am unable to do a, c and the part deriving the covariance matrix.

Any help would be greatly appreciated thank you


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.