# Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

620 questions
Filter by
Sorted by
Tagged with
194 views

### Why the identity $P_X=P_XZ(Z'P_XZ)^{-1}Z'P_X$ with $P_X=X(X'X)^{-1}X'$?

Suppose $X$ and $Z$ are matrices such that $(X,Z)$ and $P_XZ$ both have full column ranks. Here, $P_X=X(X'X)^{-1}X'$. Consider a regression model $$P_Xy=P_XZ\zeta+v\tag{A}$$ where OLS is used to ...
207 views

### How/why does matrix multiplication work to do a linear fit? [closed]

Some background: I have a B.S. in physics. I have taken linear algebra. I do work that involves doing image analysis in IDL. One thing I have to do a lot is fit a linear equation $(y=mx+b)$ to the ...
618 views

### What are the limitations of linear regression + feature / label transformation?

Regression Suppose I have data points in a matrix $X \in \mathbb{R}^{n \times m}$ as well as labels $\mathbb{R}^n$, where $n$ is the number of my data points and $m$ is the number of features per ...
97 views

68 views

### Understanding notation when finding the estimates in a linear regression model

I have been taught to calculate the estimates $\beta_0$ and $\beta_1$ (or $\vec{\beta}$) by using the following formula $$\sum_{i=1}^{n} y_i \vec{X_i}^T = \vec{\beta}^T\sum_{i=1}^{n} X_i \vec{X_i}^T$$...
139 views

### For a linear regression of $\{(i,y_i)\}_{i=0}^{n-1}$, where $(y_i)$ is increasing and non-negative, is the $y$-intercept at least $-y_{n-1}$?

Suppose we have a set of data points $\{(i,y_i)\}_{i=0}^{n-1}$, where $y_i$ are non-negative integers and where $(y_i)_{i=0}^{n-1}$ is an increasing sequence. Question: In a simple linear regression ...
6k views

### Best Fit Line with 3d Points

Okay, I need to develop an alorithm to take a collection of 3d points with x,y,and z components and find a line of best fit. I found a commonly referenced item from Geometric Tools but there doesn't ...
132 views

### Estimated simple linear regression model

Consider the simple linear regression model $y=50 + 10x + \varepsilon$ where $\varepsilon$ is $NID (0,16)$. Suppose that $n=20$ pairs of observations are used to fit this model. Generate $500$ samples ...
96 views

12k views

### Proof that trace of 'hat' matrix in linear regression is rank of X

I understand that the trace of the projection matrix (also known as the "hat" matrix) X*Inv(X'X)*X' in linear regression is equal to the rank of X. How can we prove that from first principles, i.e. ...
81 views

### Calculate the variance and expectection of $\hat{y}$ in a linear regression model

I have the following linear regression model $$y = \beta_0 + \beta_1 \cdot 40$$ where $\beta_0 = 11.1317$, $\beta_1 = 1.01$, and $40$ is simply the value of the predictor variable (I guess). As you ...
190 views

### Linear Regression Questions - Suspected Typo?

My sister just submitted an assignment and got a few questions marked incorrect (electronically) but I've just checked over them and don't believe this to be the case. Can someone either point out ...
524 views

### Understanding better linear regression

I have been trying to understand linear regression as much as possible, so I am asking you this question in order to keep doing it. There's simple linear regression, where we have just one ...
167 views

### Find least squares regression line

I have a problem where I need to find the least squares regression line. I have found $\beta_0$ and $\beta_1$ in the following equation $$y = \beta_0 + \beta_1 \cdot x + \epsilon$$ So I have both ...
Theorem: Let $Y=X\beta+\varepsilon$ where $$Y\in\mathcal M_{n\times 1}(\mathbb R),$$ $$X\in \mathcal M_{n\times p}(\mathbb R),$$ $$\beta\in\mathcal M_{n\times 1}(\mathbb R ),$$ and \varepsilon\in\...
### If $Y=X\beta+\epsilon$, prove that the least square estimator $\hat\beta$ is independent of $Y-X\hat{\beta}$
Let $Y=X\beta+\epsilon$, where $Y$ is an $n$ by $1$ vector, $X$ is an $n$ by $p$ matrix with full rank and $\epsilon$ is an $n$ by 1 vector of random errors independently and normally distribution ...