Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

Filter by
Sorted by
Tagged with
1
vote
1answer
194 views

Why the identity $P_X=P_XZ(Z'P_XZ)^{-1}Z'P_X$ with $P_X=X(X'X)^{-1}X'$?

Suppose $X$ and $Z$ are matrices such that $(X,Z)$ and $P_XZ$ both have full column ranks. Here, $P_X=X(X'X)^{-1}X'$. Consider a regression model $$ P_Xy=P_XZ\zeta+v\tag{A} $$ where OLS is used to ...
1
vote
2answers
207 views

How/why does matrix multiplication work to do a linear fit? [closed]

Some background: I have a B.S. in physics. I have taken linear algebra. I do work that involves doing image analysis in IDL. One thing I have to do a lot is fit a linear equation $(y=mx+b)$ to the ...
1
vote
1answer
618 views

What are the limitations of linear regression + feature / label transformation?

Regression Suppose I have data points in a matrix $X \in \mathbb{R}^{n \times m}$ as well as labels $\mathbb{R}^n$, where $n$ is the number of my data points and $m$ is the number of features per ...
0
votes
0answers
97 views

Why if $e$ is independent to $\hat y$, then $\hat\beta $ will be independent of $S^2$.

$\bullet$ $X$ is a matrix $n\times p$, $\bullet$ $y=X\beta +\varepsilon$ where $\varepsilon\sim\mathcal N(0,\sigma ^2I_{n\times n})$ $\bullet$ $\hat \beta=(X^TX)^{-1}X^Ty$ $\bullet$ $\hat y=X\hat\...
-1
votes
1answer
87 views

How to approximate linear relationship between two timeseries? [closed]

I have two time series A and B I would like to solve for the equation in the form $$y_t = m x_t + b$$ that transforms a point $y$ at time $t$ from series $B$ to the corresponding point $x$ at time $...
1
vote
1answer
68 views

Understanding notation when finding the estimates in a linear regression model

I have been taught to calculate the estimates $\beta_0$ and $\beta_1$ (or $\vec{\beta}$) by using the following formula $$\sum_{i=1}^{n} y_i \vec{X_i}^T = \vec{\beta}^T\sum_{i=1}^{n} X_i \vec{X_i}^T$$...
2
votes
1answer
139 views

For a linear regression of $\{(i,y_i)\}_{i=0}^{n-1}$, where $(y_i)$ is increasing and non-negative, is the $y$-intercept at least $-y_{n-1}$?

Suppose we have a set of data points $\{(i,y_i)\}_{i=0}^{n-1}$, where $y_i$ are non-negative integers and where $(y_i)_{i=0}^{n-1}$ is an increasing sequence. Question: In a simple linear regression ...
5
votes
2answers
6k views

Best Fit Line with 3d Points

Okay, I need to develop an alorithm to take a collection of 3d points with x,y,and z components and find a line of best fit. I found a commonly referenced item from Geometric Tools but there doesn't ...
0
votes
0answers
132 views

Estimated simple linear regression model

Consider the simple linear regression model $y=50 + 10x + \varepsilon$ where $\varepsilon$ is $NID (0,16)$. Suppose that $n=20$ pairs of observations are used to fit this model. Generate $500$ samples ...
0
votes
1answer
96 views

Computing vector linear regression

In eye tracking we have to compute the linear regression for pupil and gaze. The formula is: $$\begin{bmatrix} gaze_x \\ gaze_y \end{bmatrix} = \begin{bmatrix} \theta_1 \\ \theta_2 \end{bmatrix} + \...
0
votes
2answers
26 views

Linear regression: zero indexed?

I have a set of data corresponding to profit per month: Dec -> 1726 Jan -> 1252 Feb -> 1472 Mar -> 1165 ... And a linear regression algorithm that ...
1
vote
3answers
2k views

linear regression for dummies

I am trying to understand linear regression. I have a limited knowledge in math (Algebra I) but I still want to be able to learn and understand what this is. I don't need to know all the math ...
2
votes
0answers
111 views

Why divide regularization factor by size of dataset?

Suppose I'm trying to minimize a cost function: $$ J(\theta) = \frac {1} {2m} \sum _{i = 1}^ m (h_\theta (x^{(i)}) - y^{(i)})^2 $$ Adding regularization, as seen here, we get: $$ J(\theta) = \frac {...
9
votes
2answers
12k views

Proof that trace of 'hat' matrix in linear regression is rank of X

I understand that the trace of the projection matrix (also known as the "hat" matrix) X*Inv(X'X)*X' in linear regression is equal to the rank of X. How can we prove that from first principles, i.e. ...
0
votes
1answer
81 views

Calculate the variance and expectection of $\hat{y}$ in a linear regression model

I have the following linear regression model $$y = \beta_0 + \beta_1 \cdot 40$$ where $\beta_0 = 11.1317$, $\beta_1 = 1.01$, and $40$ is simply the value of the predictor variable (I guess). As you ...
0
votes
1answer
190 views

Linear Regression Questions - Suspected Typo?

My sister just submitted an assignment and got a few questions marked incorrect (electronically) but I've just checked over them and don't believe this to be the case. Can someone either point out ...
1
vote
2answers
524 views

Understanding better linear regression

I have been trying to understand linear regression as much as possible, so I am asking you this question in order to keep doing it. There's simple linear regression, where we have just one ...
1
vote
2answers
167 views

Find least squares regression line

I have a problem where I need to find the least squares regression line. I have found $\beta_0$ and $\beta_1$ in the following equation $$y = \beta_0 + \beta_1 \cdot x + \epsilon$$ So I have both ...
4
votes
3answers
5k views

Proof of Gauss-Markov theorem

Theorem: Let $Y=X\beta+\varepsilon$ where $$Y\in\mathcal M_{n\times 1}(\mathbb R),$$ $$X\in \mathcal M_{n\times p}(\mathbb R),$$ $$\beta\in\mathcal M_{n\times 1}(\mathbb R ),$$ and $$\varepsilon\in\...
3
votes
1answer
948 views

If $Y=X\beta+\epsilon$, prove that the least square estimator $\hat\beta$ is independent of $Y-X\hat{\beta}$

Let $Y=X\beta+\epsilon$, where $Y$ is an $n$ by $1$ vector, $X$ is an $n$ by $p$ matrix with full rank and $\epsilon$ is an $n$ by 1 vector of random errors independently and normally distribution ...