# Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

619 questions
Filter by
Sorted by
Tagged with
12k views

### Proof that trace of 'hat' matrix in linear regression is rank of X

I understand that the trace of the projection matrix (also known as the "hat" matrix) X*Inv(X'X)*X' in linear regression is equal to the rank of X. How can we prove that from first principles, i.e. ...
779 views

### Why does $A^TAx = A^Tb$ have infinitely many solution algebraically when $A$ has dependent columns?

This is a problem from least square approximation, where we solve the equation $A^TAx = A^Tb$ when $Ax = b$ is unsolvable. The case I am dealing with is when A has dependent columns, i.e. A is an m by ...
75 views

### Prime number intercept

Suppose I arrange my (infinite) list of prime numbers in the following way: \begin{array}{c|c}x_i&2&5&11&17&23&31&\cdots\\\hline y_i&3&7&13&19&29&37&...
7k views

### what is the variance of a constant matrix times a random vector?

$\newcommand{\Var}{\operatorname{Var}}$In this video is claimed that if the equation of errors in OLS is given by: $$u=y - X\beta$$ Then in the presence of heteroscedasticity the variance of $u$, will ...
6k views

### Best Fit Line with 3d Points

Okay, I need to develop an alorithm to take a collection of 3d points with x,y,and z components and find a line of best fit. I found a commonly referenced item from Geometric Tools but there doesn't ...
563 views

### Using linear algebra to study number theory?

I've posted a paper on arXiv that outlines a linear algebra approach to number theory. Specifically, I have the following questions: Is it possible to draw connections between the factorization ...
51 views

### Can there be a trigonometric function reaching any finite number of points in $\mathbb{R} ^2$

Today in math class we hade a discussion about linear regression, which is all about finding the best (though not perfect) linear equation that passes through a countable set of points, and people ...
279 views

82 views

### $\hat{Y} = X^T\hat{\beta}$ Matrix Dimension For Linear Regression Coefficients $\beta$

While reading about least squares implementation for machine learning I came across this passage in the following two photos: Perhaps I’m misinterpreting the meaning of $\beta$ but if $X^T$ has ...
53 views

548 views

The problem is given below: Simultaneous values of time $t$ and output $y$ from a specific sensor has been measured and is tabulated below $$\begin{array}{cc} t & y \\ \hline 1 & 17 \\ 2 ... 1answer 121 views ### Basic application of category theory to data science Linear regression is the algorithm that, given a set of vectors {\bf x}_i \in \mathbb{R}^p, and a set of targets y_i \in \mathbb{R}, returns a vector {\bf w} \in \mathbb{R}^p minimizing the ... 2answers 55 views ### Learn Noise / Error in Least Squares If We Know Its Form? Let's say I am doing linear regression and I have a data matrix A. And, I know the noise e_i, is zero mean (and perhaps we know the distribution, too):$$y_i = a_i^T x + e_i$$Obviously from ... 1answer 137 views ### Variance Estimate in linear regression In a linear regression, y=X\beta+\epsilon, where \epsilon\sim N(0, \sigma^2), X\sim R^{N \times (p+1)}. Assume the observations y_i are uncorrelated and have constant variance \sigma^2, and ... 1answer 2k views ### derivative transpose I'm reading the book "The Elements of Statistical Learning - Data Mining, Inference, and Prediction" chapter 3 and there comes a simple derivation that I don't understand: We have: ... 2answers 33 views ### Relationship between OLS estimates of slope coefficients of simple linear regression Y on X and X on Y Assume a model y = \beta_0 + \beta_1x + u. Given a sample (x_i, y_i)_{i=1}^n, we can find the OLS estimates of \beta_1, \hat{\beta_1}. Then suppose that we assume another model x = \gamma_0 + ... 1answer 134 views ### how fit a model with data following asymptotic / sigmoid pattern I'm trying to fit data. I assume that the association between dependent and indepdent variable is of the form$$T(y)=aR(x)+b$$I also know that my data are ressemble either an asymptotic function ... 2answers 41 views ### Why is a term that comes out of a variance bracket is squared? I am in a course on data analysis. The following statement is made in the notes made available to us by our professor:$$ \text{Var}[a] = \text{Var}[\bar{y} -b\bar{x}] = \text{Var}[\bar{y}] + \text{...
Can you please give me how did Andrew NG, came up with the formula for cost function $$J(\theta_0,\theta_1) = \frac{1}{2m}\sum_{i=0}^m{(H_\theta(x^i)-y^i))^2}$$ AFAIK the square is being taken to ...
We have the following: The design matrix $X \in R^{n \times d}$ The output vector $y \in R^n$ The weight vector $w \in R^d$ Let $T = \tau I_d$, where $I_d$ is the $d \times d$ identity matrix and \$\...