# Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

865 questions
Filter by
Sorted by
Tagged with
6 views

### Conditional Covariance is zero in classical linear regression model

I just read that in the classical linear regression model (Y=Xβ+ε) the Cov(β ̂,ε ̂│X)=0. How can we derive this fact? What is clear is that if X and Y are independent, then Cov(X,Y)=0. Also, for any ...
20 views

### How to write residual in terms of error in simple linear regression

It's given that $y_{i}=\beta_{0}+\beta_{1} x_{i}+\epsilon_{i}, i=1,2, \ldots, n, E\left(\epsilon_{i} \mid X\right)=0, \operatorname{var}\left(\epsilon_{i} \mid X\right)=\sigma^{2},$ for all $i$ Can I ...
17 views

### One sided test of coefficients in Simple linear regression

Let $Y_{i} = \beta_{0} + \beta_{1}X_{i} + \epsilon_{i}$ be a simple linear regression model with independent errors and iid normal distribution. I have done two sided t-test for coefficients by test ...
10 views

### Gauss-Markov Theorem proof

Is it possible to prove this part of the Gauss-Markov Theorem: w'β ̂ is BLUE (best linear unbiased estimator) for w'β, where β ̂ is the OLS estimate of β, and w is a nonzero vector. I know how to ...
25 views

### How to proof a test statistics is pivotal

Consider we have a classic normal linear regression model $$y_t = X_t\beta + u_t, u \sim NID(0,\sigma^2)$$ Where we define n observations, $\beta$ is a k-vector and 1 x k vector of regressors $X_t$ ...
13 views

### Linear regression feature is combination of other features, but has OPPOSITE correlation?

We're examining "salesperson cost per sale," where cost is just what the agent is paid in compensation. In other words, our target variable is the ratio of (salesperson pay)/(count of sales)....
15 views

There is a hypothetical machine which takes an integer $x$ and returns an integer $y$ such that $y=F(x)+\varepsilon$ where $\varepsilon$ is an integer. It is known that the function is of the form F(\... 0answers 8 views ### What is the benefit of lambda over quadratic lambda in linear regression regularization For linear regression, I got this regularized least mean errors \begin{align} \mathcal{L}_{A, \Lambda}(\theta)_1 & = ||y - X\theta||^2_{A} + ||\theta||^2_{\Lambda} \\ \mathcal{L}_{A, \Lambda}(\... 0answers 5 views ### Is it possible for out-of-sample residuals to be much smaller than in-sample ones? When doing linear regression the classic trap is to report residuals on the training data rather than testing data. However I seem to be consistently getting significantly lower errors on my test data ... 0answers 24 views ### Ways to describe matrix whose elements are linear combinations of elements of a vector? The motivation for this question comes from trying to solve the following system: $$B = AX: \left[X\right]_{ij}=\sum_{l=1}^k \alpha_{k}^{(ij)}c_k$$ WhereA,B \in \mathbb{R}^{N\times p}, \hspace{0.1in}...
6 views

Suppose we have paired observations $\left(x_{i}, Y_{i}\right)$ and assume the nonparametric model $$Y_{i}=f\left(x_{i}\right)+\epsilon_{i}, \quad i=1, \ldots, n$$ where $Y_{i}$ 's are responses, ...
6 views

### What is a relationship between standard deviation of a dependent variable and standard deviation of residuals?

In a book, ThinkStats, by Allen Downey, it uses an example of a simple linear regression model to predict babies'' birth weights using mothers' ages. Then, it mentions that standard deviation of the ...
12 views

23 views

### Why does $\frac{\delta}{\delta\beta}y^TX\beta=\frac{\delta}{\delta\beta}B^TX^Ty?$

Why does $\frac{\delta}{\delta\beta}y^TX\beta=\frac{\delta}{\delta\beta}B^TX^Ty?$ In linear regression the parameters to the function $y=X\beta + \epsilon$ can be found by calculating the derivative ...
29 views

21 views

### Can we estimate $y = \beta_1\exp(x) + \beta_2\exp(-x) + \beta_3$ using Linear Regression

Can we estimate the following relationship using linear regression. Here, $\beta_1, \beta_2$ and $\beta_3$ are parameters. $$y = \beta_1\exp(x) + \beta_2\exp(-x) + \beta_3$$
29 views

### Intuition about a matrix multiplication equality

I encountered the following formula while studying the analytical solution of the Linear regression problem in Machine learning context (Optimizing the weight w.r.t squared error) $w^{T}X^{T}y=y^{T}Xw$...
86 views

### Coefficient of Determination and Standard Error of the Model

Background explaining standard concepts and standard terminology used in linear regression and analysis of variance: It will be supposed that one has data points $(X_i,Y_i),\, i = 1,\ldots,n.$ The ...
14 views

### Closed form solution for Restricted Weighted Least Squares

From Greene, we know that the closed-form solution of a restricted least squares is: $\beta_{Constrained} = \beta_{Uncon} - (X'X)^{-1}R'[R(X'X)^{-1}R']^{-1}(R\beta_{Uncon}-r)$. Is there any similar ...
22 views

### Is there an analytical solution to MLE of linear regression with non-normal(exponential) error term?

I know that MLE of normal linear regression $y = k_1x_1+k_2x_2 + \epsilon, \epsilon\sim N(0,1)$ has a nice analytical solution. But what if the error term is exponential distribution? the error term ...
180 views

31 views

### Linear regression with non-fixed regressors and some properties

I was talking to my teacher the other day about the OLS and linear regression model $Y = \beta X + \varepsilon$. If the regressors X are fixed numbers and I don't have the normal condition on the ...
26 views

### Consistency of least squares for linear regression

Consider the generative model for linear regression w.r.t. the true parameter $w^* \in S^{d-1}$ $$y=Xw^*+e$$ with i.i.d. Gaussian error $e \sim N(0, \sigma^2I_n)$. Let $X \in \mathbb{R}^{n\times d}$ ...
20 views

### Coefficient Estimators of $\frac{1}{x^{2}}$ Weighted Least Squares Linear Regression

I have a feeling there should be a mathematical formular for determining the estimators of the coefficients of a $\frac{1}{x^{2}}$ Weighted Linear Regression. I was able to derive the estimators ($a$ ...
20 views

### How can we interpret residual plot in case we have many variables?

In Residual plots, we try to visualize & interpret whether linearity is valid or not in the linear regression model. One way to do this is to plot error term wrt to the independent variable(say x)....