# Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

1,136 questions
Filter by
Sorted by
Tagged with
1 vote
17 views

### Why does the Lasso only select only $n$ variables if $p \gg n$

$L_1$ or lasso regularization in regression problems is defined as $$\min||X\beta - y||_2^2 + \lambda ||\beta||_1$$ Multiple resources point out that for $X\in \mathbb{C}^{n\times p}$ if $p \gg n$ ...
8 views

### Which $n$ values does lasso pick if $p \gg n$

It's known that lasso regression picks only $n$ variables in a matrix $X^{n\times p}$ matrix where $p \gg n$ but which variables exactly does lasso pick? Intuitively lasso should always pick the ...
1 vote
48 views

### efficiency of estimator, $\overline{\frac{1}{X^2}}$ vs $\frac{1}{\overline{X}^2}$ vs $\frac{1}{\overline{X^2}}$

I was studying point estimator, and I tried to compare the variances of the estimators to find out which one is more efficient. (Hogg, Tanis "Probability and Statistical Inference" Ch.6) It ...
1 vote
51 views

### Why does least-squares need regularization?

If I understand regularization correctly, it helps if a least-squares problem is not well-posed thus... the problem has no solution the problem has multiple solutions a small change in the input ...
1 vote
29 views

### preventing rare extreme values in linear regression prediction

I am trying to train a model with a lot of input variables using linear regression. For technical reasons, my training data is obtained from a simulation that closely but not perfectly mirrors the ...
18 views

### Puzzled in Parametric Linear Regression Matrix Solution [closed]

If Ynx1 = Xnxpθpx1, then θ = (XTX)-1XTY. But this doesn't work IRL (e.g. X = [1,2,3]T, Y = [2,4,5]T). I cannot think of why (it should be obvious though). Any help on this part? Thank you! Edit: Also, ...
17 views

### If there is a high correlation between two variables, should we expect the high linear regression coefficient? [closed]

I have a data set with multiple features, let suppose $x_1,x_2,x_3,x_4$ and my dependent variable is $y$, when I compute the correlation matrix for $y,x_1,x_2,x_3,x_4$ then imagine the correlation ...
30 views

1 vote
12 views

### Interpreting model accuracy as proof of real association in Machine Learning

Let $\mathbb{X}, \mathbb{Y}$ be training and test sets respectively for some data we assume comes from a function$f$. Let $\hat{f}(\theta)$ be a model of $f$ with a parameter vector $\theta$. Assume ...
16 views

### How to add two points to a set of points such that the sign of slope of the linear regression fit doesn't change?

I have a set of points $\{(i, y_i)\}_{i=1}^k$, and fit a linear regression line on it, yielding $y_i = p_0 i + p_1$. Now I add two more points $(k+1, y_{k+1}), (k+2, y_{k+2})$ to the set of points, ...
8 views

### OLS Beta estimate with indicator function

I have $Y=\alpha + \beta *1_{A}+\epsilon$, then can I just as normally compute $\hat{\beta}=\frac{Cov(Y,1_{A})}{Var(1_{A})}$? Where the event $A$ and $\epsilon$ are correlated. Thanks!
9 views

### MSE of WLS estimator with biased measurements

(also posted on CV, but I will try here too) I am trying to find out if what I am looking at is a known problem. I am considering the case of weighted least squares, and I am trying to find the ...
9 views

### Linearregression of two dataframes

I have two dataframes: ...
1 vote
22 views

### Bagging Linear Model?

I have a question regarding bagging linear models. Suppose you wanna do linear regression on data X and y. Alice directly implements (OLS) Linear Regression on it. The model is A1. Bob applies bagging....
20 views

### What do you call the value you get from a regression equation?

Say I have 2 sets of data which I perform a linear regression on, now I have a regression equation in the form y=mx+b (for example). Now lets say when x = 1, the observed/real value of y = 1. So now ...
11 views

### Ordinary least squares and Gauss-Markov theorem

Let $$Y=X\beta+\varepsilon,\quad Y,\varepsilon\in\mathbb{R}^n,\quad X\in\mathbb{R}^{n\times p},\quad \beta\in\mathbb{R}^p$$ be the standard linear regression model and $\widehat{\beta}$ the OLS ...
20 views

### Minimizing the Objective Function for Derivation of Least Squares Estimator for Multiple Linear Regression

I have a Multiple Linear Regression model where the error vector $\underline{e} \sim N(\underline{0},\underline{R})$ $$\underline{z} = \underline{H}\,\underline{x}_{true} + \underline{e}$$ When I ...
11 views

### Variance of predicted value in linear regression model

I am working on a question whereby I have calculated the linear regression model $$Y_i = -20.57 + 1.7446x_i + \epsilon_i ,$$ and have been given the data that $x_i$ = 85. From this, I have calculated ...
13 views

### Computing 95% Confidence interval for the slope of a simple linear regression out of Minitab output

I created a linear regression model and I have the following Minitab output. I am trying to compute the 95% confidence interval for the slope of the true regression line. I know that the formula for ...
17 views

### Does Multiple Linear Regression work with stocks? If so, does this make sense?

I have a dataset that contains information on a stock. In a csv file, prices are organized like this I am able to get the equation for the line, where y = adjusted close prediction, x = index ...
36 views

### OLS coefficient when errors are not normally distributed

Assume that in this regression Y=β0+β1x+ϵ, where ϵ follows a Poisson distribution. Using OLS, estimate β0,β1 and cov(β1,β0). I am wondering does the distribution of errors changes the β0 and β1 OLS ...
17 views

### Bias in Reverse Regression with Measurement Error?

Lets say I want to estimate the following population regression: y = xb + v (v is the error term) and x is measured with error so that x = x + u. We know that my estimate of x is know biased by ...
10 views

### Dependent errors leading to artificially small p values?

Why do dependent regression errors in a Multiple Linear Regression model (a violation of the assumptions of the MLR model) lead to underestimated standard errors and artificially small p-values? What ...
20 views

### Interpreting a linear regression model

The relationship between annual average temperature over 10 years in various towns and the area of the UK in which the town is located was investigated. Area is described as being one of five ordered ...
1 vote
20 views

1 vote
14 views

### How is ANOVA different from a linear model?

This may be a naive question, but I realize all statistical tests of significance are inherently related to linear models and for anova in particular: I don't exactly understand the difference between ...
9 views

### Is it possible that the correlation between $\hat{b}$ and $\hat{c}$ can be negative multiple linear regression?

Given the following linear regression model as following, with two explanatory variables $x_1$ and $x_2$ and response $y$ $$y_i=a+bx_{i1}+cx_{i2}+\epsilon_{i}$$ We say that $\hat{a}, \hat{b}, \hat{c}$ ...
38 views

### Least squares estimation for linear regression model with random design

Suppose we have a model $$y_i = g(x_i) + \epsilon_i, \quad i = 1, \ldots, n$$ where $y_i \in \mathbb{R}$ is a response variable, $x_i \in \mathcal{X} \subseteq \mathbb{R}^d$, and $\epsilon_i$ is the ...
19 views

### Design matrix of a model

Suppose we have a model $y_{i} = \beta_{0} + \beta_{1} x_{i1} + \beta_{2}x_{i2} + \epsilon_{i}$. Then I need to find a general expression for $\hat{\sigma}²(X^TX)^{-1}$. My first thought was that we ...
1 vote
32 views

20 views

### Linear regression with random coefficients

I am facing a problem defined by the following equation $$\overrightarrow{x}+\overrightarrow{\epsilon}=\overrightarrow{c}$$ Where: All the vectors contain $n$ entries, $\overrightarrow{c}$ is a ...
28 views

### A question about Mallows Cp Statistics

In the book "Introduction to Multiple Linear Regression" By Peck at.el., Mallows $C_p$ value is discussed in the Chapter 10. $C_p$ is defined as C_p = \frac{SS_{\rm Res}(p)}{\widehat{\...