# Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

613 questions
37 views

46 views

160 views

### symmetry of Regression line when SDx equals SDy and non intutive results

This question is taken from Freedman In a certain class, midterm scores average out to $60$ with an SD of $15,$ as do scores on the final. The correlation between midterm scores and final ...
34 views

98 views

471 views

### Basic exponential regression

Background: I've been struggling with an exponential regression problem for about 8 months now (on and off): Vertically translated depreciation curve: Update the exponential regression coefficient ...
14 views

31 views

I was reading a particular document on linear regression and I just can't understand how the guy got to the estimation of the beta parameter. His results are as follows: $SQ(\alpha, \beta)=\sum\... 0answers 55 views ### Convert a matrices multiplications to Ax=b I have a training data matrix$S_{\tau \times n}$and actual output$y_{1\times P}$. The weighted parameters for a linear model that maps the input to the output is $$y = \alpha_{1 \times \tau}S_{\... 1answer 62 views ### Regressions in case of non-normality Our variables and residuals are not normally distributed. What we found is that regressions are usually quite robust against violations of normality. But we don't know to which degree, because our ... 1answer 181 views ### Sparse group lasso derivation of soft thresholding operator via subgradient equations In the sparse group lasso paper SGLpaper, they derive a condition for \beta=0 from the equation:$$ \frac{1}n {X^{(k)}}^T(y-\sum_{l=1}^m X^{(l)} \beta^{(l)}) = (1-\alpha)\lambda u+ \alpha \lambda v$$... 1answer 109 views ### how fit a model with data following asymptotic / sigmoid pattern I'm trying to fit data. I assume that the association between dependent and indepdent variable is of the form$$T(y)=aR(x)+b$$I also know that my data are ressemble either an asymptotic function ... 1answer 55 views ### How to minimize a function following an asymptotic pattern? When running simulation of data following a linear function (with noise) with python, I can find that the linear model gives the best fit according to the standard error of the regression: ... 0answers 76 views ### Show that estimator is the best linear estimator with smallest MSE Let's consider we have OLS model Y=X \beta+\epsilon , where E(\epsilon) = 0 and Var(\epsilon)=\sigma^2 I. Here \hat{\beta} is the least squares estimator of the parameter \beta . Let z \... 2answers 55 views ### Learn Noise / Error in Least Squares If We Know Its Form? Let's say I am doing linear regression and I have a data matrix A. And, I know the noise e_i, is zero mean (and perhaps we know the distribution, too):$$y_i = a_i^T x + e_i$$Obviously from ... 0answers 253 views ### Best fit plane using Least Squares Method and mean error I'm currently using Least Squares Method in order to calculate Best Fit Plane for a given 3 column vectors [X, Y, Z]. I've noticed that consistently I get that the average of the residuals of Z ... 1answer 176 views ### Show that \hat{\beta} and \hat{\sigma^2} are unbiased in special case of linear regression model [closed] Let's consider we have OLS model Y= X\beta+ \epsilon, where rows of matrix X are multivariate normal independent vectors with expected value 0 and variance \Sigma. Vector \epsilon is ... 0answers 52 views ### Is there a hat matrix for gradient descent? Assume gradient descent on a linear cost function converges for some sequence of examples, is it possible to define a hat matrix for gradient descent that accounts for the gradient update of these ... 1answer 25 views ### Finding the Variance of a predicted exponentiated linear value Let me give you some background on my issue. Let's say I have a simple one covariate linear regressor as follows:$$\log\alpha =\alpha_0+\alpha_1x_{i1}.$$Obviously, to find \alpha, we use ... 0answers 16 views ### Is it possible to perform the regression algorithm on two dependent variables? I'm trying to change the 3d coordinates of one reference frame 'a' to other reference frame 'b'. For few points in 'a' i'm having corresponding points in 'b'. So based on this, i want to convert all ... 0answers 22 views ### Formula For Centered Coefficient of Determination I want to show that in the inhomogeneous regression model y=\alpha+X\beta+u$$ R_*^2=max_{z\in \text{col}(X)}r_{x,y}^2, $$where r_{x,y}^2:=\frac{S_{xy}^2}{S_xS_y} is the squared empirical ... 1answer 85 views ### Simplify a = \frac{N\sum(xy)-\sum(x)\sum(y)}{N\sum(x^2)-(\sum(x))^2} to yield a = \frac{\bar{xy} - \bar{x}\bar{y}}{\bar{x^2}-\bar{x}^2} I'm working out some multivariable linear regression equations on paper for a class I'm taking, and I'm getting an erroneous factor of N in my solution according to the class. I'm sure it is my error ... 1answer 77 views ### \hat{Y} = X^T\hat{\beta} Matrix Dimension For Linear Regression Coefficients \beta While reading about least squares implementation for machine learning I came across this passage in the following two photos: Perhaps I’m misinterpreting the meaning of \beta but if X^T has ... 0answers 51 views ### formulation of least squares problems In general, to use the method of least squares, a linear stochastic system is modeled as: $$y = ax + \eta$$ where, y, is an observed variable, x is an input while \... 0answers 31 views ### Error of the intersection of two linear functions I have the following linear fits of two data sets L_1: y=(a_{1}\pm e_{1})x + (b_{1} \pm f_{1}) L_2: y=(a_{2}\pm e_{2})x + (b_{2} \pm f_{2}) How do I calculate the intersection of L_1 and L_2... 3answers 84 views ### Why isn't the linear regression coefficient not just the average vector to data points? I am having trouble intuitively understanding the correctness of the formula to compute the coefficient for the regression line in a linear regression. I know the formula is$$\frac{\sum_{i=1}^N (... 4answers 298 views ### Does least squares (approximate solution) minimize the orthogonal distance of$b$to$Ax$, or does it minimize the error projected along the$b$axis? I have always been confused about whether the approximate solution to$Ax=b$is equivalent to minimizing the average distance of all of the$b$vectors to$Ax$, or whether it is minimizing the ... 1answer 35 views ### When does the orthogonal fitted value become itself? Let$m \times a$full-column matrix$B$, and define the projection matrix onto its column space as$P(B)=B(B'B)^{-1}B'$which is$m\times m$matrix. Consider a rectangular matrix$A$, where$A$is$m ...
Suppose that I have the following model: $y_i = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + u_i$ where $\hat{\beta_k}$, k=0,1,2 , are estimated by the method of least squares, using a sample of ...
### Simplifying $\beta_1$ estimate for a simple linear regression model
For a simple linear regression model I am able to derive the normal equations and solve these to to obtain the following- $$\beta_0=\bar{Y}-\beta_1\bar{X}$$ \beta_1=\frac{\sum(X_i-\bar{X})(Y_i-\bar{...