# Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

1,320 questions
Filter by
Sorted by
Tagged with
In section 3.1.1 of Pattern Recognition and Machine Learning by Christopher Bishop, it is written that $$\ln p(\mathbf{t} | \mathbf{w}, \beta) = \frac{N}{2} \ln \beta - \frac{N}{2} \ln (2 \pi) - \beta ... -1 votes 0 answers 12 views ### Passing a linear regression equation around 5 data points, making sure the regression equation passes through the fifth point. [closed] I have a five point data set which was created by taking the natural log of a data set where x = ln(a) and y = ln(b). The transformed data set is x = {1.61, 2.30, 2.71, 3.40, 4.09} and y = {2.05, 1.43,... 2 votes 1 answer 1k views ### Linear regression with integer function I have a dataset, that only takes integer values (x and y coordiantes). E.g. my data is the following: x = (1,2,2,3,3), y = (1,2,3,3,4). I want to make a linear regression through the data, i.e.... 1 vote 1 answer 2k views ### What is the efficiency of OLS estimators? In page 3, of Asymptotic Theory for Econometricians, the following assumptions of OLS are defined: OLS model: Y= X\beta+\epsilon X is a nonstochastic and finite n x k matrix, n > k. X'X is ... 0 votes 0 answers 25 views ### Regression coefficients without matrix inversion We can find coefficients β in multiple linear regression: β = (XTX)-1XTy I see a lot of related questions on this forum asking "How do I invert this matrix?". This is followed by an answer ... 1 vote 0 answers 38 views ### Why can Total Least Squares Regression not have a solution? I was looking at this reference on TLS regression. I understand the math which says that there can be problems where no solution exists. Based on my understanding, the minimum perturbation to the C ... 12 votes 1 answer 22k views ### Derivative of Mean Squared Error I'm studying with a book and I'm at the Linear Regression part. The author is showing that we have to calculate the derivative of each part of the equation that leads to the loss. But he's using the ... 1 vote 1 answer 1k views ### How to combine several linear regression function into one I have the relationship between f and each of other five variables, the relationships are acquired by linear regression and are as follow f = a{\tiny1}x{\tiny1} + b{\tiny1} f = a{\tiny2}x{\tiny2} ... 0 votes 0 answers 82 views ### How to calculate the trace of this matrix? How do we calculate the trace of (\mathbb{I}-X(X^\prime X)^{-1}X^\prime)\mathbb{J}_n? This question stemmed from the below problem I came across: Suppose we have the linear regression model: y_i=... 1 vote 2 answers 31 views ### Why does Least Absolute Deviation regression exactly fit n measureemnts for a linear system with n independent variables? I am applying LAD regression to conduct some research. I have the following questions regarding LAD: I know LAD exactly fits n measurements for a linear system with n variables. But I cannot easily ... 0 votes 1 answer 19 views ### Help in understanding Bayesian linear regression. I am unable to use Bayesian Regression in the following question. So far I have performed the following calculations: We have our data points Y, X that are ... 0 votes 0 answers 15 views ### Prediction intervals for simple linear regression I want to understand how to construct the test statistic for the case of predictive inference for a simple linear regression model and would be grateful if someone might confirm if my derivation is ... 1 vote 1 answer 34 views ### R^2 of regression of residuals when adding an uncorrelated regressor Suppose we linearly regress Y onto X_1, obtaining residuals \epsilon_1,\ldots,\epsilon_n. Suppose further that X_2 is uncorrelated with X_1, and we linearly regress X_2 onto X_1 (with ... 1 vote 1 answer 32 views ### Local Linear Fit What do you call it when you have a set of known (x,y) data points and you estimate a y-value for a given x-value by performing a linear fit between its two known neighbor (x,y) points? As an ... 4 votes 1 answer 130 views ### Sqrt LASSO vs LASSO In the paper Square Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming they talk about Sqrt-LASSO which is simply just trying to minimize \|Ax-b\|_2 + \lambda\|x\|_1 rather than ... 0 votes 0 answers 84 views ### Continuous piecewise linear (CPWL) function fitting a dataset for which each linear piece contains at least 3 points Imagine a 2D dataset (x_i,y_i)_{i=1,...,N} and a univariate continuous piecewise linear (CPWL) function composed of K linear pieces f such each point i belongs to the segment s(i) \in \{1,...,... 0 votes 1 answer 1k views ### Rank of Principal submatrix of annihilator matrix In the classic linear regression setting, y = X\beta+\epsilon, where y\in \mathbb{R}^n, X\in \mathbb{R}^{n\times p}, \beta\in\mathbb{R}^p, and \epsilon\sim N(0,\sigma^2I). P = X(X^\top X)^... 1 vote 1 answer 44 views ### Linear models in statistics (Rencher), Theorem 7.9.c In multiple regression context (under the general assumption we know), X=(X_1,X_2) and \beta^\prime=(\beta_1^\prime,\beta_2^\prime). Let \hat{\beta_1^*}=(X_1^\prime X_1)^{-1}X_1^\prime y be the ... 3 votes 0 answers 71 views ### Prove Max(L_1,L_2,L_3)\neq L_2:L_i=\frac{S_i^2}{n_i}+\frac{(S-S_i)^2}{n-n_i};S=\sum S_i;n=\sum n_i;\frac{S_i}{n_i}>\frac{S_{i-1}}{n_{i-1}} I will state my question first, and after that, I will write how I arrived to it. You do not really need to see how I arrived to the question, but I just thought it would be rude not to explain that. ... 4 votes 2 answers 481 views ### How to fit an ODE to data? Consider the following ODE$$ y'(t)=\alpha x(t)-\beta y(t) $$and the following datasets$$ X=\{(t_0,x_0),...,(t_n,x_n)\}\\ Y=\{(t_0,y_0),...,(t_n,y_n)\} $$How can I find \alpha and \beta that ... 1 vote 1 answer 154 views ### Can anyone see a way to linearize this function for linear regression? I have the following function:$$f(x) = \dfrac{a_1}{(x+b_1)^2+c_1} + \dfrac{a_2}{(x+b_2)^2+c_2}.From multiple measurements of f at known x values I would like find the values of a_1,a_2,b_1,... 0 votes 0 answers 18 views ### Removing Independent Variable Uncorrelated with All Other Variables in Linear Regression I've been looking at Wooldrige's Introductory Econometrics and came across the following section related to omitted variables in multiple regression here The section essentially says that if an ... 0 votes 0 answers 31 views ### Using complex phases to do linear regression The following is non-standard but was interested to see if there is value in following this path. Consider a linear regression problem without intercept, so simply, y=ax and some data is provided (... 1 vote 0 answers 23 views ### Applying Block Bootstrap to Simulate Returns and Conduct OLS Regression for Beta Calculation [closed] I am facing a methodological issue in my Master's thesis and hope someone can provide some guidance. Background: I have a time series of returns for the S&P 500, a variance swap, and a put option ... 0 votes 0 answers 22 views ### How do nonlinear relationships affect casuality determination Let's assume that I have only one independent variable and one dependent, and I have a great model with minimal error which deals well with predicting. Let's also assume that I do no know the true ... 1 vote 0 answers 61 views ### Use prediction interval to get possible values for x given y? I have the linear model y = \beta x + \alpha + \epsilon with \epsilon i.i.d normally distributed with variance \sigma^2. I fit the linear regression using OLS and compute a prediction interval ... 0 votes 0 answers 46 views ### Appropriateness of one observation per unique combination of dummy variables I am wondering what conclusions you can draw regarding the coefficients of an OLS model when you only have one observation per combination of unique dummy variables. I have seen someone else do this ... 0 votes 0 answers 29 views ### Proportion of variance in a linear regression model with a covaring predictor Given a model: \begin{align}Y_{i}=Z_{i}*\beta * X_{i} + Z_{i}\tag{Eq. 1}&\end{align} I am interested in a closed formula for the proportion of variance explained by the predictor variable X, ... 0 votes 0 answers 23 views ### Correlation vs Regression for a simple task Hello everyone and thank you for taking the time with my issue! I want to apologize in advance if my question would've fit better on stack exchange, but I decided that the question is more related to ... 1 vote 2 answers 56 views ### Weighted Least Squares versus ordinary least squares wiki page If I have a (X, Y) dataset and want to model y = f(x, \beta). In that case for OLS, I would havee(x_i, \beta) = f(x_i, \beta) - y_i$$Then obviously I would have$$SSE = \sum_{i}e(x_i, \beta)^2$... 0 votes 0 answers 16 views ### Use WLS on difference of data to calculate slope, and OLS for intercept I have a sequence of data observed in the past X days, which I want to assign higher weights to more recent ones so it makes sense to use WLS over OLS. However, does it makes sense to use WLS to just ... 0 votes 1 answer 34 views ### Linear regression solution comparison I tried to solve this exercise several times now using different methods but I am not really sure if my solution is correct. Please help me and tell me what your solution to this problem is, so I can ... 0 votes 2 answers 122 views ### Converting a multivariate linear correlation into a univariate one (predicting real estate prices in NY) I am building a prediction model of real estate properties in New York based on a few inputs: Area (size) of the property Year it was built Number of bedrooms Number of parking spots The data is ... 1 vote 0 answers 10 views ### Understanding the Equivalence Conditions between Variance of Estimators in Linear Regression Models Let$Y=\beta X+\epsilon$be standard linear model,$\hat{\beta}$and$\tilde{\beta}$estimators of$\beta$and$a\in R^{p}$. We know that: $$Var(a'\tilde{\beta}) \le Var(a'\hat{\beta}).$$ Is the ... 1 vote 0 answers 29 views ### Is there a continuous time analog of linear regression for SDEs? The ordinary linear regression $$Y_n = \alpha+\beta X_n +\epsilon_n$$ has a closed form solution for$\beta$$$\beta = \frac{\operatorname{Cov}(X, Y)}{\sigma_X^2}.$$ Question: Is there a continuous ... 0 votes 1 answer 32 views ### Omitted Variable Bias- linear regression My question relates to determining the direction of bias when the regression coefficient. The original simple linear regression model gives a coefficient$\beta_1= − 0.002108$After including an ... 0 votes 0 answers 15 views ### Verifying the Distribution of a Standardized Least Squares Estimator in Simple Linear Regression I have the following linear regression, $$Y_i=\beta_0+\beta_1 x_i+\epsilon_i$$ where$\epsilon_i^{\prime}$s are independent$N\left(0, \sigma^2\right)$random variables. Let$\hat{\beta}_i$denote ... 0 votes 1 answer 41 views ### Linear$\log$models change in betas I'm trying to solve an exercise but I find it difficult to interpret. I have a linear-$\log$model like this:$y = 1 + 0.55\ln(x) + 3z - 2.2w + \text{error term}$. I wonder what happens to$\beta_1$... 1 vote 0 answers 15 views ### A corollary of Frisch-Waugh-Lowell Theorem The formulation is just a special case of FWL. Say we have a partitioned regression,$Y=X_1\beta_1+X_2\beta_2+\epsilon$but with$X_2$be$n\times 1$and$\beta_2$a constant. Let$b_1,b_2$be two OLS ... 0 votes 0 answers 24 views ### Linear Regression with gaussian mixture prior In linear regression, we assume that the output variable is Normally distributed, i.e.,$p(y) = N (y | \mathbf{w}^T\mathbf{x}, \sigma^2_y)$. I want to assign a mixture of Gaussian prior to each ... 1 vote 0 answers 14 views ### Repository of linear and nonlinear base functions for linear regression is there any repository of linear and nonlinear base functions. I would like to compare some of them according to possibility to represent data. I think about a set as following: Name Dimension ... 1 vote 0 answers 27 views ### How to use Multiple Regression Analysis to get to linear equation I am working on a paper about MQ gas sensors and found this other study (https://www.researchgate.net/publication/... 0 votes 0 answers 7 views ### Confidence Interval for Regression Values From this article, I understand the idea behind the Confidence Interval for the Individual Response. However I don't understand the part on ... 0 votes 0 answers 20 views ### Express the regularized weight in ridge regression in terms of the linear regression solution . We would like to minimize the quantity$E_{in}(\vec{w})=\frac{1}{N}\sum_{i=1}^N(\vec{w}^{T}\vec{x_n}-y_n)^2$under the constraint$\vec{w}^T\Gamma^T\Gamma\vec{w}\leq C$where$\Gamma$is a matrix,$C$... 1 vote 0 answers 59 views ### Is the expectation of the error of any projection of$\mathbb{E}[Y\mid X]$onto subspace zero? If we consider the following linear predictor of$Y$based on$X: $$Y_{\mathbf{b}}=\boldsymbol{\Sigma}_{Y, \mathbf{X}} \boldsymbol{\Sigma}_{\mathbf{X}}^{-1}\left(\mathbf{X}-\boldsymbol{\mu}_{\mathbf{... 0 votes 1 answer 169 views ### Can a (bounded) linear least-squares problem include a scale factor in its solution? I have a system of equations.$$ \small \begin{aligned} (1 - p_0)u_0 + (q_0 - 1)v_0 + u_1 - v_1 = r_0 - c t_0 \\\\ (1 - p_1)u_1 + (q_1 - 1)v_1 + u_0 - v_0 = r_1 - c t_1 \\\\ (1 - p_2)u_2 + (q_2 - 1)... 0 votes 0 answers 11 views ### Finding parameters for function approximation I am working on some project in Matlab, where I defined some functionR(x, h, H, L, N)$and I want to find such$h, H, L, N$such that$R(x, h, H, L, N)$is approximated by some sine wave in other ... 0 votes 0 answers 21 views ### closed form expression for training error in ridge regression I'm reading the paper a random matrix approach to neural network but i'm stucked at page 5. They start from$\Sigma \in \mathbb{R}^{n\times T}$where$T$is the number of data points and$n$is the ... 0 votes 0 answers 60 views ### Is conditional expectation of the error of best linear predictor given$X$is$0$(Is it true that$y = a^*+b^*x + \eta$, where$E[\eta|x]=0$)? For simplicity, assume we are working with simple regression where the predictor$x\in\mathbb{R}$. First write$y=E[y \mid x]+u$, where the variance of$u$is a constant, and$E[u|x]=0$. I understand$...
I'm going through a derivation of the normal equation for the parameter vector $\beta$ of the linear regression model. Given a model $y = X\beta + \epsilon$, where $y$ is the vector of dependent ...