Questions tagged [linear-regression]
For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.
1,320
questions
0
votes
0
answers
22
views
Bishop gradient calculation
In section 3.1.1 of Pattern Recognition and Machine Learning by Christopher Bishop, it is written that
$$\ln p(\mathbf{t} | \mathbf{w}, \beta) = \frac{N}{2} \ln \beta - \frac{N}{2} \ln (2 \pi) - \beta ...
- 37.6k
-1
votes
0
answers
12
views
Passing a linear regression equation around 5 data points, making sure the regression equation passes through the fifth point. [closed]
I have a five point data set which was created by taking the natural log of a data set where x = ln(a) and y = ln(b). The transformed data set is x = {1.61, 2.30, 2.71, 3.40, 4.09} and y = {2.05, 1.43,...
- 7
2
votes
1
answer
1k
views
Linear regression with integer function
I have a dataset, that only takes integer values ($x$ and $y$ coordiantes). E.g. my data is the following:
$x = (1,2,2,3,3), y = (1,2,3,3,4)$. I want to make a linear regression through the data, i.e....
CommunityBot
- 1
1
vote
1
answer
2k
views
What is the efficiency of OLS estimators?
In page 3, of Asymptotic Theory for Econometricians, the following assumptions of OLS are defined:
OLS model: $Y= X\beta+\epsilon$
$X$ is a nonstochastic and finite n x k matrix, n > k.
$X'X$ is ...
CommunityBot
- 1
0
votes
0
answers
25
views
Regression coefficients without matrix inversion
We can find coefficients β in multiple linear regression:
β = (XTX)-1XTy
I see a lot of related questions on this forum asking "How do I invert this matrix?". This is followed by an answer ...
- 109
1
vote
0
answers
38
views
Why can Total Least Squares Regression not have a solution?
I was looking at this reference on TLS regression. I understand the math which says that there can be problems where no solution exists. Based on my understanding, the minimum perturbation to the $C$ ...
- 707
12
votes
1
answer
22k
views
Derivative of Mean Squared Error
I'm studying with a book and I'm at the Linear Regression part. The author is showing that we have to calculate the derivative of each part of the equation that leads to the loss.
But he's using the ...
- 111
1
vote
1
answer
1k
views
How to combine several linear regression function into one
I have the relationship between f and each of other five variables, the relationships are acquired by linear regression and are as follow
$f = a{\tiny1}x{\tiny1} + b{\tiny1}$
$f = a{\tiny2}x{\tiny2} ...
CommunityBot
- 1
0
votes
0
answers
82
views
How to calculate the trace of this matrix?
How do we calculate the trace of $(\mathbb{I}-X(X^\prime X)^{-1}X^\prime)\mathbb{J}_n$? This question stemmed from the below problem I came across:
Suppose we have the linear regression model: $y_i=...
- 403
1
vote
2
answers
31
views
Why does Least Absolute Deviation regression exactly fit n measureemnts for a linear system with n independent variables?
I am applying LAD regression to conduct some research. I have the following questions regarding LAD:
I know LAD exactly fits n measurements for a linear system with n variables. But I cannot easily ...
- 1,851
0
votes
1
answer
19
views
Help in understanding Bayesian linear regression.
I am unable to use Bayesian Regression in the following
question. So far I have performed the following calculations:
We have our data points Y, X that are ...
- 1,851
0
votes
0
answers
15
views
Prediction intervals for simple linear regression
I want to understand how to construct the test statistic for the case of predictive inference for a simple linear regression model and would be grateful if someone might confirm if my derivation is ...
1
vote
1
answer
34
views
$R^2$ of regression of residuals when adding an uncorrelated regressor
Suppose we linearly regress $Y$ onto $X_1$, obtaining residuals $\epsilon_1,\ldots,\epsilon_n$. Suppose further that $X_2$ is uncorrelated with $X_1$, and we linearly regress $X_2$ onto $X_1$ (with ...
- 1,851
1
vote
1
answer
32
views
Local Linear Fit
What do you call it when you have a set of known $(x,y)$ data points and you estimate a $y$-value for a given $x$-value by performing a linear fit between its two known neighbor $(x,y)$ points?
As an ...
- 531
4
votes
1
answer
130
views
Sqrt LASSO vs LASSO
In the paper Square Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming they talk about Sqrt-LASSO which is simply just trying to minimize $\|Ax-b\|_2 + \lambda\|x\|_1$ rather than ...
- 9,057
0
votes
0
answers
84
views
Continuous piecewise linear (CPWL) function fitting a dataset for which each linear piece contains at least 3 points
Imagine a 2D dataset $(x_i,y_i)_{i=1,...,N}$ and a univariate continuous piecewise linear (CPWL) function composed of $K$ linear pieces $f$ such each point $i$ belongs to the segment $s(i) \in \{1,...,...
0
votes
1
answer
1k
views
Rank of Principal submatrix of annihilator matrix
In the classic linear regression setting,
$y = X\beta+\epsilon$, where $y\in \mathbb{R}^n$, $X\in \mathbb{R}^{n\times p}$, $\beta\in\mathbb{R}^p$, and $\epsilon\sim N(0,\sigma^2I)$.
$P = X(X^\top X)^...
CommunityBot
- 1
1
vote
1
answer
44
views
Linear models in statistics (Rencher), Theorem 7.9.c
In multiple regression context (under the general assumption we know), $X=(X_1,X_2)$ and $\beta^\prime=(\beta_1^\prime,\beta_2^\prime)$. Let $\hat{\beta_1^*}=(X_1^\prime X_1)^{-1}X_1^\prime y$ be the ...
- 2,462
3
votes
0
answers
71
views
Prove $Max(L_1,L_2,L_3)\neq L_2$:$L_i=\frac{S_i^2}{n_i}+\frac{(S-S_i)^2}{n-n_i}$;$S=\sum S_i$;$n=\sum n_i$;$\frac{S_i}{n_i}>\frac{S_{i-1}}{n_{i-1}}$
I will state my question first, and after that, I will write how I arrived to it.
You do not really need to see how I arrived to the question, but I just thought it would be rude not to explain that.
...
- 31
4
votes
2
answers
481
views
How to fit an ODE to data?
Consider the following ODE
$$
y'(t)=\alpha x(t)-\beta y(t)
$$
and the following datasets
$$
X=\{(t_0,x_0),...,(t_n,x_n)\}\\
Y=\{(t_0,y_0),...,(t_n,y_n)\}
$$
How can I find $\alpha$ and $\beta$ that ...
- 34.7k
1
vote
1
answer
154
views
Can anyone see a way to linearize this function for linear regression?
I have the following function:
$$f(x) = \dfrac{a_1}{(x+b_1)^2+c_1} + \dfrac{a_2}{(x+b_2)^2+c_2}.$$
From multiple measurements of $f$ at known $x$ values I would like find the values of $a_1,a_2,b_1,...
- 67.2k
0
votes
0
answers
18
views
Removing Independent Variable Uncorrelated with All Other Variables in Linear Regression
I've been looking at Wooldrige's Introductory Econometrics and came across the following section related to omitted variables in multiple regression here
The section essentially says that if an ...
0
votes
0
answers
31
views
Using complex phases to do linear regression
The following is non-standard but was interested to see if there is value in following this path.
Consider a linear regression problem without intercept, so simply, $y=ax$ and some data is provided $(...
- 307
1
vote
0
answers
23
views
Applying Block Bootstrap to Simulate Returns and Conduct OLS Regression for Beta Calculation [closed]
I am facing a methodological issue in my Master's thesis and hope someone can provide some guidance. Background: I have a time series of returns for the S&P 500, a variance swap, and a put option ...
- 11
0
votes
0
answers
22
views
How do nonlinear relationships affect casuality determination
Let's assume that I have only one independent variable and one dependent, and I have a great model with minimal error which deals well with predicting. Let's also assume that I do no know the true ...
- 91
1
vote
0
answers
61
views
Use prediction interval to get possible values for x given y?
I have the linear model $y = \beta x + \alpha + \epsilon$ with $\epsilon$ i.i.d normally distributed with variance $\sigma^2$.
I fit the linear regression using OLS and compute a prediction interval ...
- 1,571
0
votes
0
answers
46
views
Appropriateness of one observation per unique combination of dummy variables
I am wondering what conclusions you can draw regarding the coefficients of an OLS model when you only have one observation per combination of unique dummy variables.
I have seen someone else do this ...
0
votes
0
answers
29
views
Proportion of variance in a linear regression model with a covaring predictor
Given a model:
\begin{align}Y_{i}=Z_{i}*\beta * X_{i} + Z_{i}\tag{Eq. 1}&\end{align}
I am interested in a closed formula for the proportion of variance explained by the predictor variable $X$, ...
- 181
0
votes
0
answers
23
views
Correlation vs Regression for a simple task
Hello everyone and thank you for taking the time with my issue! I want to apologize in advance if my question would've fit better on stack exchange, but I decided that the question is more related to ...
- 1
1
vote
2
answers
56
views
Weighted Least Squares versus ordinary least squares wiki page
If I have a $(X, Y)$ dataset and want to model $y = f(x, \beta)$. In that case for OLS, I would have
$$e(x_i, \beta) = f(x_i, \beta) - y_i$$
Then obviously I would have
$$SSE = \sum_{i}e(x_i, \beta)^2$...
CommunityBot
- 1
0
votes
0
answers
16
views
Use WLS on difference of data to calculate slope, and OLS for intercept
I have a sequence of data observed in the past X days, which I want to assign higher weights to more recent ones so it makes sense to use WLS over OLS.
However, does it makes sense to use WLS to just ...
- 39
0
votes
1
answer
34
views
Linear regression solution comparison
I tried to solve this exercise several times now using different methods but I am not really sure if my solution is correct.
Please help me and tell me what your solution to this problem is, so I can ...
- 531
0
votes
2
answers
122
views
Converting a multivariate linear correlation into a univariate one (predicting real estate prices in NY)
I am building a prediction model of real estate properties in New York based on a few inputs:
Area (size) of the property
Year it was built
Number of bedrooms
Number of parking spots
The data is ...
- 10.2k
1
vote
0
answers
10
views
Understanding the Equivalence Conditions between Variance of Estimators in Linear Regression Models
Let $Y=\beta X+\epsilon$ be standard linear model, $\hat{\beta}$ and $\tilde{\beta}$ estimators of $\beta$ and $a\in R^{p}$.
We know that:
$$
Var(a'\tilde{\beta}) \le Var(a'\hat{\beta}).
$$
Is the ...
- 333
1
vote
0
answers
29
views
Is there a continuous time analog of linear regression for SDEs?
The ordinary linear regression
$$Y_n = \alpha+\beta X_n +\epsilon_n$$
has a closed form solution for $\beta$
$$\beta = \frac{\operatorname{Cov}(X, Y)}{\sigma_X^2}.$$
Question:
Is there a continuous ...
- 1,161
0
votes
1
answer
32
views
Omitted Variable Bias- linear regression
My question relates to determining the direction of bias when the regression coefficient.
The original simple linear regression model gives a coefficient $\beta_1= − 0.002108$
After including an ...
- 461
0
votes
0
answers
15
views
Verifying the Distribution of a Standardized Least Squares Estimator in Simple Linear Regression
I have the following linear regression,
$$
Y_i=\beta_0+\beta_1 x_i+\epsilon_i
$$
where $\epsilon_i^{\prime}$ s are independent $N\left(0, \sigma^2\right)$ random variables. Let $\hat{\beta}_i$ denote ...
- 461
0
votes
1
answer
41
views
Linear $\log$ models change in betas
I'm trying to solve an exercise but I find it difficult to interpret.
I have a linear-$\log$ model like this: $y = 1 + 0.55\ln(x) + 3z - 2.2w + \text{error term}$.
I wonder what happens to $\beta_1$ ...
- 16.6k
1
vote
0
answers
15
views
A corollary of Frisch-Waugh-Lowell Theorem
The formulation is just a special case of FWL. Say we have a partitioned regression, $Y=X_1\beta_1+X_2\beta_2+\epsilon$ but with $X_2$ be $n\times 1$ and $\beta_2$ a constant.
Let $b_1,b_2$ be two OLS ...
- 305
0
votes
0
answers
24
views
Linear Regression with gaussian mixture prior
In linear regression, we assume that the output variable is Normally distributed, i.e., $p(y) = N (y | \mathbf{w}^T\mathbf{x}, \sigma^2_y)$. I want to assign a mixture of Gaussian prior to each ...
- 15
1
vote
0
answers
14
views
Repository of linear and nonlinear base functions for linear regression
is there any repository of linear and nonlinear base functions. I would like to compare some of them according to possibility to represent data. I think about a set as following:
Name
Dimension
...
- 11
1
vote
0
answers
27
views
How to use Multiple Regression Analysis to get to linear equation
I am working on a paper about MQ gas sensors and found this other study (https://www.researchgate.net/publication/...
- 11
0
votes
0
answers
7
views
Confidence Interval for Regression Values
From this article, I understand the idea behind the Confidence Interval for the Individual Response.
However I don't understand the part on ...
- 15
0
votes
0
answers
20
views
Express the regularized weight in ridge regression in terms of the linear regression solution .
We would like to minimize the quantity
$E_{in}(\vec{w})=\frac{1}{N}\sum_{i=1}^N(\vec{w}^{T}\vec{x_n}-y_n)^2$
under the constraint
$\vec{w}^T\Gamma^T\Gamma\vec{w}\leq C$ where $\Gamma$ is a matrix, $C$ ...
- 11
1
vote
0
answers
59
views
Is the expectation of the error of any projection of $\mathbb{E}[Y\mid X]$ onto subspace zero?
If we consider the following linear predictor of $Y$ based on $X$:
$$
Y_{\mathbf{b}}=\boldsymbol{\Sigma}_{Y, \mathbf{X}} \boldsymbol{\Sigma}_{\mathbf{X}}^{-1}\left(\mathbf{X}-\boldsymbol{\mu}_{\mathbf{...
- 617
0
votes
1
answer
169
views
Can a (bounded) linear least-squares problem include a scale factor in its solution?
I have a system of equations.
$$
\small
\begin{aligned}
(1 - p_0)u_0 + (q_0 - 1)v_0 + u_1 - v_1 = r_0 - c t_0 \\\\
(1 - p_1)u_1 + (q_1 - 1)v_1 + u_0 - v_0 = r_1 - c t_1 \\\\
(1 - p_2)u_2 + (q_2 - 1)...
- 34.7k
0
votes
0
answers
11
views
Finding parameters for function approximation
I am working on some project in Matlab, where I defined some function $R(x, h, H, L, N)$ and I want to find such $h, H, L, N$ such that $R(x, h, H, L, N)$ is approximated by some sine wave in other ...
- 171
0
votes
0
answers
21
views
closed form expression for training error in ridge regression
I'm reading the paper a random matrix approach to neural network but i'm stucked at page 5.
They start from
$\Sigma \in \mathbb{R}^{n\times T}$ where $T$ is the number of data points and $n$ is the ...
- 155
0
votes
0
answers
60
views
Is conditional expectation of the error of best linear predictor given $X$ is $0$ (Is it true that $y = a^*+b^*x + \eta$, where $E[\eta|x]=0$)?
For simplicity, assume we are working with simple regression where the predictor $x\in\mathbb{R}$.
First write $y=E[y \mid x]+u$, where the variance of $u$ is a constant, and $E[u|x]=0$. I understand $...
- 617
0
votes
0
answers
22
views
Derivation of normal equation for linear regression parameters
I'm going through a derivation of the normal equation for the parameter vector $\beta$ of the linear regression model. Given a model $y = X\beta + \epsilon$, where $y$ is the vector of dependent ...
- 183