Questions tagged [regression]
This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.
2,700
questions
0
votes
0
answers
8
views
Bayesian Ridge Regression applied to binary classification. How to compare the closed-form solution with the MCMC solution?
This post is hidden. It was deleted 2 mins ago by TinkeringbellMod.
Closed. This question is off-topic. It is not currently accepting answers.
This question does not appear to be about the software ...
0
votes
0
answers
36
views
Is conditional expectation of the error of best linear predictor given $X$ is $0$ (Is it true that $y = a^*+b^*x + \eta$, where $E[\eta|x]=0$)?
For simplicity, assume we are working with simple regression where the predictor $x\in\mathbb{R}$.
First write $y=E[y \mid x]+u$, where the variance of $u$ is a constant, and $E[u|x]=0$. I understand $...
0
votes
2
answers
66
views
Why doesn't the linear regression preserve the standard deviation?
If we model $Y = \beta X$, we can estimate $\beta$ to minimize
$$\sum (Y_i - \beta X_i)^2$$
Taking derivatives and solving for 0, we get $\sum 2\beta X_i^2 - 2Y_1X_i = 0 \implies \beta = \frac{\sum ...
0
votes
0
answers
20
views
Endogeneity Analysis without the access of raw data?
I currently have the correlation/covariance matrix for a set of variables and the results of regression analysis but lack access to the raw dataset. Under these constraints, is it feasible to conduct ...
0
votes
0
answers
19
views
2SLS and Instrumental Variables
I don't understanding the estimation of IV thoroughoutly. The following is my puzzle.
In the first stage, we have
$$D_i = \alpha + \beta Z_i + u_i$$
then we get predicotrs $\hat{D_i}$.
In my mind, $\...
0
votes
0
answers
70
views
In regression, why is the order of terms reversed from mathematical convention? [closed]
In the context of regression, I have found that result is usually presented with the intercept first and then the higher-order terms in ascending order; e.g. for polynomial regression:
$$ y = b_0 + ...
0
votes
1
answer
35
views
Population OLS coefficient in simple regression?
The population OLS coefficient for some $X_i \in \mathbb{R}^d, Y \in \mathbb{R}$ for the model $Y = \beta’X + e$ is defined as
$$
\beta =\mathbb{E}[X_iX_i']^{-1}\mathbb{E}[X_iY_i]
$$
and if $X$ is a ...
0
votes
0
answers
37
views
Expectation $X|Y$ in a regression problem: Simplify $E ( K_h(x-x_i) | y_{i})$
Let
\begin{equation}
Y = f(X) + \epsilon, \ \epsilon \sim \left(0,1 \right),
\end{equation}
where
\begin{equation}
f(X) = E \left(Y|X \right)
\end{equation}
is some nonparametric regression function. ...
0
votes
0
answers
10
views
In sparse ridge regression, why we have this property
In ridge regression, we can estimate $\hat y$=$X(X^TX+\lambda I)^{-1}y$,where $X$ is covariate matrix with n rows and p column. And my teacher says that we can use SVD to rewrite this formula as:$\hat ...
0
votes
0
answers
17
views
Creating a regression model of scaling data from five points for ATAR
I am looking to calculate the scaled results for the ATAR (Australian Tertiary Admission Rank) subjects that someone inputs from the recently released 2023 data. For example if someone got a 59.00 in ...
1
vote
1
answer
30
views
What considerations should I have into account when linearizing a non-linear model for linear regressions?
I'm looking for some bibliography about what I should/must have into account when I have a model and experimental data that can be expressed in a way such that I can use a linear regression method to ...
0
votes
0
answers
82
views
Using sum of squares regression for fitting a sinusoid
I am trying to use the least squares regression to fit a curve to a table of values representing a sine wave. Similar to the question:
Least squares regression of sine wave
Except I want to fit a more ...
0
votes
0
answers
17
views
Convergence of coefficients in multivariate regression
In this thread, the convergence of coefficient for univariate dependent variable is proven. I wonder, assuming the same setup, how can the convergence be extended to multivariate as:
$$Y=XW+\epsilon$$
...
0
votes
2
answers
61
views
How do you find the equation for a best fit line that passes a specific point?
All I have been given are a set of x and y coordinates, and another point that the best-fit line should pass through (not the origin). I know how to use linear regression to find the slope and ...
0
votes
0
answers
23
views
Calculation of inverse regression [migrated]
I am reading the paper on consistency and sparsity for sliced inverse regression in high dimensions(https://doi.org/10.1214/17-AOS1561). I'm curious about the conclusion on the page 585
Let $y=\beta^...
3
votes
0
answers
42
views
Sqrt LASSO vs LASSO
In this paper they talk about Sqrt-LASSO which is simply just trying to minimize $\|Ax-b\|_2 + \lambda\|x\|_1$ rather than the regular LASSO $\|Ax-b\|_2^2 + \lambda\|x\|_1$.
Can anyone point out the ...
0
votes
0
answers
12
views
How is it Possible to Generate Tables and Graphs Optimizing the Associated Shichman-Hodges Slope $λ_j$ and Verify the Regression Approach
1. Introduction
Linear regression Equations for $λ_j$ are derived here:
How is it Possible to Optimize the Shichman-Hodges Slope Parameters from the Left and Right using Least Square Linear ...
0
votes
0
answers
35
views
Regression with a Nonlinear ODE (exponentiated derivative)
I have data (x, y) which I believe is generated by a differential equation of the following form:
$$A\frac{d^2y}{dx^2}+B(\frac{dy}{dx})^{C}+D=0$$
I can estimate the initial values, so given $A$, $B$, $...
0
votes
0
answers
37
views
Fitting a curve between 2 poses (position and orientation) when the length of the curve is known
I have two visual aruco markers on a flexible line and a camera. I can calculate the pose (position and orientation) of each marker. I have measured the distance between the markers when the line is ...
0
votes
2
answers
83
views
Exponential regression
I am trying to figure out a way to get the function parameters from the different points of an exponential function $f(x)=ab^x + c$. I figured it out for $f(x) = ab^x$, but can't find the answer when ...
0
votes
3
answers
115
views
Find a line of regression with the following points
Find a line of regression in the form
\begin{align}
y &= \frac{c}{1+a e^{-b(x)}} \\[4mm]
\end{align}
with the points
$(69.99, 44), (84.71, 54), (93.66, 60), (116.36, 74)$
I can easily find the ...
0
votes
0
answers
9
views
Derivation Regression Factor Analysis
can someone derive the last formula for the variance ?
I dont get the result and its in the script of my university:
"
(ii) Regression methodology: now assume the model $\mathrm{x}=\boldsymbol{\...
0
votes
0
answers
32
views
Approximating the half-life of a shock to a system?
I found the following statement in here regarding the effect of twice lagged differences of CO2 ($\Delta C$) in the atmosphere on the once lagged values, i.e.
$$\Delta C_{\text{ @ }t=-1}= 0.83 \times ...
2
votes
1
answer
70
views
Cubic equation coefficients from 4 points
For a cubic curve (Bezier) of the form: $ax^3 + bx^2 + cx + d = y$.
I have a given set of four points $P_0, P_1, P_2, P_3$. Such that, $P_0$ is the origin and the other three are equidistant along the ...
0
votes
1
answer
33
views
Can a single dummy variable be made to meet multiple criteria?
Let say I have criteria $1, 2, 3,$ and $4$. I would like the Dummy variable to be 1 only if a certain minimum amount of criteria are met. For example, if $3$ of the $4$ are true, then Dummy$ =1$. If $...
3
votes
2
answers
173
views
How to fit and ODE to data?
Consider the following ODE
$$
y'(t)=\alpha x(t)-\beta y(t)
$$
and the following datasets
$$
X=\{(t_0,x_0),...,(t_n,x_n)\}\\
Y=\{(t_0,y_0),...,(t_n,y_n)\}
$$
How can I find $\alpha$ and $\beta$ that ...
-1
votes
1
answer
68
views
Matrix product in linear regression [closed]
I do not understand why $(X^TX)^{-1}X^T * (X^TX)^{-1}X^T = (X^TX)^{=1}*I$
I keep trying to work it out and I just don't see it. This equality shows up when deriving the variance formula for ...
0
votes
1
answer
69
views
Derivation of $SSE =S_{yy} - S_{xy}^2/S_{xx}$
I have seen this result and I am trying to figure out how to derive it from $SSE = \sum(Y_i - \hat{Y})^2$. I know that $r = \frac{s_{xy}}{\sqrt{s_{xx}s_{yy}}} $ and I have seen online to use and ...
0
votes
0
answers
31
views
Distinguishing various types of regression
I want to understand the pairwise relationship between four types of regression: Bayesian Linear Regression, Gaussian Process Regression, Kernel Regression (Nadaraya-Watson), and Kernel Ridge ...
1
vote
0
answers
40
views
$l_{\infty}$ convergence of OLS prediction error
I have found many resources talking about the $l_2$ convergence of OLS estimation error and prediction error.
I have also read about the analysis of the $l_2$ convergence of lasso estimation error and ...
0
votes
0
answers
15
views
Is OLS estimator sub-Gaussian?
Suppose we have $n$ observations $(x_i,y_i)\in\mathbb{R}^{p+1}$. Suppose the underlying model is $y = \eta^{*T} x + \varepsilon$, and $x, \varepsilon$ are sub-Gaussian with parameter $\sigma_x, \...
1
vote
1
answer
65
views
Seber, Ex. 1.b.3 - Calculating variance and showing unbiasedness
Exercise 3 of Sebers Linear Regression Analysis states the following:
I tried to solve both problems but only managed to (kinda) solve the first one and have no idea how to tackle the second one. ...
0
votes
1
answer
108
views
likelihood of data given parameter or likelihood of parameter given data?
What is more correct? Likelihood of data given parameter or likelihood of parameter given data? In https://en.wikipedia.org/wiki/Likelihood_function for instance, we see both
likelihood of $\hat{\...
0
votes
1
answer
32
views
Linear $\log$ models change in betas
I'm trying to solve an exercise but I find it difficult to interpret.
I have a linear-$\log$ model like this: $y = 1 + 0.55\ln(x) + 3z - 2.2w + \text{error term}$.
I wonder what happens to $\beta_1$ ...
0
votes
1
answer
49
views
Inequality for the trace of the hat matrix in Ridge regression
I was recently reviewing a research paper and came across an inequality expressed as follows:
\begin{align}
& \text{tr}\Big[\Big(\frac{1}{np}X^\top X + \rho B\Big)^{-1} \Big(\frac{1}{np}X^\top X\...
1
vote
0
answers
48
views
Find an equation that fits closely to these points in 3D space
I am trying to find an equation $f(x,y) \rightarrow z$ that can be used to return the values in this table, where we'll say the horizontal heading is the $x$ values and the vertical $y$.
My thinking ...
0
votes
1
answer
46
views
Independence of pure error and lack-of-fit error in simple linear regression with repeated observations
Let $x_1,\ldots,x_n$ be distinct regressor variables.
For each $x_i$, there are $n_i$ observations $Y_{i1},\ldots,Y_{in_i}$ such that
$$Y_{ij}=\alpha x_i+\beta+\epsilon_{ij},$$
where $\epsilon_{ij}\...
0
votes
1
answer
55
views
Proving Divergence Free Behavior of Matrix Valued Radial Basis Function
I'm am using radial basis functions to interpolate magnetic fields, which are divergence free. I have found several research papers that state that the following takes a scalar valued Radial Basis ...
0
votes
0
answers
72
views
Prove that variance of estimator in GLS is less than variance of estimator in OLS
In my statistics class we showed that for Ordinary Least Squares regression:
$$
var[\hat{\beta}_{OLS}] = (X^TX)^{-1}X^T\sigma^2X(X^TX)^{-1}
$$
and for Generalized Least Squares regression (ie. ...
1
vote
0
answers
23
views
Measuring the total influence of a subset of observations on a particular regression coefficient.
Suppose I have run a multiple regression model:
Y = B0 + x1B1 + x2B2 +..+ xnBn, weighted by w, from a dataset with such covariates and the weight variable of size N. Say there is another column in the ...
1
vote
0
answers
50
views
Lower bound using Largest Singular Value of Pseudo-Inverse
For uni I've got this exercise where i need to prove the following:
$$\sigma_{max}(X^\dagger)^2 \ ||Xw-Xw^* ||_2^2 \geq ||w - w^* ||_2^2$$
where $\sigma_{max}(A) = max_{u \in \mathbb{R}^d \ ||u||_2 = ...
0
votes
0
answers
23
views
Theoretical analysis for regression imputation
I have been looking at the literature of missing data problems. Many of them talks about multiple imputation, but I'm just interested in what bias occur and how to correct the bias if we employ a ...
0
votes
0
answers
39
views
Validation of Normal Equations with Pseudo-Inverses
In an exercise for uni I am asked to prove that $w^* \in \mathbb{R}^d$ is a solution to $\min_{w \in \mathbb{R}^d}\lVert y - Xw \rVert_{2}^{2}$ if and only if $X^\dagger X w^* = X^\dagger y$, where $X^...
0
votes
0
answers
28
views
Multiple Regression Analysis Residuals
Can you please review my analysis based on the following plots. It is a multiple predictors linear regression model The model form assumption is met based on the plots above. we can see that there is ...
0
votes
0
answers
57
views
Conditional expectation on continuos random variable with zero density obtained from non-zero density variables.
Let $Y$, $X_1$ and $X_2$ be three continuous real random variable with $f(x_1, x_2) >0$ everywhere on $R^2$ and denote by $g(x_1, x_2) = E[Y|X_1 = x_1, X_2 = x_2]$. Then $g(0,0) = E[Y|X_1 = 0, X_2 =...
1
vote
0
answers
30
views
Computational complexity of weighted least squares
According to this answer, for $N$ observations and $C$ variables, the computational complexity of solving a linear regression problem $(X^\intercal X)^{-1}X^\intercal \mathbf y$ is $O(C^2N)$.
I am not ...
0
votes
0
answers
14
views
Regress on matrix on vector to obtain rank-one approximation in Partial Least Square
I'm trying to understand the equation for partial least squares (PLS) regression from the paper "A survey of partial least squares (PLS) methods, with emphasis on the two-blocks case". I'm ...
3
votes
1
answer
98
views
Are these two definitions of the coefficient of determination $R^2$ equal?
I want to do multiple linear regression as explained on this Wikipedia site: I am given data
$$
yx=(~(y_1,x_{11},\ldots,x_{1p}),\ldots, (y_n,x_{n1},\ldots,x_{np})~)
$$
of $n$-many samples where for ...
0
votes
0
answers
23
views
Determine type of function from points
Assume $A$ is a set of points in $\mathbb{R^2}$. The points of $A$ were generated from a function that is either linear, polynomial, or exponential, but you don't know which. Is it possible to ...
1
vote
1
answer
22
views
Is there a data-driven method to solve omitted variable problem?
Suppose $Z$ is correlated with predictor $X$ and it is unobserved. One classical way of determining the coefficient $\beta_1$ in $Y = \beta_0 + \beta_1 X + \epsilon$ is finding an instrumental ...