Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Bayesian Ridge Regression applied to binary classification. How to compare the closed-form solution with the MCMC solution?

This post is hidden. It was deleted 2 mins ago by TinkeringbellMod. Closed. This question is off-topic. It is not currently accepting answers. This question does not appear to be about the software ...
Emily's user avatar
  • 1
0 votes
0 answers
36 views

Is conditional expectation of the error of best linear predictor given $X$ is $0$ (Is it true that $y = a^*+b^*x + \eta$, where $E[\eta|x]=0$)?

For simplicity, assume we are working with simple regression where the predictor $x\in\mathbb{R}$. First write $y=E[y \mid x]+u$, where the variance of $u$ is a constant, and $E[u|x]=0$. I understand $...
maskeran's user avatar
  • 511
0 votes
2 answers
66 views

Why doesn't the linear regression preserve the standard deviation?

If we model $Y = \beta X$, we can estimate $\beta$ to minimize $$\sum (Y_i - \beta X_i)^2$$ Taking derivatives and solving for 0, we get $\sum 2\beta X_i^2 - 2Y_1X_i = 0 \implies \beta = \frac{\sum ...
ammo 45's user avatar
0 votes
0 answers
20 views

Endogeneity Analysis without the access of raw data?

I currently have the correlation/covariance matrix for a set of variables and the results of regression analysis but lack access to the raw dataset. Under these constraints, is it feasible to conduct ...
Harshavardhana Srinivasan's user avatar
0 votes
0 answers
19 views

2SLS and Instrumental Variables

I don't understanding the estimation of IV thoroughoutly. The following is my puzzle. In the first stage, we have $$D_i = \alpha + \beta Z_i + u_i$$ then we get predicotrs $\hat{D_i}$. In my mind, $\...
HSINSHUO's user avatar
0 votes
0 answers
70 views

In regression, why is the order of terms reversed from mathematical convention? [closed]

In the context of regression, I have found that result is usually presented with the intercept first and then the higher-order terms in ascending order; e.g. for polynomial regression: $$ y = b_0 + ...
Grendel13G's user avatar
0 votes
1 answer
35 views

Population OLS coefficient in simple regression?

The population OLS coefficient for some $X_i \in \mathbb{R}^d, Y \in \mathbb{R}$ for the model $Y = \beta’X + e$ is defined as $$ \beta =\mathbb{E}[X_iX_i']^{-1}\mathbb{E}[X_iY_i] $$ and if $X$ is a ...
tmako's user avatar
  • 196
0 votes
0 answers
37 views

Expectation $X|Y$ in a regression problem: Simplify $E ( K_h(x-x_i) | y_{i})$

Let \begin{equation} Y = f(X) + \epsilon, \ \epsilon \sim \left(0,1 \right), \end{equation} where \begin{equation} f(X) = E \left(Y|X \right) \end{equation} is some nonparametric regression function. ...
A_Mondial's user avatar
0 votes
0 answers
10 views

In sparse ridge regression, why we have this property

In ridge regression, we can estimate $\hat y$=$X(X^TX+\lambda I)^{-1}y$,where $X$ is covariate matrix with n rows and p column. And my teacher says that we can use SVD to rewrite this formula as:$\hat ...
MengXing Chen's user avatar
0 votes
0 answers
17 views

Creating a regression model of scaling data from five points for ATAR

I am looking to calculate the scaled results for the ATAR (Australian Tertiary Admission Rank) subjects that someone inputs from the recently released 2023 data. For example if someone got a 59.00 in ...
Wosca's user avatar
  • 3
1 vote
1 answer
30 views

What considerations should I have into account when linearizing a non-linear model for linear regressions?

I'm looking for some bibliography about what I should/must have into account when I have a model and experimental data that can be expressed in a way such that I can use a linear regression method to ...
CosmeticMichu's user avatar
0 votes
0 answers
82 views

Using sum of squares regression for fitting a sinusoid

I am trying to use the least squares regression to fit a curve to a table of values representing a sine wave. Similar to the question: Least squares regression of sine wave Except I want to fit a more ...
Adam Labuš's user avatar
0 votes
0 answers
17 views

Convergence of coefficients in multivariate regression

In this thread, the convergence of coefficient for univariate dependent variable is proven. I wonder, assuming the same setup, how can the convergence be extended to multivariate as: $$Y=XW+\epsilon$$ ...
statwoman's user avatar
  • 253
0 votes
2 answers
61 views

How do you find the equation for a best fit line that passes a specific point?

All I have been given are a set of x and y coordinates, and another point that the best-fit line should pass through (not the origin). I know how to use linear regression to find the slope and ...
RehearsedToast's user avatar
0 votes
0 answers
23 views

Calculation of inverse regression [migrated]

I am reading the paper on consistency and sparsity for sliced inverse regression in high dimensions(https://doi.org/10.1214/17-AOS1561). I'm curious about the conclusion on the page 585 Let $y=\beta^...
Gareth's user avatar
  • 11
3 votes
0 answers
42 views

Sqrt LASSO vs LASSO

In this paper they talk about Sqrt-LASSO which is simply just trying to minimize $\|Ax-b\|_2 + \lambda\|x\|_1$ rather than the regular LASSO $\|Ax-b\|_2^2 + \lambda\|x\|_1$. Can anyone point out the ...
jeffj1355's user avatar
0 votes
0 answers
12 views

How is it Possible to Generate Tables and Graphs Optimizing the Associated Shichman-Hodges Slope $λ_j$ and Verify the Regression Approach

1. Introduction Linear regression Equations for $λ_j$ are derived here: How is it Possible to Optimize the Shichman-Hodges Slope Parameters from the Left and Right using Least Square Linear ...
Stephen Elliott's user avatar
0 votes
0 answers
35 views

Regression with a Nonlinear ODE (exponentiated derivative)

I have data (x, y) which I believe is generated by a differential equation of the following form: $$A\frac{d^2y}{dx^2}+B(\frac{dy}{dx})^{C}+D=0$$ I can estimate the initial values, so given $A$, $B$, $...
Bananacus's user avatar
0 votes
0 answers
37 views

Fitting a curve between 2 poses (position and orientation) when the length of the curve is known

I have two visual aruco markers on a flexible line and a camera. I can calculate the pose (position and orientation) of each marker. I have measured the distance between the markers when the line is ...
Wesley's user avatar
  • 1
0 votes
2 answers
83 views

Exponential regression

I am trying to figure out a way to get the function parameters from the different points of an exponential function $f(x)=ab^x + c$. I figured it out for $f(x) = ab^x$, but can't find the answer when ...
user_31415926535's user avatar
0 votes
3 answers
115 views

Find a line of regression with the following points

Find a line of regression in the form \begin{align} y &= \frac{c}{1+a e^{-b(x)}} \\[4mm] \end{align} with the points $(69.99, 44), (84.71, 54), (93.66, 60), (116.36, 74)$ I can easily find the ...
ell's user avatar
  • 9
0 votes
0 answers
9 views

Derivation Regression Factor Analysis

can someone derive the last formula for the variance ? I dont get the result and its in the script of my university: " (ii) Regression methodology: now assume the model $\mathrm{x}=\boldsymbol{\...
mth's user avatar
  • 1
0 votes
0 answers
32 views

Approximating the half-life of a shock to a system?

I found the following statement in here regarding the effect of twice lagged differences of CO2 ($\Delta C$) in the atmosphere on the once lagged values, i.e. $$\Delta C_{\text{ @ }t=-1}= 0.83 \times ...
JAP's user avatar
  • 553
2 votes
1 answer
70 views

Cubic equation coefficients from 4 points

For a cubic curve (Bezier) of the form: $ax^3 + bx^2 + cx + d = y$. I have a given set of four points $P_0, P_1, P_2, P_3$. Such that, $P_0$ is the origin and the other three are equidistant along the ...
Norma's user avatar
  • 23
0 votes
1 answer
33 views

Can a single dummy variable be made to meet multiple criteria?

Let say I have criteria $1, 2, 3,$ and $4$. I would like the Dummy variable to be 1 only if a certain minimum amount of criteria are met. For example, if $3$ of the $4$ are true, then Dummy$ =1$. If $...
MLux's user avatar
  • 1
3 votes
2 answers
173 views

How to fit and ODE to data?

Consider the following ODE $$ y'(t)=\alpha x(t)-\beta y(t) $$ and the following datasets $$ X=\{(t_0,x_0),...,(t_n,x_n)\}\\ Y=\{(t_0,y_0),...,(t_n,y_n)\} $$ How can I find $\alpha$ and $\beta$ that ...
sam wolfe's user avatar
  • 3,337
-1 votes
1 answer
68 views

Matrix product in linear regression [closed]

I do not understand why $(X^TX)^{-1}X^T * (X^TX)^{-1}X^T = (X^TX)^{=1}*I$ I keep trying to work it out and I just don't see it. This equality shows up when deriving the variance formula for ...
AdmiralMunson's user avatar
0 votes
1 answer
69 views

Derivation of $SSE =S_{yy} - S_{xy}^2/S_{xx}$

I have seen this result and I am trying to figure out how to derive it from $SSE = \sum(Y_i - \hat{Y})^2$. I know that $r = \frac{s_{xy}}{\sqrt{s_{xx}s_{yy}}} $ and I have seen online to use and ...
Jackanap3s's user avatar
0 votes
0 answers
31 views

Distinguishing various types of regression

I want to understand the pairwise relationship between four types of regression: Bayesian Linear Regression, Gaussian Process Regression, Kernel Regression (Nadaraya-Watson), and Kernel Ridge ...
Tanishq Kumar's user avatar
1 vote
0 answers
40 views

$l_{\infty}$ convergence of OLS prediction error

I have found many resources talking about the $l_2$ convergence of OLS estimation error and prediction error. I have also read about the analysis of the $l_2$ convergence of lasso estimation error and ...
maskeran's user avatar
  • 511
0 votes
0 answers
15 views

Is OLS estimator sub-Gaussian?

Suppose we have $n$ observations $(x_i,y_i)\in\mathbb{R}^{p+1}$. Suppose the underlying model is $y = \eta^{*T} x + \varepsilon$, and $x, \varepsilon$ are sub-Gaussian with parameter $\sigma_x, \...
maskeran's user avatar
  • 511
1 vote
1 answer
65 views

Seber, Ex. 1.b.3 - Calculating variance and showing unbiasedness

Exercise 3 of Sebers Linear Regression Analysis states the following: I tried to solve both problems but only managed to (kinda) solve the first one and have no idea how to tackle the second one. ...
l337n00b's user avatar
  • 475
0 votes
1 answer
108 views

likelihood of data given parameter or likelihood of parameter given data?

What is more correct? Likelihood of data given parameter or likelihood of parameter given data? In https://en.wikipedia.org/wiki/Likelihood_function for instance, we see both likelihood of $\hat{\...
Daniel S.'s user avatar
  • 823
0 votes
1 answer
32 views

Linear $\log$ models change in betas

I'm trying to solve an exercise but I find it difficult to interpret. I have a linear-$\log$ model like this: $y = 1 + 0.55\ln(x) + 3z - 2.2w + \text{error term}$. I wonder what happens to $\beta_1$ ...
Alessandro Tursilli's user avatar
0 votes
1 answer
49 views

Inequality for the trace of the hat matrix in Ridge regression

I was recently reviewing a research paper and came across an inequality expressed as follows: \begin{align} & \text{tr}\Big[\Big(\frac{1}{np}X^\top X + \rho B\Big)^{-1} \Big(\frac{1}{np}X^\top X\...
LL Tony's user avatar
  • 41
1 vote
0 answers
48 views

Find an equation that fits closely to these points in 3D space

I am trying to find an equation $f(x,y) \rightarrow z$ that can be used to return the values in this table, where we'll say the horizontal heading is the $x$ values and the vertical $y$. My thinking ...
user787670's user avatar
0 votes
1 answer
46 views

Independence of pure error and lack-of-fit error in simple linear regression with repeated observations

Let $x_1,\ldots,x_n$ be distinct regressor variables. For each $x_i$, there are $n_i$ observations $Y_{i1},\ldots,Y_{in_i}$ such that $$Y_{ij}=\alpha x_i+\beta+\epsilon_{ij},$$ where $\epsilon_{ij}\...
ashpool's user avatar
  • 6,828
0 votes
1 answer
55 views

Proving Divergence Free Behavior of Matrix Valued Radial Basis Function

I'm am using radial basis functions to interpolate magnetic fields, which are divergence free. I have found several research papers that state that the following takes a scalar valued Radial Basis ...
16toron's user avatar
0 votes
0 answers
72 views

Prove that variance of estimator in GLS is less than variance of estimator in OLS

In my statistics class we showed that for Ordinary Least Squares regression: $$ var[\hat{\beta}_{OLS}] = (X^TX)^{-1}X^T\sigma^2X(X^TX)^{-1} $$ and for Generalized Least Squares regression (ie. ...
Random_Student's user avatar
1 vote
0 answers
23 views

Measuring the total influence of a subset of observations on a particular regression coefficient.

Suppose I have run a multiple regression model: Y = B0 + x1B1 + x2B2 +..+ xnBn, weighted by w, from a dataset with such covariates and the weight variable of size N. Say there is another column in the ...
Lukas Wood's user avatar
1 vote
0 answers
50 views

Lower bound using Largest Singular Value of Pseudo-Inverse

For uni I've got this exercise where i need to prove the following: $$\sigma_{max}(X^\dagger)^2 \ ||Xw-Xw^* ||_2^2 \geq ||w - w^* ||_2^2$$ where $\sigma_{max}(A) = max_{u \in \mathbb{R}^d \ ||u||_2 = ...
Jord van Eldik's user avatar
0 votes
0 answers
23 views

Theoretical analysis for regression imputation

I have been looking at the literature of missing data problems. Many of them talks about multiple imputation, but I'm just interested in what bias occur and how to correct the bias if we employ a ...
maskeran's user avatar
  • 511
0 votes
0 answers
39 views

Validation of Normal Equations with Pseudo-Inverses

In an exercise for uni I am asked to prove that $w^* \in \mathbb{R}^d$ is a solution to $\min_{w \in \mathbb{R}^d}\lVert y - Xw \rVert_{2}^{2}$ if and only if $X^\dagger X w^* = X^\dagger y$, where $X^...
Jord van Eldik's user avatar
0 votes
0 answers
28 views

Multiple Regression Analysis Residuals

Can you please review my analysis based on the following plots. It is a multiple predictors linear regression model The model form assumption is met based on the plots above. we can see that there is ...
Dna A's user avatar
  • 13
0 votes
0 answers
57 views

Conditional expectation on continuos random variable with zero density obtained from non-zero density variables.

Let $Y$, $X_1$ and $X_2$ be three continuous real random variable with $f(x_1, x_2) >0$ everywhere on $R^2$ and denote by $g(x_1, x_2) = E[Y|X_1 = x_1, X_2 = x_2]$. Then $g(0,0) = E[Y|X_1 = 0, X_2 =...
Ldt's user avatar
  • 45
1 vote
0 answers
30 views

Computational complexity of weighted least squares

According to this answer, for $N$ observations and $C$ variables, the computational complexity of solving a linear regression problem $(X^\intercal X)^{-1}X^\intercal \mathbf y$ is $O(C^2N)$. I am not ...
baltimore_orioles's user avatar
0 votes
0 answers
14 views

Regress on matrix on vector to obtain rank-one approximation in Partial Least Square

I'm trying to understand the equation for partial least squares (PLS) regression from the paper "A survey of partial least squares (PLS) methods, with emphasis on the two-blocks case". I'm ...
Yohanes Yordan's user avatar
3 votes
1 answer
98 views

Are these two definitions of the coefficient of determination $R^2$ equal?

I want to do multiple linear regression as explained on this Wikipedia site: I am given data $$ yx=(~(y_1,x_{11},\ldots,x_{1p}),\ldots, (y_n,x_{n1},\ldots,x_{np})~) $$ of $n$-many samples where for ...
mrpotato's user avatar
0 votes
0 answers
23 views

Determine type of function from points

Assume $A$ is a set of points in $\mathbb{R^2}$. The points of $A$ were generated from a function that is either linear, polynomial, or exponential, but you don't know which. Is it possible to ...
Jack Humphries's user avatar
1 vote
1 answer
22 views

Is there a data-driven method to solve omitted variable problem?

Suppose $Z$ is correlated with predictor $X$ and it is unobserved. One classical way of determining the coefficient $\beta_1$ in $Y = \beta_0 + \beta_1 X + \epsilon$ is finding an instrumental ...
maskeran's user avatar
  • 511

1
2 3 4 5
54