Skip to main content

Questions tagged [regression-analysis]

This tag is for questions about regression analysis. In statistical modeling, regression analysis is a statistical process for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors').

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

Technique for finding expected value Weighted Ridge Regression Coefficients

Context: We would like to approximate a linear function $f(\mathbf{x})$ at the point $\boldsymbol{\xi} \in \mathbb{R}^D$ using samples of size $N$ around $\boldsymbol{\xi}$. Assume that the input data ...
user9781778's user avatar
0 votes
0 answers
11 views

Is there a way to modelize a partial predictor in a classification problem with an unbalanced target?

I would like to share with you a classification issue I faced during the modelling process. I have to create a model for an unbalanced binary target by 4 predictors where one of them has 45% of wrong ...
rambo17's user avatar
0 votes
0 answers
42 views

Appropriateness of one observation per unique combination of dummy variables

I am wondering what conclusions you can draw regarding the coefficients of an OLS model when you only have one observation per combination of unique dummy variables. I have seen someone else do this ...
user25435163's user avatar
1 vote
0 answers
27 views

How to use Multiple Regression Analysis to get to linear equation

I am working on a paper about MQ gas sensors and found this other study (https://www.researchgate.net/publication/...
kml's user avatar
  • 11
1 vote
0 answers
58 views

Is the expectation of the error of any projection of $\mathbb{E}[Y\mid X]$ onto subspace zero?

If we consider the following linear predictor of $Y$ based on $X$: $$ Y_{\mathbf{b}}=\boldsymbol{\Sigma}_{Y, \mathbf{X}} \boldsymbol{\Sigma}_{\mathbf{X}}^{-1}\left(\mathbf{X}-\boldsymbol{\mu}_{\mathbf{...
maskeran's user avatar
  • 573
0 votes
0 answers
54 views

Is conditional expectation of the error of best linear predictor given $X$ is $0$ (Is it true that $y = a^*+b^*x + \eta$, where $E[\eta|x]=0$)?

For simplicity, assume we are working with simple regression where the predictor $x\in\mathbb{R}$. First write $y=E[y \mid x]+u$, where the variance of $u$ is a constant, and $E[u|x]=0$. I understand $...
maskeran's user avatar
  • 573
0 votes
0 answers
41 views

Why does the poisson regression beta coefficient ML estimate have no closed form solution?

Every source I have read says that there is no closed form solution for the beta coefficient but I have not seen an explanation as to why. I tried to solve for the beta coefficient on my own to see ...
decapicone's user avatar
0 votes
0 answers
15 views

Covariance between estimated random effects $\hat{\boldsymbol{b}}$ and real idiosyncratic error vector $\boldsymbol{\epsilon}$ in a Linear Mixed Model

Let us assume a linear mixed model of the form $$ \boldsymbol{y} = \boldsymbol{X}\boldsymbol{\beta} + \boldsymbol{Z}\boldsymbol{b} + \boldsymbol{\epsilon} $$ where $\boldsymbol{X}$ is the fixed-effect ...
Benykō-Zamurai's user avatar
0 votes
0 answers
32 views

Under what conditions does a Normal random variable have approximately the same conditional distribution at every value of a discrete variable?

Let $X$ be a random variable with known distribution $N(\mu, \sigma)$. Let $Y$ be a count variable, i.e., it is a discrete random variable with known finite expectation and finite variance, which ...
virtuolie's user avatar
  • 165
1 vote
1 answer
33 views

What considerations should I have into account when linearizing a non-linear model for linear regressions?

I'm looking for some bibliography about what I should/must have into account when I have a model and experimental data that can be expressed in a way such that I can use a linear regression method to ...
CosmeticMichu's user avatar
4 votes
1 answer
110 views

Sqrt LASSO vs LASSO

In the paper Square Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming they talk about Sqrt-LASSO which is simply just trying to minimize $\|Ax-b\|_2 + \lambda\|x\|_1$ rather than ...
jeffj1355's user avatar
0 votes
0 answers
42 views

Regression with a Nonlinear ODE (exponentiated derivative)

I have data (x, y) which I believe is generated by a differential equation of the following form: $$A\frac{d^2y}{dx^2}+B(\frac{dy}{dx})^{C}+D=0$$ I can estimate the initial values, so given $A$, $B$, $...
Bananacus's user avatar
0 votes
0 answers
20 views

How can we reduce the effect of measurement noise in a regression problem?

We organize the input and output samples of a linear time invariant (LTI) system into two matrices, $Y$ and $G$, following a specific pattern. It is understood that a linear relationship exists ...
apa's user avatar
  • 3
2 votes
0 answers
100 views

How can I properly show that the Residual Sum of Squares can be written in matrix form?

First, I apologize if I am violating any community laws or have improper tags I am a newbie. However, I am tasked with showing, $$RSS(\boldsymbol{\beta}) = \sum_{i=1}^{N} (y_i - \beta_0 - \sum_{j =1}^{...
BeefBroccoli's user avatar
0 votes
0 answers
28 views

Degrees of Freedom in PCA

Suppose we are doing PCA over a historic time series of temperatures. The feature for the PCA to be explained is the time when the temperature was observed every hour. Let’s say for the sake of ...
Identicon's user avatar
0 votes
1 answer
69 views

Independence of pure error and lack-of-fit error in simple linear regression with repeated observations

Let $x_1,\ldots,x_n$ be distinct regressor variables. For each $x_i$, there are $n_i$ observations $Y_{i1},\ldots,Y_{in_i}$ such that $$Y_{ij}=\alpha x_i+\beta+\epsilon_{ij},$$ where $\epsilon_{ij}\...
ashpool's user avatar
  • 7,006
1 vote
0 answers
23 views

Measuring the total influence of a subset of observations on a particular regression coefficient.

Suppose I have run a multiple regression model: Y = B0 + x1B1 + x2B2 +..+ xnBn, weighted by w, from a dataset with such covariates and the weight variable of size N. Say there is another column in the ...
Lukas Wood's user avatar
0 votes
0 answers
36 views

Multiple Regression Analysis Residuals

Can you please review my analysis based on the following plots. It is a multiple predictors linear regression model The model form assumption is met based on the plots above. we can see that there is ...
Dna A's user avatar
  • 13
0 votes
1 answer
45 views

Linear Regression Homoskedasticity doubt

I am trying to plot the Linear Regression between $(X,Y)$ where $X$ is the regressor/predictor and $Y$ is the output. The values are np.random.seed(42) ...
Upstart's user avatar
  • 2,632
0 votes
0 answers
48 views

If $x^{\top}\left(X^{\top} X\right)^{-1} x = (x-\bar X)^T(\sum_{i=1}^n(x_i-\bar X)(x_i-\bar X )^T)^{-1}(x-\bar X)$

Notation: $x$ is the new observation feature, $X$ is the design matrix from the training data points $x_i, i= 1,..., n$, and $\bar X$ is $\frac{1}{n}\sum_{i=1}^{n} x_i$. The reason I think it is ...
maskeran's user avatar
  • 573
3 votes
0 answers
75 views

Advantage of Exponentiated Gradient over Gradient Descent

Suppose I am trying to fit a function $y(x) = \sum_{n} c_n f_n(x)$, where the set $f_n(x)$ are "expert predictions". At each time step I receive one evaluated value $y(x_t)$, which I use to ...
Thomas C's user avatar
1 vote
2 answers
145 views

Optimization with rank constraint

Following some problems I like to explore, I found myself looking at minimizing the following: $$ \min_{B \text{ with rk}(B)<q} \sum_{i=1}^n (y_i-x_i^TBx_i)^2 $$ where $y_i \in \mathbb{R}, x_i \in \...
Pierre21's user avatar
0 votes
0 answers
28 views

Understanding Cook's Distance

Does Cooks Distance tell us how much the estimated parameter values change when the ith observation is removed or how much the fitted values change when the ith observation is removed? I'm being told ...
WDW's user avatar
  • 1
0 votes
0 answers
20 views

How to calculate sample standard deviation of the intercepts that are obtained from all possible combinations (having $n \ge 3$ points)?

Starting with an example: Having the points $P_1(1,29),P_2(2,50),P_3(4,88)$. Then there are $4$ possible ways to choose points to find the "best" line (using ordinary least-squares). The $4$ ...
Hussain-Alqatari's user avatar
0 votes
0 answers
26 views

Regression calculation of initial individual weight considering variations in weighing dates and missing data

I need to determine the initial individual weight of each animal as accurately as possible, taking into account the following challenges: The animals may be weighed on different days. There may be ...
pkdpqowjodiv's user avatar
0 votes
1 answer
104 views

Linear Algebra/Matrices Reference Request

I am learning multivariate regression, with matrix equations and heavy use of: Transposes Inverses Identity, idempotent matrices Can anyone refer some books where I can learn the properties of ...
Starlight's user avatar
  • 1,834
0 votes
0 answers
44 views

When does averaging affect a linear regression?

Given the following: data $(x_{data},y_{data})$, where x values may be measured multiple times. For example, $$ x_{data} = ( 1, 2, 2, 3,3,3, 4 ) $$ $$ y_{data}=(8,10,9,1,2,1.5,−7) $$ And a fitting ...
ions me's user avatar
  • 419
0 votes
1 answer
54 views

Finding the form of $z$ in terms of $x$ and $y$, letting $z(x,0)$ to be $0$ forcefully.

In a practical experiment, we have two independent variables, $x>0$ and $y \ge 0$, and one dependent variable, $z \ge 0$. Theoretically, we know that at any fixed $x$, we have the relation between $...
Hussain-Alqatari's user avatar
1 vote
0 answers
41 views

How do I translate the coefficients of an OLS confidence interval into a range of actual values?

I'm using statsmodels to perform an OLS simple regression on the Palmer's penguins dataset. The regression uses birds' bill length as the sole independent variable and their body mass as the target. ...
NaiveBae's user avatar
  • 171
1 vote
1 answer
40 views

Probability w.r.t. sample

In FAST LEARNING RATES FOR PLUG-IN CLASSIFIERS by Audibert, Tsybakov, they use the following notation in the context of binary classification: Let $(X, Y)$ be a random couple taking values in $Z = R^d\...
newbie's user avatar
  • 647
1 vote
2 answers
1k views

Covariance of Residuals and Fitted Values in Linear Regression

Consider the simple linear regression model $Y_i = \beta_0 + \beta_1x_i + \epsilon_i$ where $\epsilon_i \sim^{indep} N(0, \sigma^2)$ for $i = 1,...,n$. Let $\hat{\beta_{0}}$ and $\hat{\beta_{1}}$ be ...
spooleey's user avatar
  • 456
0 votes
1 answer
31 views

create a function from data (that probably doesn't fit) using many many many calibrating parameters

I have the following: $\lambda_1 = \frac{const_{A}}{value1_1} + \frac{const_{B}}{value2_1} + \frac{const_{c}}{value3_1} $ $\lambda_2 = \frac{const_{A}}{value1_2} + \frac{const_{B}}{value2_2} + \frac{...
The Coding Karp's user avatar
1 vote
0 answers
23 views

Trying to figure out multicollinearity.

I am learning multiple regression at the momemnt. And maybe I'm just sleep deprived, but I am supposed to assess whether increasing student teacher ratios (total enrollment/teachers, str) will improve ...
SamL11B's user avatar
  • 31
0 votes
1 answer
110 views

How do I calculate the regression equation of $y = ax^2$ (find the coefficient $a$ that gives the smallest sum of squares of errors)?

The input is a group of points $(x,y)$. I am trying to find the regression equation of $y = ax^2$ (I am trying to find the coefficient $a$ that gives the smallest sum of squares of errors from the ...
Sung Woo Han's user avatar
0 votes
0 answers
21 views

Looking for a fitting function

I want to fit some data points that follow this trend: I've tried an exponential of this kind: $f(x)= a\exp(-bx)$ but it's not good enough. Could you give me any hint? Many thanks.
user9867's user avatar
  • 387
1 vote
2 answers
319 views

Book recommendation on linear and nonlinear regression

I am doing a very complex and in-depth course on regression (studying math), but the professor is flying through it and the book often does the same, or is very hard to understand. I wanted to know if ...
ThighCrush's user avatar
0 votes
1 answer
43 views

Cosine model design matrix non-lineal model

I need to applied the T- student testing to the parameters $\beta_0, \beta_1, \beta_2,\beta_3,\beta_4$ which model is: $$ y = \beta_0 + \beta_1 t + \beta_2 Cos(\beta_3 t+\beta_4)$$ to do that, I need ...
mielvil's user avatar
  • 28
0 votes
1 answer
384 views

Multiple regression by successive orthogonalization

I was studying The Elements of Statistical Learning book and trying to understand the section where multiple linear regression is explained by successive orthogonalization procedure, i.e. Gram-Schmidt ...
ConventionalProgrammer's user avatar
0 votes
1 answer
63 views

Can power regression help finding optimal fitting polynomial?

I would like to non-heuristically (dis)prove the following statement: "The degree of the optimal polynomial to fit to some data corresponds with the closest integer to the resulting exponent from ...
GeoArt's user avatar
  • 165
2 votes
2 answers
400 views

Linearization before curve fitting

I am currently studying at college curve fitting to a set of experimental data. One of our activities/homework was to fit the curve $y=ax^b$ to the set of points of the position of a free-falling ...
ordptt's user avatar
  • 1,069
0 votes
0 answers
406 views

What's the difference between a general linear model and a generalized linear model?

I need to use a model for my Master's thesis. Looking beyond multiple linear regression, I have found extensions like the general linear model, and the generalized linear model. What is the difference ...
Alessandro Bertulli's user avatar
4 votes
2 answers
86 views

Line of best fit for $\{(n,n+\sin n) : n \in \mathbb{Z}\}$

It seems intuitive that the line of best fit for $\{(n,n+\sin n) : n\in \mathbb{Z}\}$ should be $y=x$. More concretely, it seems like a reasonable conjecture would be: If $y = m_k x + b_k$ is the ...
Patch's user avatar
  • 4,305
1 vote
0 answers
93 views

Is this linear regression?

I'm not sure how to distinguish linear from nonlinear regression. Linear regression should be linear in the parameters. I have the doubt if the following equation can be considered as a linear ...
dreamco9's user avatar
0 votes
1 answer
45 views

Least Squares two different forms for the residual

Least squares residuals are given in the basic form: $$r_i = y_i - f(x_i, \theta)$$ where $y_i$ is the observation and $f(x_i,\theta)$ is the value predicted by the model. I came across another form ...
oliverjones's user avatar
  • 4,209
2 votes
1 answer
292 views

Variance-stabilizing transformation on a simple linear regression

I am currently working with variance-stabilizer method and readed something about it from my textbook. I want to understand it better so I would like to consider a case where I for instance have a ...
NabbKitha's user avatar
  • 540
0 votes
1 answer
150 views

Simple linear regression (sum of residuals and predictor)

Show explicitly that the following identity holds under a Simple Linear Regression: $$ \ \sum_{i=1}^n r_i \hat{\mu_i} =0$$ with residuals $ r_i = y_i − \hat{\mu_i} $ and $\hat{\mu_i} = \hat{\beta_0}+\...
Anita's user avatar
  • 77
2 votes
1 answer
323 views

How to explain covariance in logistic regression + analogy to linear regression

Introduction Linear model In linear regression we predict continuous variable $Y \in R^n$ with use of $n \times p$ deterministic plan matrix $X$ and theoretical model (let's ignore intercept ...
Brzoskwinia's user avatar
0 votes
1 answer
37 views

Econometrics/Statistics Regression Question [closed]

As you can see from the provided picture given Heart attack given rate per 100,000 population. I was able to successfully ran my regression; but now I am trying to figure out how to alter my ...
mva's user avatar
  • 109
1 vote
1 answer
21 views

Use regression to find common noise component

Suppose I have three mutually independent non-Gaussian noise $E_A$, $E_B$, $E_C$. There are two variables generated by linear combinations of these noise components: $M=pE_A+qE_B$, $N=rE_B$. By linear ...
graphitump's user avatar
0 votes
1 answer
50 views

What is the best way to estimate the parameters of a logistic regresion model?

I recently read about logistic regression model. $$y=\frac{1}{1+e^{-(\beta_0+\beta_1x)}}$$ where y is a categorical variable with either 0 or 1 output. What seems to be perplexing to me is, I can see ...
Aleph's user avatar
  • 164

1
2 3 4 5
8