Questions tagged [regression-analysis]
This tag is for questions about regression analysis. In statistical modeling, regression analysis is a statistical process for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors').
393
questions
0
votes
0
answers
21
views
Independence of pure error and lack-of-fit error in simple linear regression with repeated observations
Let $x_1,\ldots,x_n$ be distinct regressor variables.
For each $x_i$, there are $n_i$ observations $Y_{i1},\ldots,Y_{in_i}$ such that
$$Y_{ij}=\alpha x_i+\beta+\epsilon_{ij},$$
where $\epsilon_{ij}\...
- 6,748
1
vote
0
answers
20
views
Measuring the total influence of a subset of observations on a particular regression coefficient.
Suppose I have run a multiple regression model:
Y = B0 + x1B1 + x2B2 +..+ xnBn, weighted by w, from a dataset with such covariates and the weight variable of size N. Say there is another column in the ...
- 11
0
votes
0
answers
19
views
Multiple Regression Analysis Residuals
Can you please review my analysis based on the following plots. It is a multiple predictors linear regression model The model form assumption is met based on the plots above. we can see that there is ...
- 11
0
votes
1
answer
37
views
Linear Regression Homoskedasticity doubt
I am trying to plot the Linear Regression between $(X,Y)$ where $X$ is the regressor/predictor and $Y$ is the output. The values are
np.random.seed(42)
...
- 2,627
0
votes
0
answers
47
views
If $x^{\top}\left(X^{\top} X\right)^{-1} x = (x-\bar X)^T(\sum_{i=1}^n(x_i-\bar X)(x_i-\bar X )^T)^{-1}(x-\bar X)$
Notation: $x$ is the new observation feature, $X$ is the design matrix from the training data points $x_i, i= 1,..., n$, and $\bar X$ is $\frac{1}{n}\sum_{i=1}^{n} x_i$.
The reason I think it is ...
- 361
3
votes
0
answers
28
views
Advantage of Exponentiated Gradient over Gradient Descent
Suppose I am trying to fit a function $y(x) = \sum_{n} c_n f_n(x)$, where the set $f_n(x)$ are "expert predictions". At each time step I receive one evaluated value $y(x_t)$, which I use to ...
- 31
1
vote
2
answers
113
views
Optimization with rank constraint
Following some problems I like to explore, I found myself looking at minimizing the following:
$$ \min_{B \text{ with rk}(B)<q} \sum_{i=1}^n (y_i-x_i^TBx_i)^2 $$
where $y_i \in \mathbb{R}, x_i \in \...
- 13
0
votes
0
answers
18
views
Understanding Cook's Distance
Does Cooks Distance tell us how much the estimated parameter values change when the ith observation is removed or how much the fitted values change when the ith observation is removed?
I'm being told ...
- 1
0
votes
0
answers
11
views
How to calculate sample standard deviation of the intercepts that are obtained from all possible combinations (having $n \ge 3$ points)?
Starting with an example:
Having the points $P_1(1,29),P_2(2,50),P_3(4,88)$. Then there are $4$ possible ways to choose points to find the "best" line (using ordinary least-squares). The $4$ ...
- 4,868
0
votes
0
answers
25
views
Regression calculation of initial individual weight considering variations in weighing dates and missing data
I need to determine the initial individual weight of each animal as accurately as possible, taking into account the following challenges:
The animals may be weighed on different days.
There may be ...
0
votes
1
answer
90
views
Linear Algebra/Matrices Reference Request
I am learning multivariate regression, with matrix equations and heavy use of:
Transposes
Inverses
Identity, idempotent matrices
Can anyone refer some books where I can learn the properties of ...
- 1,456
0
votes
0
answers
18
views
General question about lasso variable selection consistency
In the context of high dimension problem, I observe a pattern that some paper start with variable selection consistency, i.e. the probability that the estimated important variable set equals the true ...
- 361
0
votes
0
answers
26
views
When does averaging affect a linear regression?
Given the following:
data $(x_{data},y_{data})$, where x values may be measured multiple times.
For example,
$$ x_{data} = ( 1, 2, 2, 3,3,3, 4 ) $$
$$ y_{data}=(8,10,9,1,2,1.5,−7) $$
And a fitting ...
- 419
0
votes
1
answer
49
views
Finding the form of $z$ in terms of $x$ and $y$, letting $z(x,0)$ to be $0$ forcefully.
In a practical experiment, we have two independent variables, $x>0$ and $y \ge 0$, and one dependent variable, $z \ge 0$.
Theoretically, we know that at any fixed $x$, we have the relation between $...
- 4,868
0
votes
0
answers
19
views
One-sample test clustered data
I have $n$ measures and I would like to get a p-value to know if the mean is significantly different from 0. However, each measure belongs to a cluster and hence all the $n$ measures are not perfectly ...
- 13
1
vote
0
answers
36
views
How do I translate the coefficients of an OLS confidence interval into a range of actual values?
I'm using statsmodels to perform an OLS simple regression on the Palmer's penguins dataset. The regression uses birds' bill length as the sole independent variable and their body mass as the target. ...
- 171
0
votes
0
answers
24
views
In search of vector autoregression models supporting uniform bounds on coordinate-wise derivatives
This question is motivated by the desire to build mathematical models that forecast vector-valued discrete time series while guaranteeing a kind of "continuity" via uniform bounds on the ...
- 51
1
vote
1
answer
34
views
Probability w.r.t. sample
In FAST LEARNING RATES FOR PLUG-IN CLASSIFIERS by Audibert, Tsybakov, they use the following notation in the context of binary classification:
Let $(X, Y)$ be a random couple taking values in $Z = R^d\...
- 627
0
votes
0
answers
13
views
Poisson log-linear model for estimating large size count data
Assume that the random variable $E_i = E|x_i$ satisfies Poisson processes.
Then we could consider Poisson Log Linear Regression to estimate $\mathbb{E}[E_i] = \lambda_i$, the conditional expectation ...
- 1
0
votes
0
answers
43
views
Finding a suitable form factor for given conditions
This is basically a physics problem but I will try my best to highlight the mathematics behind it.
Suppose I have two functions:
$$T(z,B)=\frac{\text{z}^3 e^{-3 A(\text{z})-B^2 \text{z}^2}}{4 \pi \...
- 121
1
vote
2
answers
545
views
Covariance of Residuals and Fitted Values in Linear Regression
Consider the simple linear regression model
$Y_i = \beta_0 + \beta_1x_i + \epsilon_i$
where $\epsilon_i \sim^{indep} N(0, \sigma^2)$ for $i = 1,...,n$. Let $\hat{\beta_{0}}$ and $\hat{\beta_{1}}$ be ...
- 426
0
votes
0
answers
89
views
QR decomposition for Linear independence
I have the following linear model:
$\mathbf{\tau} = \mathbf{\Phi} \mathbf{\pi}$
where $\mathbf{\tau}$ is a known vector, $\mathbf{\Phi}$ a known regressor and $\mathbf{\pi}$ is a vector of some ...
- 11
0
votes
0
answers
43
views
How to compute expectancy and variance of Recursive-Least-Squares 1-step-ahead residuals
I'm trying to learn about the following technique: CUSUM of RLS 1-step-ahead residuals as described in (Brown, Durbin, Evans 1975) in order to determine whether a significant change occurred in the ...
0
votes
0
answers
15
views
Estimate $\hat{E}[Y|X=x]$ for lognormal distribution
We assume that $Y$ given $X=x$ has lognormal distribution with parameters $\beta_0+\beta_1ln(x)$ and $\sigma^2$, i.e.
$$(Y|X=x)\sim LN(\beta_0+\beta_1\ln(x),\sigma^2).$$
We want to find estimate of ...
- 41
0
votes
1
answer
30
views
create a function from data (that probably doesn't fit) using many many many calibrating parameters
I have the following:
$\lambda_1 = \frac{const_{A}}{value1_1} + \frac{const_{B}}{value2_1} + \frac{const_{c}}{value3_1} $
$\lambda_2 = \frac{const_{A}}{value1_2} + \frac{const_{B}}{value2_2} + \frac{...
1
vote
0
answers
21
views
Trying to figure out multicollinearity.
I am learning multiple regression at the momemnt. And maybe I'm just sleep deprived, but I am supposed to assess whether increasing student teacher ratios (total enrollment/teachers, str) will improve ...
- 31
0
votes
1
answer
59
views
How do I calculate the regression equation of $y = ax^2$ (find the coefficient $a$ that gives the smallest sum of squares of errors)?
The input is a group of points $(x,y)$. I am trying to find the regression equation of $y = ax^2$ (I am trying to find the coefficient $a$ that gives the smallest sum of squares of errors from the ...
0
votes
0
answers
23
views
Mixed linear models variance estimation in R - random effect varience zero
I have a general model that looks like this :
$Y_i=\vec 1j_i*mu +\vec 1_jib_i+\varepsilon_i$
and I have data that looks like this:
...
0
votes
0
answers
21
views
Looking for a fitting function
I want to fit some data points that follow this trend:
I've tried an exponential of this kind: $f(x)= a\exp(-bx)$ but it's not good enough. Could you give me any hint?
Many thanks.
- 387
0
votes
0
answers
20
views
Fixing heteroskedasticity in multiple linear regression
I am currently working on model with several variables, yet only one of them causes heteroskedasticity in the model. All the methods for solving such case that I am aware of only work for single-...
1
vote
2
answers
135
views
Book recommendation on linear and nonlinear regression
I am doing a very complex and in-depth course on regression (studying math), but the professor is flying through it and the book often does the same, or is very hard to understand. I wanted to know if ...
- 359
0
votes
1
answer
37
views
Cosine model design matrix non-lineal model
I need to applied the T- student testing to the parameters $\beta_0, \beta_1, \beta_2,\beta_3,\beta_4$ which model is:
$$ y = \beta_0 + \beta_1 t + \beta_2 Cos(\beta_3 t+\beta_4)$$
to do that, I need ...
- 28
0
votes
1
answer
193
views
Multiple regression by successive orthogonalization
I was studying The Elements of Statistical Learning book and trying to understand the section where multiple linear regression is explained by successive orthogonalization procedure, i.e. Gram-Schmidt ...
0
votes
1
answer
59
views
Can power regression help finding optimal fitting polynomial?
I would like to non-heuristically (dis)prove the following statement:
"The degree of the optimal polynomial to fit to some data corresponds with the closest integer to the resulting exponent from ...
- 155
2
votes
2
answers
253
views
Linearization before curve fitting
I am currently studying at college curve fitting to a set of experimental data. One of our activities/homework was to fit the curve $y=ax^b$ to the set of points of the position of a free-falling ...
- 1,009
0
votes
0
answers
190
views
What's the difference between a general linear model and a generalized linear model?
I need to use a model for my Master's thesis. Looking beyond multiple linear regression, I have found extensions like the general linear model, and the generalized linear model. What is the difference ...
4
votes
2
answers
80
views
Line of best fit for $\{(n,n+\sin n) : n \in \mathbb{Z}\}$
It seems intuitive that the line of best fit for $\{(n,n+\sin n) : n\in \mathbb{Z}\}$ should be $y=x$.
More concretely, it seems like a reasonable conjecture would be:
If $y = m_k x + b_k$ is the ...
- 4,183
1
vote
0
answers
90
views
Is this linear regression?
I'm not sure how to distinguish linear from nonlinear regression. Linear regression should be linear in the parameters. I have the doubt if the following equation can be considered as a linear ...
- 75
0
votes
1
answer
44
views
Least Squares two different forms for the residual
Least squares residuals are given in the basic form:
$$r_i = y_i - f(x_i, \theta)$$
where $y_i$ is the observation and $f(x_i,\theta)$ is the value predicted by the model. I came across another form ...
- 4,169
2
votes
1
answer
223
views
Variance-stabilizing transformation on a simple linear regression
I am currently working with variance-stabilizer method and readed something about it from my textbook. I want to understand it better so I would like to consider a case where I for instance have a ...
- 500
0
votes
1
answer
104
views
Simple linear regression (sum of residuals and predictor)
Show explicitly that the following identity holds under a Simple Linear Regression:
$$
\ \sum_{i=1}^n r_i \hat{\mu_i} =0$$
with residuals $ r_i = y_i − \hat{\mu_i} $ and $\hat{\mu_i} = \hat{\beta_0}+\...
- 77
2
votes
1
answer
248
views
How to explain covariance in logistic regression + analogy to linear regression
Introduction
Linear model
In linear regression we predict continuous variable $Y \in R^n$ with use of $n \times p$ deterministic plan matrix $X$ and theoretical model (let's ignore intercept ...
- 241
0
votes
1
answer
33
views
Econometrics/Statistics Regression Question [closed]
As you can see from the provided picture given Heart attack given rate per 100,000 population. I was able to successfully ran my regression; but now I am trying to figure out how to alter my ...
- 49
1
vote
1
answer
21
views
Use regression to find common noise component
Suppose I have three mutually independent non-Gaussian noise $E_A$, $E_B$, $E_C$. There are two variables generated by linear combinations of these noise components: $M=pE_A+qE_B$, $N=rE_B$. By linear ...
- 193
0
votes
1
answer
42
views
What is the best way to estimate the parameters of a logistic regresion model?
I recently read about logistic regression model.
$$y=\frac{1}{1+e^{-(\beta_0+\beta_1x)}}$$
where y is a categorical variable with either 0 or 1 output.
What seems to be perplexing to me is, I can see ...
- 153
0
votes
0
answers
22
views
What type of statistical test should I use for this specific example?
I am doing a research project analyzing COVID-19 Cases and its effect on unemployment rates within a country. So, for example, I have the percentage of the population that have COVID-19 in a country ...
- 1
1
vote
0
answers
30
views
What function would model Mercury's orbital velocity around the Sun?
I am working on a mathematical investigation for my school work and in my investigation, I am trying to model Mercury's velocity around the sun. I picked up data for the velocity from the NASA ...
- 169
0
votes
1
answer
94
views
Using GPA and Class Rank/Percentile Data to create a regression based on the assumption of a normal distribution.
I was interested in seeing if I can use just individual data points, knowing what the percentile of those GPA values is to be able create a normal distribution to predict all other GPA values.
For ...
1
vote
1
answer
78
views
Proving the Mulitple Coefficient of Determination Formula (correlated explanatory variables)
I stumbled upon the following formula for the coefficient of determination:
$$1-R_{y(x_1,x_2...x_n)}^2=\left(1-\rho_{y,x_1}^2\right)\left(1-\rho_{y,x_2(x_1)}^2\right)\left(1-\rho_{y,x_3(x_1,x_2)}^2\...
- 31
1
vote
1
answer
635
views
Bias of ridge estimator
The ridge estimator $(\hat{\beta}_R)$, and the expected value, are defined as;
\begin{align}
\hat{\beta}_R &= \left( X'X + kI \right)^{-1}X'y, \ k \geq 0 \\
\text{E}\left( \hat{\beta}_R \...
- 167