Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

Filter by
Sorted by
Tagged with
-1
votes
0answers
8 views

Proving a sufficient condition for restricted strong convexity

I am stuck at some sub question of this problem and I would be glad to get some help. Suppose X is a design matrix with known sparsity level, and it is normalized; [1]: https://i.stack.imgur.com/lL4Pr....
0
votes
0answers
18 views

Piecewise quadratic function

I'm reading this paper on the sparse-group lasso, and it states at a specific point that the function is a piecewise quadratic: (section 3.3, page 8) $$\left\|S(X^{(l)}y/n, \lambda \alpha) \right\|_2^...
0
votes
0answers
18 views

Formula for negative binomial regression

I would like to list the mathematical formula of a negative binomial regression for my paper. Unfortunately I have no real knowledge of statistics or higher mathematics. I would like to use nonlinear ...
0
votes
0answers
27 views

Determining correlations of derivatives of a function given only measurements of that function

Cross-posted from statistics stackexchange: Say we have a permanent-magnet DC motor that roughly obeys the system equation $\ddot{x}(t) = \alpha \dot{x}(t) + \beta u(t) + \gamma$, where $x(t)$ is the ...
0
votes
0answers
14 views

Input with two numeric values in multiple linear regression

I have to perform a multiple linear regression with a total of 7 (say: A, B, C…., G) numeric inputs/dependent variables and one numeric output/independent variable (say: Y). One of seven inputs (say C)...
0
votes
0answers
13 views

Posterior calculation in Bayesian regression

Assuming a linear model $y = x\beta + \epsilon$ where $\epsilon$ is normally distributed as $\mathcal{N}( {\epsilon}|{0}, {\sigma^{2}})$ and $\beta$ with a normal prior distribution $\mathcal{N}( {\...
-1
votes
0answers
25 views

Proving unbiasedness of the population variance [closed]

How can I prove that the population variance is unbiased (E(s^2 Y)= Var Y when the sample Y1...Yn is i.id.
1
vote
1answer
39 views

Regression of y on x and x on y for SSE=0

Question: Suppose ($x_i$, $y_i$), i=$1,...,n$ is a set of pairs of observations. Consider the simple linear regressions of y on and x on y. Show that SSE=$0$ for both models if and only if both ...
-1
votes
0answers
34 views

Regression curve that can only go down, never up

I have a regression curve that I use to model road condition over time. $$y=21-e^{a x}$$ In this scenario, the way a road's condition works is: the condition can only ever go down (deteriorate); a ...
2
votes
0answers
51 views

Tune exponential regression coefficient via Generalized Reduced Gradient (GRG)

I have an exponential regression equation that I use to predict the future condition of roads: $$y=21-e^{a x}$$ I've come up with an initial estimate for $a$ using the normal equation: $$a=\frac{\sum_{...
0
votes
1answer
87 views
+50

Options for optimizing exponential regression

I have an exponential regression equation that I use to predict the future condition of roads: $$y=21-e^{a x}$$ I've come up with an initial estimate for $a$ using the normal equation. $$a=\frac{\sum_{...
1
vote
1answer
59 views

Tune an exponential regression estimate using calculus

I have an exponential regression equation that I use to predict the future condition of roads: $$y=21-e^{a x}$$ Using the normal equation $$a=\frac{\sum_{i=1}^n x_iz_i } { \sum_{i=1}^n x_i^2 }=\frac{\...
2
votes
1answer
29 views

What data do i use to calculate the variance around a predicted value given from a simple regression equation?

Trying to figure out what piece of data goes where in the attached formula for calculating the variance around a predicted value given from a simple regression equation $y_d = a+b*x_d$. I have ...
3
votes
0answers
92 views

Exponential regression GLM

Consider some positive random variables $X^1, X^2$ and $Y\sim Exp(p)$ where $p=\beta_0+\beta_1X^1 + \beta_2X^2$. We have a random sample $\{X^1_i, X^2_i, Y_i\}$. Now, estimate $\beta_1, \beta_2$ is ...
0
votes
0answers
16 views

Nonlinear Regression, least squares

I am trying to solve a non-linear least squares problem like this. $$g(\sum _{1\le j\le J}c_jx_j^i) - f_{mod(i, q)} = y_i\text{ }(1\le i\le I)$$ We want to find $c_j$'s and $f_i$'s where $x^i_j $, $...
1
vote
1answer
32 views

Equivalence between OLS estimators in matrix and summation form

I am struggling to reconcile the OLS estimators that I commonly see expressed in matrix and summation form. In matrix form, it takes the following form: $\hat β$ = $(X'X)^{-1}X'y$ In summation form, ...
0
votes
1answer
42 views

Nonlinear Regression

Elevation (feet) Surface (sq-ft) 325 100 326 1350 327 10,100 328 31,250 329 80,150 Graph and Equation I have been working with this data set attempting to use a nonlinear regression to generate ...
2
votes
1answer
147 views

What does the $A$ in $Y=A+Be^{CX}$ represent?

I'm a novice trying to learn about statistics in the infrastructure asset management industry. I have an existing exponential regression equation that is used to find the condition of a given asset: $...
0
votes
1answer
26 views

How to compute confidence intervals and standard error for nonlinear regression with three parameters?

I have been working on a personal project trying to emulate the nonlinear regression functionality of Mathematica for three free parameters. I am able to accurately fit functions, yet I am unsure how ...
0
votes
0answers
37 views

Maximum likelihood for regression

I've started studying maximum likelihood estimation for regression. What I am trying to do is to understand the process step by step. That is how I've structured the material I've read: The ...
0
votes
0answers
16 views

Interpreting Coefficients in Log-Log Model with Multiple Regressors

If we have a multiple linear regression model $Y=\beta_1+\beta_2 X_2+\beta_3 X_3 +u$ Then we interpret $\beta_2$ as the resultant increase in $Y$ when $X_2$ rises by one unit, holding $X_3$ constant. ...
1
vote
0answers
42 views

Comparing two nonlinear regression models with related parameters

We have a nonlinear regression model with $m$ parameters ($\alpha_1,\alpha_2, ..., \alpha_m$) and $n$ regressors $(X_1, .... X_n)$, predicting an outcome Y: $Y = f(\alpha_1,\alpha_2, ..., \alpha_m; ...
0
votes
0answers
30 views

Expectation of idempotent matrix multiplied by error vector

I have a $T \times T$ idempotent and symmetric matrix $M_X$, defined as $M_X$ = $I_T - X(X'X)^{-1}X'$, where $X$ is a $T \times K$ matrix of $T$ observations of $K$ regressors. For the following ...
0
votes
0answers
16 views

Which method would you use to ascertain the most important independent variables?

I am trying to ascertain which independent variables matter the most as they pertain to the dependent variable. The two methods I have used are giving slightly different answers. I have tried two: ...
0
votes
1answer
21 views

Using GPA and Class Rank/Percentile Data to create a regression based on the assumption of a normal distribution.

I was interested in seeing if I can use just individual data points, knowing what the percentile of those GPA values is to be able create a normal distribution to predict all other GPA values. For ...
0
votes
0answers
23 views

difference in difference model with a very interesting (two-sided) platform dataset (comments needed!)

I have a dataset from a platform where consumers receive services from providers. Each service provider serves a specific set of consumers, and each consumer only receives services from a particular ...
1
vote
0answers
43 views

linear regression simpson's paradox

I need advice on this problem. It is related to Simpson's Paradox. Consider three binary variables $X, Y, Z$ and all taking values in $\{0, 1\}.$ Consider the following inequalities: $P(X = 1) > P(...
0
votes
0answers
11 views

"log of the variance at the second level" in the Stata statistical software package?

I was conducting a logit fixed-effects regression, and Stata (software package) reported to me the "lnsig2u", which apparently is the log of the variance at the second level. I could not ...
0
votes
0answers
14 views

how to find F-test statistic to this null hypothesis given the two models?

k=20,n=100 SSreg=207.5 , SStot=900.9 B1=B2-0 SSreg=100.9 , SStot=900.9 Here is the formula I thought I need to use: (Rss1-Rss2)/(k2-k1)/(Rss2/(n-K2)) however in this example in specific I don't have ...
0
votes
0answers
11 views

Binary to Multiclass logistic regression and vice versa

As I was working on a problem, I came across the mention of logistic regression being used for binary and multiclass problems. Specifically, I am very keen on the problem with the below equations. How ...
0
votes
1answer
32 views

Bias of ridge estimator

The ridge estimator $(\hat{\beta}_R)$, and the expected value, are defined as; \begin{align} \hat{\beta}_R &= \left( X'X + kI \right)^{-1}X'y, \ k \geq 0 \\ \text{E}\left( \hat{\beta}_R \...
0
votes
1answer
24 views

Conditional Expectation of Response Variable given Predictor Variable in Statistical Modeling

I'm a bit confused about the implication of the following: Suppose we are given a set of data points $(X_i, Y_i), i=1,2,...,n$, where $X_i$ is the predictor variable, and $Y_i$ is the response ...
6
votes
2answers
261 views

Beta regression

Consider some positive random variables $X^1, X^2$ and $Y\sim Beta(p; 1)$ where $p=\beta_1X^1 + \beta_2X^2$. We have a random sample $\{X^1_i, X^2_i, Y_i\}$. Now, estimating $\beta_1, \beta_2$ is not ...
0
votes
0answers
21 views

Overfitting the pinball loss in quantile regression

I have a question about the pinball loss, $$\rho_\tau(y, \hat{y}) = (y - \hat{y}) (\tau - \mathbb{I}(y-\hat{y}<0)),$$ which is often applied in quantile regression and typically looks like Given ...
0
votes
0answers
20 views

errors associated with each observations based on their distance to a linear regression plane

This is in reference to outlier analysis by Charu C Aggarwal. Let $D$ be a dataset of dimension $N \times d$ where N is the number of observations and d is the dimensions (or variables). Here, $D$ is ...
1
vote
0answers
30 views

Conjugate Bayesian analysis of linear regression with correlated residuals

I am interested in a Bayesian treatment of (univariate) linear regression in the presence of correlated residuals, but I am somewhat stuck trying to come up with a neat parametrization for a conjugate ...
0
votes
0answers
27 views

Transform linear regression model with non-constant variance to constant variance

I have the following linear regression model: $$Y_i = \alpha + \beta x + \varepsilon_i, i=1,...,n,$$ with $E(\varepsilon_i)=0$, and with $Var(\varepsilon_i)=a_i\sigma^2$ for all i, with $a_i$ known. I ...
0
votes
1answer
35 views

Variance of ridge regression estimator

These are the facts as I know them. The ridge regression estimator, $\hat{\beta}_R$, is given as; \begin{equation} \hat{\beta}_R = \left(X'X + kI \right)^{-1}X'y, \ k \geq 0 \end{equation} and the ...
0
votes
0answers
11 views

Avenues for further study of a linear model?

I have been assigned a project where I take a dataset and fit a regression model. I have found that the model I have fitted is poor, even after making several updates to the variables used in order to ...
0
votes
0answers
9 views

Validity of scatter plots for multivariate regression

I was just wondering how reliable scatter plots are in the context of multivariate regression. Say, for example, I want to fit the following model: ...
0
votes
0answers
14 views

Weighted least squares with sample variance

I am taking a look at some practice problems for the weighted least squares estimator. However I encountered a problem where I am second-guessing what my W matrix should be. I know what the other ...
0
votes
1answer
22 views

Why is $\arg\max \space f(x) = \arg\min \space \left\{-\log \space f(x) \right\}$?

Why is $$\arg\max \space f(x) = \arg\min \space \left\{-\log \space f(x) \right\}$$ The left side is the $x$ value where the function $f(x)$ has the maximum value. The right side is confusing me. The ...
0
votes
0answers
35 views

The least squares estimators $\hat{\beta_0}$ and $\hat{\beta_1}$ are uncorrelated.

Consider the simple linear regression model $y=\beta_0+\beta_1x+\epsilon$ with $\epsilon \sim\ \text{NID}(0,\sigma^2)$. Show that the least squares estimators $\hat{\beta_0}$ and $\hat{\beta_1}$ are ...
0
votes
1answer
19 views

Variance inflation factor with two predictors

I think this may be a simple question, but if we have two predictor variables where our regression model can be expressed by an equation of the form $$Y=\beta_0+\beta_1X_{t1}+\beta_2X_{t2}+\epsilon_t$$...
0
votes
0answers
11 views

transformation using Barlett's method

I am working in regression analysis and there was a problem that asked to use Barlett's method to obtain a transformation to make the variance of Poisson approximately constant ($\sigma^2=\Omega(\mu)=\...
0
votes
0answers
26 views

Do regression trees usually lead to continuous regression functions?

I found these slides online which talk about Bagging. If each tree has a corresponding regression function do regression trees usually lead to continuous regression functions?
0
votes
0answers
20 views

Reference Request: Generalization bounds for kernel ridge regression

Let $A$ be an $n\times m$ matrix, $(X_n,Y_n)_{n=1}^N$ be a set of i.i.d. random vectors taking values in $\mathbb{R}^m\times \mathbb{R}^k$ and define the $M$-estimator $S^{(N)}$ by: $$ S^{(N)}(x)\...
0
votes
0answers
17 views

Removing discrete/dichotomous explanatory variables from a regression.

I'm currently trying to fit a multiple regression model containing 7 variables. However, only five are continuous, and for some reason I feel like it makes more sense to remove the other 2 variables (...
1
vote
0answers
28 views

Best-Subset Regression based on BIC versus Forward Selection based on AIC

I am trying to get a better grasp of BIC and AIC scores. I know BIC has a harsher penalty than AIC regarding model size (it prefers smaller, less complex models). Suppose there is a situation where I ...
0
votes
0answers
16 views

Time Series Regression when X are not i.i.d

Suppose we have two time series $x_t$ and $y_t$ and both are i.i.d. The content of $y_t$ and $x_t$ do not matter. However, we run the following regression \begin{align*} y_t = a + b z_t, z_t = x_t + ...

1
2 3 4 5
51