Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

2,509 questions
Filter by
Sorted by
Tagged with
8 views

Proving a sufficient condition for restricted strong convexity

I am stuck at some sub question of this problem and I would be glad to get some help. Suppose X is a design matrix with known sparsity level, and it is normalized; : https://i.stack.imgur.com/lL4Pr....
18 views

87 views
+50

29 views

What data do i use to calculate the variance around a predicted value given from a simple regression equation?

Trying to figure out what piece of data goes where in the attached formula for calculating the variance around a predicted value given from a simple regression equation $y_d = a+b*x_d$. I have ...
92 views

Exponential regression GLM

Consider some positive random variables $X^1, X^2$ and $Y\sim Exp(p)$ where $p=\beta_0+\beta_1X^1 + \beta_2X^2$. We have a random sample $\{X^1_i, X^2_i, Y_i\}$. Now, estimate $\beta_1, \beta_2$ is ...
16 views

26 views

How to compute confidence intervals and standard error for nonlinear regression with three parameters?

I have been working on a personal project trying to emulate the nonlinear regression functionality of Mathematica for three free parameters. I am able to accurately fit functions, yet I am unsure how ...
37 views

Maximum likelihood for regression

I've started studying maximum likelihood estimation for regression. What I am trying to do is to understand the process step by step. That is how I've structured the material I've read: The ...
16 views

Interpreting Coefficients in Log-Log Model with Multiple Regressors

If we have a multiple linear regression model $Y=\beta_1+\beta_2 X_2+\beta_3 X_3 +u$ Then we interpret $\beta_2$ as the resultant increase in $Y$ when $X_2$ rises by one unit, holding $X_3$ constant. ...
42 views

11 views

"log of the variance at the second level" in the Stata statistical software package?

I was conducting a logit fixed-effects regression, and Stata (software package) reported to me the "lnsig2u", which apparently is the log of the variance at the second level. I could not ...
14 views

how to find F-test statistic to this null hypothesis given the two models?

k=20,n=100 SSreg=207.5 , SStot=900.9 B1=B2-0 SSreg=100.9 , SStot=900.9 Here is the formula I thought I need to use: (Rss1-Rss2)/(k2-k1)/(Rss2/(n-K2)) however in this example in specific I don't have ...
11 views

Binary to Multiclass logistic regression and vice versa

As I was working on a problem, I came across the mention of logistic regression being used for binary and multiclass problems. Specifically, I am very keen on the problem with the below equations. How ...
32 views

Bias of ridge estimator

The ridge estimator $(\hat{\beta}_R)$, and the expected value, are defined as; \begin{align} \hat{\beta}_R &= \left( X'X + kI \right)^{-1}X'y, \ k \geq 0 \\ \text{E}\left( \hat{\beta}_R \...
24 views

Conditional Expectation of Response Variable given Predictor Variable in Statistical Modeling

I'm a bit confused about the implication of the following: Suppose we are given a set of data points $(X_i, Y_i), i=1,2,...,n$, where $X_i$ is the predictor variable, and $Y_i$ is the response ...
261 views

Beta regression

Consider some positive random variables $X^1, X^2$ and $Y\sim Beta(p; 1)$ where $p=\beta_1X^1 + \beta_2X^2$. We have a random sample $\{X^1_i, X^2_i, Y_i\}$. Now, estimating $\beta_1, \beta_2$ is not ...
21 views

Overfitting the pinball loss in quantile regression

I have a question about the pinball loss, $$\rho_\tau(y, \hat{y}) = (y - \hat{y}) (\tau - \mathbb{I}(y-\hat{y}<0)),$$ which is often applied in quantile regression and typically looks like Given ...
20 views

errors associated with each observations based on their distance to a linear regression plane

This is in reference to outlier analysis by Charu C Aggarwal. Let $D$ be a dataset of dimension $N \times d$ where N is the number of observations and d is the dimensions (or variables). Here, $D$ is ...
30 views

Conjugate Bayesian analysis of linear regression with correlated residuals

I am interested in a Bayesian treatment of (univariate) linear regression in the presence of correlated residuals, but I am somewhat stuck trying to come up with a neat parametrization for a conjugate ...
27 views

Transform linear regression model with non-constant variance to constant variance

I have the following linear regression model: $$Y_i = \alpha + \beta x + \varepsilon_i, i=1,...,n,$$ with $E(\varepsilon_i)=0$, and with $Var(\varepsilon_i)=a_i\sigma^2$ for all i, with $a_i$ known. I ...
35 views

Variance of ridge regression estimator

These are the facts as I know them. The ridge regression estimator, $\hat{\beta}_R$, is given as; \begin{equation} \hat{\beta}_R = \left(X'X + kI \right)^{-1}X'y, \ k \geq 0 \end{equation} and the ...
11 views

Avenues for further study of a linear model?

I have been assigned a project where I take a dataset and fit a regression model. I have found that the model I have fitted is poor, even after making several updates to the variables used in order to ...
9 views

Validity of scatter plots for multivariate regression

I was just wondering how reliable scatter plots are in the context of multivariate regression. Say, for example, I want to fit the following model: ...
14 views

Weighted least squares with sample variance

I am taking a look at some practice problems for the weighted least squares estimator. However I encountered a problem where I am second-guessing what my W matrix should be. I know what the other ...
22 views

Why is $\arg\max \space f(x) = \arg\min \space \left\{-\log \space f(x) \right\}$?

Why is $$\arg\max \space f(x) = \arg\min \space \left\{-\log \space f(x) \right\}$$ The left side is the $x$ value where the function $f(x)$ has the maximum value. The right side is confusing me. The ...
35 views

The least squares estimators $\hat{\beta_0}$ and $\hat{\beta_1}$ are uncorrelated.

Consider the simple linear regression model $y=\beta_0+\beta_1x+\epsilon$ with $\epsilon \sim\ \text{NID}(0,\sigma^2)$. Show that the least squares estimators $\hat{\beta_0}$ and $\hat{\beta_1}$ are ...
I think this may be a simple question, but if we have two predictor variables where our regression model can be expressed by an equation of the form $$Y=\beta_0+\beta_1X_{t1}+\beta_2X_{t2}+\epsilon_t$$...