# Questions tagged [regression-analysis]

This tag is for questions about regression analysis. In statistical modeling, regression analysis is a statistical process for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors').

329 questions
Filter by
Sorted by
Tagged with
11 views

### A normally distributed LRG (see attached) a covariance matrix. Transform the model to LRG model. Prove that it is an LRG model. [closed]

Assume the common linear regression model such that the variances do not have to be equal anymore. Now we assume Yi = b1 + b2xi + Zi for i = 1, . . . , N, where L(Z) = NN (0, Σ) and Σ a covariance ...
26 views

### Does high standard error and high r-square imply spurious regression?

Does a regression passed on time series data with one independent variable and one dependent variable which yields parameters with very high standard errors (t-values) and also a high r-squared imply ...
15 views

Least absolute deviations problem minimization $\min_{β∈R}|y_j − x_{j1}β_1|$, for j = 1,....N Here y is the dependent variable and x is the independent variable. What happens to the case when $x_{... 0answers 11 views ### Which covariance matrix should I use for treating heteroskedasticity in my panel data? I have a data set with panel structure (panel data) with 78 individuals observed over 5 three-year periods. I have 10 dependent variables an 1 independent variable. I applied logarithmic ... 1answer 25 views ### Taylor series approximation of Gaussian Q function ,$Q\left( x \right) = \frac{1}{{\sqrt {2\pi } }}\int_x^\infty {{e^{ - \frac{{{v^2}}}{2}}}dv} $I am trying to find a Taylor series expansion for Gaussian Q function. I have seen that error function$Erfc(x)$is an approximation of$Q(x)$(Is my assumption correct?).$Erfc(x)$has Taylor ... 0answers 29 views ### How to conduct DFFITS and DFBETAS for a multiclass logistic regression problem? Can anyone help me out how to conduct DFFITS and DFBETAS for a multiclass logistic regression problem? That is, do we have a software(preferably R) implementation of it? I also couldn't find the ... 1answer 20 views ### What are the “moment conditions” in the GMM method? Also: GMM vs IV vs 2-stage least squares? GMM = generalized method of moments IV = instrumental variables 2SLS = Two stage least squares OLS = ordinary least squares I keep seeing talk of 'moment conditions' or 'moment equations', but don'... 0answers 10 views ### Is there any other way to conduct ROC on multiclass logistic regression model? Suppose I want to conduct a Receiver Operating Characteristics Analysis on multi-class logistic regression model. Suppose there are three classes$X,Y,Z$. One of the ways is proceed by conducting ROC ... 0answers 16 views ### Multicollinearity in logistic regression model Suppose I want to fit a logistic regression model to my training data.However, I know that some of my regressors may be dependent on each other i.e. some of them are functions of common covariates for ... 0answers 12 views ### Ss converting coefficients of a multivariable linear regression to odds ratio appropriate? I'm currently working on finishing up a paper and there is some discussion regarding whether or not it is appropriate to convert variable coefficients from a multivariable regression to odds ratio ... 0answers 26 views ### Compute grid orientation, spacings, skew and origin from a set of 2D points I have a collection of 2D points that are originated from a regular grid (rectangular grid most of the time, but possibly skewed as well). Each point can be located on the grid from its Row and Column ... 0answers 13 views ### Why E(Y) and not Y in simple linear regression equation? I was watching this video on YouTube of simple linear regression equation, and he said that E(Y)=Beta0 + Beta1x + elipson, and said that there is a whole distribution for the possible Y values, and ... 0answers 16 views ### What is the connection between MSE in regression and MSE for prob. distributions? When assigning a goodness of fit to the least squares regression, one often naturally takes the mean squared error (MSE) or average residual: $$MSE = \frac{1}{n}\sum_{i}{(y_{i}-\hat{f}_{D}(x_{i}))^2}$$... 0answers 9 views ### For the regression model in matrix form$W=X\beta+\varepsilon$, specify the explicit forms of$W,X$and$\beta$. I need help from the great experts regarding this tricky question Two-sample t-test with equal variances: Suppose that we have$X_1,…,X_(n_1 )$are i.i.d from a normal Population A with mean$μ_X$... 0answers 17 views ### Proof as to why in a least square exponential regression model, partial derivatives, equated to zero, give the minimum value of a function. i have a data set given of (x1; y1) ... (xn; yn), the best fit of the data is assumed to be y=a*exp(bx), with a and be being variables (Sorry, for the equation formating, I do not have a reputation of ... 0answers 26 views ### Removing noise from signal? (correlated noise and any function) I am trying to find any type of function that will allow me to discern between the function and correlated noise. I can only think of examples that don't work, like: ... 1answer 35 views ### Can anyone advise me on how to solve this minimisation problem? I have the following system of 3 equations and 3 unknowns: $$c_{0} = \frac{x_0}{x_0 + x_1},\ \ c_{1} = \frac{x_1}{x_1 + x_2},\ \ \ c_{2} = \frac{x_2}{x_2 + x_0},$$ where$c_i\!\in\!(0,1)$are known ... 0answers 17 views ### The mean of prediction variance in Bayes inference In Bayes inference, we define p(w) and p(y|X, w) as following. $$p(w) = N(w | 0, \sigma_w^2 I) \\ p(y | X, w) = N(y | Xw, \sigma_y^2 I)$$ and we can calculate p(w|X, w) as following. $$p(w | X, y)... 0answers 50 views ### How do you use the context of a problem to determine what your error values mean in linear regression? I have been told that you must consider the context of a problem in order to determine what your error metrics are telling you. Question: What thought process would you go through to determine how ... 1answer 49 views ### Are these two regressions equivalent? I want to make regression of Y on basis which consists of two sets \phi_1(X), \phi_2(X), ..., \phi_K(X) and \psi_1(Z),\psi_2(Z),...,\psi_M(Z). X and Z are independent random variables I ... 1answer 17 views ### Do we factor in sample size in choice of number of folds in cross-validation? My understanding is that there is no formal theory/rule on how to choose the number of folds (k) in k-fold Cross-Validation. I have read that a good choice lies between 15-20 percent of the data in ... 1answer 37 views ### What is “Cross-Validation Error” in plain English? Say you use Cross-Validation to fit a regression model to a dataset. You get a bunch of CV-scores (cross-validation errors). What exactly is a Cross-Validation Error? 0answers 13 views ### Understand MSE plot of: MSE v.s. principal components used in model I have fitted a linear regression model (through principal components regression) to a high-dimensional (training) dataset using cross validation. I have plotted the "mean squared error of ... 0answers 11 views ### Detail in regard to high dimensional data I have run into (versions) of the following formulation multiple times: "In high dimensional regression we have a setting in which the number of \bf{unknown\ parameters\ p} is larger than the ... 2answers 32 views ### Fitting to top of point cloud rather than the middle - non-linear regression with negative residuals Problem: There are some types of estimatino problems where the noise function is not normal, and where the mean of the residuals can not/should not be 0. A common example is trying to resolve the ... 0answers 25 views ### Meaning of dimensional reduction in principal components regression. When performing principal components regression (PCR) on a high-dimensional dataset we are often looking for a few informative predictor variables in an ocean of uninformative variables. Principal ... 0answers 11 views ### Finding an approximating function for the following data without using regression I have the following data : x=[1,2,3,4,5,6...,22] y=[40, 50, 80, 100, 130, 170, 210, 250, 300, 360, 420, 490, 560, 630, 720, 810, 910, 1010, 1110, 1260, 1380, 1520, 1660] I have observed the following:... 0answers 21 views ### How will the confidence band change? I was thinking of a particular problem of multiple linear regression. Suppose,if I drop the intercept term from a linear model will the confidence band for each of the regression coefficients become ... 0answers 23 views ### MARS model: How to select hinge-functions? I try to understand the "MARS"-model (Multivariate adaptive regression splines). I understand it is built using hinge functions, but I don't get it how they are selected in the "forward pass". ... 0answers 23 views ### Nonlinear Regression with machine learning methods Consider the following example: I have a big set of test data with input x (of dimension 10) and output y. The plot y vs x shows that y depends nonlinear on x. I want to construct a ... 1answer 33 views ### Linear or non-linear regression Given an equation, say, y^{1/n} = x^{1/n} + z^{1/n} and a bunch of 3-dimensional sample points, what is the best way to find the optimal value for n that best fits the sample points? I suppose ... 0answers 26 views ### can we using Pearson Correlation as estimater to create function look like linear regression I found in the field of Marketing Research someone using this way we have data [ y, x1,x2,x3] so calculation Pearson's correlation like this ... 0answers 20 views ### Proof Hat Matrix diagonal value in n dimension I have this question for one homework : Consider a simple regression model with size λ and H the hat matrix then prove that h_i = \frac{\sum_{k=1}^n (X_k - X_i)^2}{n \sum_{k=1}^n (X_k - \bar X)^2}... 0answers 11 views ### Nested Negative Binomial (potentially zero-inflated) Regression I am currently working on methods for a thesis project. I will be modeling the outcome of disease incidence for two different diseases using negative binomial regression. It will most likely be zero-... 0answers 16 views ### How exactly is the two sample t-test a special case of a general linear model? "The two sample t-test can be regarded as a special case of the general linear model: Suppose Y_{1j} and Y_{2j} are two independent groups of random variables. The two-sample test assumes Y_{ij}\... 0answers 22 views ### How to transform fitted value in levels to a predicted log value I understand that if I estimate a linear regression model where the dependent variable is \ln y, then to get predicted values in levels I need to do something like: \hat{Y} = \alpha * \exp \left(\... 1answer 31 views ### Simple linear regression: the intercept and the parameter estimator At page 53 of the famous book The Elements of Statistical Learning by Hastie et al., given a univariate regression model of the type$$Y=X\beta + \epsilon$$the estimator of the \beta parameter is ... 0answers 32 views ### Nonlinear regression Let's consider the model$$y_1=\cos^2(\theta)+\epsilon_1,\\ y_2=\sin^2(\theta)+\epsilon_2,$$where \epsilon=(\epsilon_1, \epsilon_2)^\top, \epsilon\sim N_{2}(0,\sigma^2I), \theta\in\left[0, \... 0answers 38 views ### prove that (\hat{\beta}-\beta)'(X'X)^{-1}(\hat{\beta}-\beta) and SSE are independent We have$$(\hat{\beta}-\beta)'(X'X)^{-1}(\hat{\beta}-\beta)$$and SSE =\sum_{i=1}^{n}(\hat{y_{i}}-y_{i})^{2}=\sum_{i=1}^{n}y_{i}^{2}-\hat{\beta'}X'y and I have to prove that these 2 quantities are ... 0answers 19 views ### Fitting a non-linear model to multiple datasets I have a non-linear model M(p_1,\ldots,p_n) that describes a dataset D, where p_1, \ldots, p_n are the parameters of the model. My goal is to predict these parameters along with the ... 0answers 23 views ### Rewrite a linear mixed-effects model for 1\leq i \leq n and for 1\leq j \leq J A linear mixed effects model defined for 1\leq i \leq n and for 1\leq j \leq J by:$$Y_{ij}=a+(b+Z_i)t_j+\epsilon_{ij}$$where$a\in\mathbb{R}$,$b\in\mathbb{R}$,$t_j\in\mathbb{R}$for$1\leq j ...
96 views

I'm doing practice problems and i'm having a bit of difficulty with this one. I'm not used to deal with quadratic forms in regression. I tried my possible to answer it but i'm not sure it's right? ...
67 views

### How to prove sum of errors follow a chi square with $n-2$ degree of freedom in simple linear regression

In simple linear regression, the model is \begin{equation} Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i \end{equation} where $\varepsilon_i$ are i.i.d., and \begin{equation} \varepsilon_i \sim N(0, \...
14 views

### T statistic for intercept and coefficient

So, assuming that W0 is the intercept and W1 is the coefficient in a simple linear regression model, the way to calculate a t statistic for W1 is (W1-0) /std error of w1 Now, my question is how do I ...
377 views

### Residual analysis in Python

when doing residual analysis do we first fit our model on our entire training set and calculate residuals between fitted values and actual values? Or do we first fit our model on the training+testing ...
15 views

### interpreting residual plots for model assumptions

I have the following output from SAS for a data set I entered, using a linear model. From what I have learned, I made the following two conclusion: The residuals do not lie in a band around zero, so ...
25 views

### How is using the QR decomposition in the linear least squares solution beneficial? How is it more preferred?

Please Help !! I am curious, as to why using the $\mathbf{X}=\mathbf{QR}$ decomposition in the linear least squares solution, $\boldsymbol{\beta}=(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y}$,...
15 views

### Probit vs Linear Probability Model for binary independent variable

I am new to binary regression models. From my understanding, the probit is better than the LPR because it allows for varying marginal effects and constraint Y_hat to be within 0 & 1. However, if ...
So, I'm new to linear regression and this question might be easy, but I didn't find a similar question (one with the same regression model) on the internet. Suppose that $X_1$ and $X_2$ are two ...