Questions tagged [regression-analysis]

This tag is for questions about regression analysis. In statistical modeling, regression analysis is a statistical process for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors').

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

A normally distributed LRG (see attached) a covariance matrix. Transform the model to LRG model. Prove that it is an LRG model. [closed]

Assume the common linear regression model such that the variances do not have to be equal anymore. Now we assume Yi = b1 + b2xi + Zi for i = 1, . . . , N, where L(Z) = NN (0, Σ) and Σ a covariance ...
0
votes
0answers
26 views

Does high standard error and high r-square imply spurious regression?

Does a regression passed on time series data with one independent variable and one dependent variable which yields parameters with very high standard errors (t-values) and also a high r-squared imply ...
1
vote
0answers
15 views

Least absolute deviations problem minimization

Least absolute deviations problem minimization $\min_{β∈R}|y_j − x_{j1}β_1|$, for j = 1,....N Here y is the dependent variable and x is the independent variable. What happens to the case when $x_{...
0
votes
0answers
11 views

Which covariance matrix should I use for treating heteroskedasticity in my panel data?

I have a data set with panel structure (panel data) with 78 individuals observed over 5 three-year periods. I have 10 dependent variables an 1 independent variable. I applied logarithmic ...
0
votes
1answer
25 views

Taylor series approximation of Gaussian Q function , $Q\left( x \right) = \frac{1}{{\sqrt {2\pi } }}\int_x^\infty {{e^{ - \frac{{{v^2}}}{2}}}dv} $

I am trying to find a Taylor series expansion for Gaussian Q function. I have seen that error function $Erfc(x)$ is an approximation of $Q(x)$ (Is my assumption correct?). $Erfc(x)$ has Taylor ...
1
vote
0answers
29 views

How to conduct DFFITS and DFBETAS for a multiclass logistic regression problem?

Can anyone help me out how to conduct DFFITS and DFBETAS for a multiclass logistic regression problem? That is, do we have a software(preferably R) implementation of it? I also couldn't find the ...
0
votes
1answer
20 views

What are the “moment conditions” in the GMM method? Also: GMM vs IV vs 2-stage least squares?

GMM = generalized method of moments IV = instrumental variables 2SLS = Two stage least squares OLS = ordinary least squares I keep seeing talk of 'moment conditions' or 'moment equations', but don'...
0
votes
0answers
10 views

Is there any other way to conduct ROC on multiclass logistic regression model?

Suppose I want to conduct a Receiver Operating Characteristics Analysis on multi-class logistic regression model. Suppose there are three classes $X,Y,Z$. One of the ways is proceed by conducting ROC ...
0
votes
0answers
16 views

Multicollinearity in logistic regression model

Suppose I want to fit a logistic regression model to my training data.However, I know that some of my regressors may be dependent on each other i.e. some of them are functions of common covariates for ...
0
votes
0answers
12 views

Ss converting coefficients of a multivariable linear regression to odds ratio appropriate?

I'm currently working on finishing up a paper and there is some discussion regarding whether or not it is appropriate to convert variable coefficients from a multivariable regression to odds ratio ...
0
votes
0answers
26 views

Compute grid orientation, spacings, skew and origin from a set of 2D points

I have a collection of 2D points that are originated from a regular grid (rectangular grid most of the time, but possibly skewed as well). Each point can be located on the grid from its Row and Column ...
0
votes
0answers
13 views

Why E(Y) and not Y in simple linear regression equation?

I was watching this video on YouTube of simple linear regression equation, and he said that E(Y)=Beta0 + Beta1x + elipson, and said that there is a whole distribution for the possible Y values, and ...
0
votes
0answers
16 views

What is the connection between MSE in regression and MSE for prob. distributions?

When assigning a goodness of fit to the least squares regression, one often naturally takes the mean squared error (MSE) or average residual: $$MSE = \frac{1}{n}\sum_{i}{(y_{i}-\hat{f}_{D}(x_{i}))^2}$$...
0
votes
0answers
9 views

For the regression model in matrix form $W=X\beta+\varepsilon$, specify the explicit forms of $W,X$ and $\beta$.

I need help from the great experts regarding this tricky question Two-sample t-test with equal variances: Suppose that we have $X_1,…,X_(n_1 )$ are i.i.d from a normal Population A with mean $μ_X$ ...
0
votes
0answers
17 views

Proof as to why in a least square exponential regression model, partial derivatives, equated to zero, give the minimum value of a function.

i have a data set given of (x1; y1) ... (xn; yn), the best fit of the data is assumed to be y=a*exp(bx), with a and be being variables (Sorry, for the equation formating, I do not have a reputation of ...
1
vote
0answers
26 views

Removing noise from signal? (correlated noise and any function)

I am trying to find any type of function that will allow me to discern between the function and correlated noise. I can only think of examples that don't work, like: ...
0
votes
1answer
35 views

Can anyone advise me on how to solve this minimisation problem?

I have the following system of 3 equations and 3 unknowns: $$c_{0} = \frac{x_0}{x_0 + x_1},\ \ c_{1} = \frac{x_1}{x_1 + x_2},\ \ \ c_{2} = \frac{x_2}{x_2 + x_0},$$ where $c_i\!\in\!(0,1)$ are known ...
0
votes
0answers
17 views

The mean of prediction variance in Bayes inference

In Bayes inference, we define p(w) and p(y|X, w) as following. $$ p(w) = N(w | 0, \sigma_w^2 I) \\ p(y | X, w) = N(y | Xw, \sigma_y^2 I) $$ and we can calculate p(w|X, w) as following. $$ p(w | X, y)...
0
votes
0answers
50 views

How do you use the context of a problem to determine what your error values mean in linear regression?

I have been told that you must consider the context of a problem in order to determine what your error metrics are telling you. Question: What thought process would you go through to determine how ...
0
votes
1answer
49 views

Are these two regressions equivalent?

I want to make regression of $Y$ on basis which consists of two sets $\phi_1(X), \phi_2(X), ..., \phi_K(X) $ and $\psi_1(Z),\psi_2(Z),...,\psi_M(Z)$. $X$ and $Z$ are independent random variables I ...
0
votes
1answer
17 views

Do we factor in sample size in choice of number of folds in cross-validation?

My understanding is that there is no formal theory/rule on how to choose the number of folds (k) in k-fold Cross-Validation. I have read that a good choice lies between 15-20 percent of the data in ...
0
votes
1answer
37 views

What is “Cross-Validation Error” in plain English?

Say you use Cross-Validation to fit a regression model to a dataset. You get a bunch of CV-scores (cross-validation errors). What exactly is a Cross-Validation Error?
0
votes
0answers
13 views

Understand MSE plot of: MSE v.s. principal components used in model

I have fitted a linear regression model (through principal components regression) to a high-dimensional (training) dataset using cross validation. I have plotted the "mean squared error of ...
0
votes
0answers
11 views

Detail in regard to high dimensional data

I have run into (versions) of the following formulation multiple times: "In high dimensional regression we have a setting in which the number of $\bf{unknown\ parameters\ p}$ is larger than the ...
0
votes
2answers
32 views

Fitting to top of point cloud rather than the middle - non-linear regression with negative residuals

Problem: There are some types of estimatino problems where the noise function is not normal, and where the mean of the residuals can not/should not be 0. A common example is trying to resolve the ...
1
vote
0answers
25 views

Meaning of dimensional reduction in principal components regression.

When performing principal components regression (PCR) on a high-dimensional dataset we are often looking for a few informative predictor variables in an ocean of uninformative variables. Principal ...
1
vote
0answers
11 views

Finding an approximating function for the following data without using regression

I have the following data : x=[1,2,3,4,5,6...,22] y=[40, 50, 80, 100, 130, 170, 210, 250, 300, 360, 420, 490, 560, 630, 720, 810, 910, 1010, 1110, 1260, 1380, 1520, 1660] I have observed the following:...
0
votes
0answers
21 views

How will the confidence band change?

I was thinking of a particular problem of multiple linear regression. Suppose,if I drop the intercept term from a linear model will the confidence band for each of the regression coefficients become ...
0
votes
0answers
23 views

MARS model: How to select hinge-functions?

I try to understand the "MARS"-model (Multivariate adaptive regression splines). I understand it is built using hinge functions, but I don't get it how they are selected in the "forward pass". ...
0
votes
0answers
23 views

Nonlinear Regression with machine learning methods

Consider the following example: I have a big set of test data with input $x$ (of dimension 10) and output $y$. The plot $y$ vs $x$ shows that $y$ depends nonlinear on $x$. I want to construct a ...
1
vote
1answer
33 views

Linear or non-linear regression

Given an equation, say, $y^{1/n} = x^{1/n} + z^{1/n}$ and a bunch of 3-dimensional sample points, what is the best way to find the optimal value for $n$ that best fits the sample points? I suppose ...
0
votes
0answers
26 views

can we using Pearson Correlation as estimater to create function look like linear regression

I found in the field of Marketing Research someone using this way we have data [ y, x1,x2,x3] so calculation Pearson's correlation like this ...
0
votes
0answers
20 views

Proof Hat Matrix diagonal value in n dimension

I have this question for one homework : Consider a simple regression model with size λ and H the hat matrix then prove that $h_i = \frac{\sum_{k=1}^n (X_k - X_i)^2}{n \sum_{k=1}^n (X_k - \bar X)^2}...
1
vote
0answers
11 views

Nested Negative Binomial (potentially zero-inflated) Regression

I am currently working on methods for a thesis project. I will be modeling the outcome of disease incidence for two different diseases using negative binomial regression. It will most likely be zero-...
0
votes
0answers
16 views

How exactly is the two sample t-test a special case of a general linear model?

"The two sample t-test can be regarded as a special case of the general linear model: Suppose $Y_{1j}$ and $Y_{2j}$ are two independent groups of random variables. The two-sample test assumes $Y_{ij}\...
0
votes
0answers
22 views

How to transform fitted value in levels to a predicted log value

I understand that if I estimate a linear regression model where the dependent variable is $\ln y$, then to get predicted values in levels I need to do something like: $\hat{Y} = \alpha * \exp \left(\...
0
votes
1answer
31 views

Simple linear regression: the intercept and the parameter estimator

At page 53 of the famous book The Elements of Statistical Learning by Hastie et al., given a univariate regression model of the type $$Y=X\beta + \epsilon$$ the estimator of the $\beta$ parameter is ...
0
votes
0answers
32 views

Nonlinear regression

Let's consider the model $$y_1=\cos^2(\theta)+\epsilon_1,\\ y_2=\sin^2(\theta)+\epsilon_2,$$ where $\epsilon=(\epsilon_1, \epsilon_2)^\top$, $\epsilon\sim N_{2}(0,\sigma^2I)$, $\theta\in\left[0, \...
1
vote
0answers
38 views

prove that $(\hat{\beta}-\beta)'(X'X)^{-1}(\hat{\beta}-\beta)$ and SSE are independent

We have $$(\hat{\beta}-\beta)'(X'X)^{-1}(\hat{\beta}-\beta)$$ and SSE $=\sum_{i=1}^{n}(\hat{y_{i}}-y_{i})^{2}=\sum_{i=1}^{n}y_{i}^{2}-\hat{\beta'}X'y$ and I have to prove that these 2 quantities are ...
0
votes
0answers
19 views

Fitting a non-linear model to multiple datasets

I have a non-linear model $M(p_1,\ldots,p_n)$ that describes a dataset $D$, where $p_1, \ldots, p_n$ are the parameters of the model. My goal is to predict these parameters along with the ...
0
votes
0answers
23 views

Rewrite a linear mixed-effects model for $1\leq i \leq n$ and for $1\leq j \leq J$

A linear mixed effects model defined for $1\leq i \leq n$ and for $1\leq j \leq J$ by: $$Y_{ij}=a+(b+Z_i)t_j+\epsilon_{ij}$$ where $a\in\mathbb{R}$, $b\in\mathbb{R}$, $t_j\in\mathbb{R}$ for $1\leq j ...
0
votes
1answer
96 views

Linear regression (quadratic form)

I'm doing practice problems and i'm having a bit of difficulty with this one. I'm not used to deal with quadratic forms in regression. I tried my possible to answer it but i'm not sure it's right? ...
0
votes
1answer
67 views

How to prove sum of errors follow a chi square with $n-2$ degree of freedom in simple linear regression

In simple linear regression, the model is \begin{equation} Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i \end{equation} where $\varepsilon_i$ are i.i.d., and \begin{equation} \varepsilon_i \sim N(0, \...
0
votes
1answer
14 views

T statistic for intercept and coefficient

So, assuming that W0 is the intercept and W1 is the coefficient in a simple linear regression model, the way to calculate a t statistic for W1 is (W1-0) /std error of w1 Now, my question is how do I ...
0
votes
1answer
377 views

Residual analysis in Python

when doing residual analysis do we first fit our model on our entire training set and calculate residuals between fitted values and actual values? Or do we first fit our model on the training+testing ...
1
vote
1answer
15 views

interpreting residual plots for model assumptions

I have the following output from SAS for a data set I entered, using a linear model. From what I have learned, I made the following two conclusion: The residuals do not lie in a band around zero, so ...
0
votes
0answers
25 views

How is using the QR decomposition in the linear least squares solution beneficial? How is it more preferred?

Please Help !! I am curious, as to why using the $\mathbf{X}=\mathbf{QR}$ decomposition in the linear least squares solution, $\boldsymbol{\beta}=(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y}$,...
0
votes
0answers
15 views

Probit vs Linear Probability Model for binary independent variable

I am new to binary regression models. From my understanding, the probit is better than the LPR because it allows for varying marginal effects and constraint Y_hat to be within 0 & 1. However, if ...
1
vote
0answers
39 views

Obtaining the OLS estimators

So, I'm new to linear regression and this question might be easy, but I didn't find a similar question (one with the same regression model) on the internet. Suppose that $X_1$ and $X_2$ are two ...
0
votes
0answers
22 views

Compute a 95% confidence interval for the mean increase

I'm asked to compute a 95% confidence interval for the mean increase in Y when x1 increases by one unit. Can someone help me out with this? It is for multiple linear regression. Would I just find a 95%...

1
2 3 4 5
7