# Questions tagged [regression-analysis]

This tag is for questions about regression analysis. In statistical modeling, regression analysis is a statistical process for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors').

393 questions
Filter by
Sorted by
Tagged with
21 views

18 views

### Understanding Cook's Distance

Does Cooks Distance tell us how much the estimated parameter values change when the ith observation is removed or how much the fitted values change when the ith observation is removed? I'm being told ...
11 views

### How to calculate sample standard deviation of the intercepts that are obtained from all possible combinations (having $n \ge 3$ points)?

Starting with an example: Having the points $P_1(1,29),P_2(2,50),P_3(4,88)$. Then there are $4$ possible ways to choose points to find the "best" line (using ordinary least-squares). The $4$ ...
25 views

### Regression calculation of initial individual weight considering variations in weighing dates and missing data

I need to determine the initial individual weight of each animal as accurately as possible, taking into account the following challenges: The animals may be weighed on different days. There may be ...
90 views

### Linear Algebra/Matrices Reference Request

I am learning multivariate regression, with matrix equations and heavy use of: Transposes Inverses Identity, idempotent matrices Can anyone refer some books where I can learn the properties of ...
18 views

### General question about lasso variable selection consistency

In the context of high dimension problem, I observe a pattern that some paper start with variable selection consistency, i.e. the probability that the estimated important variable set equals the true ...
26 views

### When does averaging affect a linear regression?

Given the following: data $(x_{data},y_{data})$, where x values may be measured multiple times. For example, $$x_{data} = ( 1, 2, 2, 3,3,3, 4 )$$ $$y_{data}=(8,10,9,1,2,1.5,−7)$$ And a fitting ...
49 views

13 views

### Poisson log-linear model for estimating large size count data

Assume that the random variable $E_i = E|x_i$ satisfies Poisson processes. Then we could consider Poisson Log Linear Regression to estimate $\mathbb{E}[E_i] = \lambda_i$, the conditional expectation ...
This is basically a physics problem but I will try my best to highlight the mathematics behind it. Suppose I have two functions: $$T(z,B)=\frac{\text{z}^3 e^{-3 A(\text{z})-B^2 \text{z}^2}}{4 \pi \... 1 vote 2 answers 545 views ### Covariance of Residuals and Fitted Values in Linear Regression Consider the simple linear regression model Y_i = \beta_0 + \beta_1x_i + \epsilon_i where \epsilon_i \sim^{indep} N(0, \sigma^2) for i = 1,...,n. Let \hat{\beta_{0}} and \hat{\beta_{1}} be ... 0 votes 0 answers 89 views ### QR decomposition for Linear independence I have the following linear model: \mathbf{\tau} = \mathbf{\Phi} \mathbf{\pi} where \mathbf{\tau} is a known vector, \mathbf{\Phi} a known regressor and \mathbf{\pi} is a vector of some ... 0 votes 0 answers 43 views ### How to compute expectancy and variance of Recursive-Least-Squares 1-step-ahead residuals I'm trying to learn about the following technique: CUSUM of RLS 1-step-ahead residuals as described in (Brown, Durbin, Evans 1975) in order to determine whether a significant change occurred in the ... 0 votes 0 answers 15 views ### Estimate \hat{E}[Y|X=x] for lognormal distribution We assume that Y given X=x has lognormal distribution with parameters \beta_0+\beta_1ln(x) and \sigma^2, i.e.$$(Y|X=x)\sim LN(\beta_0+\beta_1\ln(x),\sigma^2).$$We want to find estimate of ... 0 votes 1 answer 30 views ### create a function from data (that probably doesn't fit) using many many many calibrating parameters I have the following: \lambda_1 = \frac{const_{A}}{value1_1} + \frac{const_{B}}{value2_1} + \frac{const_{c}}{value3_1}  \lambda_2 = \frac{const_{A}}{value1_2} + \frac{const_{B}}{value2_2} + \frac{... 1 vote 0 answers 21 views ### Trying to figure out multicollinearity. I am learning multiple regression at the momemnt. And maybe I'm just sleep deprived, but I am supposed to assess whether increasing student teacher ratios (total enrollment/teachers, str) will improve ... 0 votes 1 answer 59 views ### How do I calculate the regression equation of y = ax^2 (find the coefficient a that gives the smallest sum of squares of errors)? The input is a group of points (x,y). I am trying to find the regression equation of y = ax^2 (I am trying to find the coefficient a that gives the smallest sum of squares of errors from the ... 0 votes 0 answers 23 views ### Mixed linear models variance estimation in R - random effect varience zero I have a general model that looks like this : Y_i=\vec 1j_i*mu +\vec 1_jib_i+\varepsilon_i and I have data that looks like this: ... 0 votes 0 answers 21 views ### Looking for a fitting function I want to fit some data points that follow this trend: I've tried an exponential of this kind: f(x)= a\exp(-bx) but it's not good enough. Could you give me any hint? Many thanks. 0 votes 0 answers 20 views ### Fixing heteroskedasticity in multiple linear regression I am currently working on model with several variables, yet only one of them causes heteroskedasticity in the model. All the methods for solving such case that I am aware of only work for single-... 1 vote 2 answers 135 views ### Book recommendation on linear and nonlinear regression I am doing a very complex and in-depth course on regression (studying math), but the professor is flying through it and the book often does the same, or is very hard to understand. I wanted to know if ... 0 votes 1 answer 37 views ### Cosine model design matrix non-lineal model I need to applied the T- student testing to the parameters \beta_0, \beta_1, \beta_2,\beta_3,\beta_4 which model is:$$ y = \beta_0 + \beta_1 t + \beta_2 Cos(\beta_3 t+\beta_4)$$to do that, I need ... 0 votes 1 answer 193 views ### Multiple regression by successive orthogonalization I was studying The Elements of Statistical Learning book and trying to understand the section where multiple linear regression is explained by successive orthogonalization procedure, i.e. Gram-Schmidt ... 0 votes 1 answer 59 views ### Can power regression help finding optimal fitting polynomial? I would like to non-heuristically (dis)prove the following statement: "The degree of the optimal polynomial to fit to some data corresponds with the closest integer to the resulting exponent from ... 2 votes 2 answers 253 views ### Linearization before curve fitting I am currently studying at college curve fitting to a set of experimental data. One of our activities/homework was to fit the curve y=ax^b to the set of points of the position of a free-falling ... 0 votes 0 answers 190 views ### What's the difference between a general linear model and a generalized linear model? I need to use a model for my Master's thesis. Looking beyond multiple linear regression, I have found extensions like the general linear model, and the generalized linear model. What is the difference ... 4 votes 2 answers 80 views ### Line of best fit for \{(n,n+\sin n) : n \in \mathbb{Z}\} It seems intuitive that the line of best fit for \{(n,n+\sin n) : n\in \mathbb{Z}\} should be y=x. More concretely, it seems like a reasonable conjecture would be: If y = m_k x + b_k is the ... 1 vote 0 answers 90 views ### Is this linear regression? I'm not sure how to distinguish linear from nonlinear regression. Linear regression should be linear in the parameters. I have the doubt if the following equation can be considered as a linear ... 0 votes 1 answer 44 views ### Least Squares two different forms for the residual Least squares residuals are given in the basic form:$$r_i = y_i - f(x_i, \theta)$$where y_i is the observation and f(x_i,\theta) is the value predicted by the model. I came across another form ... 2 votes 1 answer 223 views ### Variance-stabilizing transformation on a simple linear regression I am currently working with variance-stabilizer method and readed something about it from my textbook. I want to understand it better so I would like to consider a case where I for instance have a ... 0 votes 1 answer 104 views ### Simple linear regression (sum of residuals and predictor) Show explicitly that the following identity holds under a Simple Linear Regression:$$ \ \sum_{i=1}^n r_i \hat{\mu_i} =0$$with residuals  r_i = y_i − \hat{\mu_i}  and \hat{\mu_i} = \hat{\beta_0}+\... 2 votes 1 answer 248 views ### How to explain covariance in logistic regression + analogy to linear regression Introduction Linear model In linear regression we predict continuous variable Y \in R^n with use of n \times p deterministic plan matrix X and theoretical model (let's ignore intercept ... 0 votes 1 answer 33 views ### Econometrics/Statistics Regression Question [closed] As you can see from the provided picture given Heart attack given rate per 100,000 population. I was able to successfully ran my regression; but now I am trying to figure out how to alter my ... 1 vote 1 answer 21 views ### Use regression to find common noise component Suppose I have three mutually independent non-Gaussian noise E_A, E_B, E_C. There are two variables generated by linear combinations of these noise components: M=pE_A+qE_B, N=rE_B. By linear ... 0 votes 1 answer 42 views ### What is the best way to estimate the parameters of a logistic regresion model? I recently read about logistic regression model.$$y=\frac{1}{1+e^{-(\beta_0+\beta_1x)}}$$where y is a categorical variable with either 0 or 1 output. What seems to be perplexing to me is, I can see ... 0 votes 0 answers 22 views ### What type of statistical test should I use for this specific example? I am doing a research project analyzing COVID-19 Cases and its effect on unemployment rates within a country. So, for example, I have the percentage of the population that have COVID-19 in a country ... 1 vote 0 answers 30 views ### What function would model Mercury's orbital velocity around the Sun? I am working on a mathematical investigation for my school work and in my investigation, I am trying to model Mercury's velocity around the sun. I picked up data for the velocity from the NASA ... 0 votes 1 answer 94 views ### Using GPA and Class Rank/Percentile Data to create a regression based on the assumption of a normal distribution. I was interested in seeing if I can use just individual data points, knowing what the percentile of those GPA values is to be able create a normal distribution to predict all other GPA values. For ... 1 vote 1 answer 78 views ### Proving the Mulitple Coefficient of Determination Formula (correlated explanatory variables) I stumbled upon the following formula for the coefficient of determination:$$1-R_{y(x_1,x_2...x_n)}^2=\left(1-\rho_{y,x_1}^2\right)\left(1-\rho_{y,x_2(x_1)}^2\right)\left(1-\rho_{y,x_3(x_1,x_2)}^2\...
The ridge estimator $(\hat{\beta}_R)$, and the expected value, are defined as; \begin{align} \hat{\beta}_R &= \left( X'X + kI \right)^{-1}X'y, \ k \geq 0 \\ \text{E}\left( \hat{\beta}_R \...