Questions tagged [regression-analysis]

This tag is for questions about regression analysis. In statistical modeling, regression analysis is a statistical process for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors').

Filter by
Sorted by
Tagged with
2 votes
1 answer
52 views

Variance-stabilizing transformation on a simple linear regression

I am currently working with variance-stabilizer method and readed something about it from my textbook. I want to understand it better so I would like to consider a case where I for instance have a ...
user avatar
0 votes
0 answers
11 views

Intuition of R-squared in a quantile

I assume, R-squared (correlation of determination) can be used as a measure of the goodness-of-fit in a quantile plot (QQ-plot). In a QQ-plot we measure between empirical quantiles (the ranked sample) ...
user avatar
  • 111
0 votes
0 answers
18 views

Alternative Regression

Consider the least square regression $y ∈ R^n$ on $X ∈ R^{n\times k}$, and the alternative regression on $Z = XC$, where $C ∈ R^{k×k}$ is a invertible matrix. Compare the least square estimators and ...
user avatar
  • 11
0 votes
1 answer
38 views

Simple linear regression (sum of residuals and predictor)

Show explicitly that the following identity holds under a Simple Linear Regression: $$ \ \sum_{i=1}^n r_i \hat{\mu_i} =0$$ with residuals $ r_i = y_i − \hat{\mu_i} $ and $\hat{\mu_i} = \hat{\beta_0}+\...
user avatar
  • 77
1 vote
0 answers
12 views

How to explain covariance in logistic regression + analogy to linear regression

Introduction Linear model In linear regression we predict continuous variable $Y \in R^n$ with use of $n \times p$ deterministic plan matrix $X$ and theoretical model (let's ignore intercept ...
user avatar
0 votes
1 answer
32 views

Econometrics/Statistics Regression Question [closed]

As you can see from the provided picture given Heart attack given rate per 100,000 population. I was able to successfully ran my regression; but now I am trying to figure out how to alter my ...
user avatar
  • 49
0 votes
0 answers
20 views

How is variance of estimation parameters $\hat{\beta_0}$ and $\hat{\beta_1}$ in simple linear regression influenced by adding new data points

We know that: $$Var(\hat{\beta_0})=\sigma^2(\frac{1}{n}+\frac{\overline{x}^2}{s_{xx}})$$ and $$Var(\hat{\beta_1})=\frac{\sigma^2}{S_{xx}}$$ How do they change when a new data point $(x_i)$ is ...
user avatar
  • 51
0 votes
0 answers
48 views

Variance of least squares estimates simple linear regression

An experiment is performed. Data follows model: $y={\beta_0}+{\beta_1}x_{1}$ Does the variance of the LS-estimates for $\hat{\beta_1}$ and $\hat{\beta_0}$ increase or decrease when more points are ...
user avatar
  • 1
1 vote
1 answer
18 views

Use regression to find common noise component

Suppose I have three mutually independent non-Gaussian noise $E_A$, $E_B$, $E_C$. There are two variables generated by linear combinations of these noise components: $M=pE_A+qE_B$, $N=rE_B$. By linear ...
user avatar
0 votes
0 answers
22 views

Sorting features based on performance number which is a linear combination of features

I have a dataset that looks like table of data with features as columns (on/off) and resulting performance number in performance column The way to read this data is as follows: When a combination of ...
user avatar
0 votes
1 answer
22 views

What is the best way to estimate the parameters of a logistic regresion model?

I recently read about logistic regression model. $$y=\frac{1}{1+e^{-(\beta_0+\beta_1x)}}$$ where y is a categorical variable with either 0 or 1 output. What seems to be perplexing to me is, I can see ...
user avatar
  • 153
0 votes
0 answers
21 views

What type of statistical test should I use for this specific example?

I am doing a research project analyzing COVID-19 Cases and its effect on unemployment rates within a country. So, for example, I have the percentage of the population that have COVID-19 in a country ...
user avatar
  • 1
0 votes
0 answers
20 views

Multiple Linear Regression when the values of independent variables are not fixed

How should I deal with the problem below involving a multiple linear regression? $z = f(x,y)$, but in the vectors $[x_1,x_2,...,x_n]$ and $[y_1,y_2,...,y_n]$, $x_n$ and $y_n$ are not fixed, i.e., each ...
user avatar
1 vote
0 answers
24 views

What function would model Mercury's orbital velocity around the Sun?

I am working on a mathematical investigation for my school work and in my investigation, I am trying to model Mercury's velocity around the sun. I picked up data for the velocity from the NASA ...
user avatar
0 votes
1 answer
31 views

Using GPA and Class Rank/Percentile Data to create a regression based on the assumption of a normal distribution.

I was interested in seeing if I can use just individual data points, knowing what the percentile of those GPA values is to be able create a normal distribution to predict all other GPA values. For ...
user avatar
0 votes
0 answers
28 views

Partial Autocorrelation given Autocorrelation Function

I am on this question from the Chapter 1 in Introduction to Statistical Time Series (Wayne A Fuller): Let $$ \rho(h)=\left\{ \begin{array}{ll} 1 & \quad h=0 \\ a &...
user avatar
1 vote
1 answer
45 views

Proving the Mulitple Coefficient of Determination Formula (correlated explanatory variables)

I stumbled upon the following formula for the coefficient of determination: $$1-R_{y(x_1,x_2...x_n)}^2=\left(1-\rho_{y,x_1}^2\right)\left(1-\rho_{y,x_2(x_1)}^2\right)\left(1-\rho_{y,x_3(x_1,x_2)}^2\...
user avatar
0 votes
1 answer
44 views

Bias of ridge estimator

The ridge estimator $(\hat{\beta}_R)$, and the expected value, are defined as; \begin{align} \hat{\beta}_R &= \left( X'X + kI \right)^{-1}X'y, \ k \geq 0 \\ \text{E}\left( \hat{\beta}_R \...
user avatar
  • 59
0 votes
0 answers
22 views

errors associated with each observations based on their distance to a linear regression plane

This is in reference to outlier analysis by Charu C Aggarwal. Let $D$ be a dataset of dimension $N \times d$ where N is the number of observations and d is the dimensions (or variables). Here, $D$ is ...
user avatar
0 votes
1 answer
53 views

Variance of ridge regression estimator

These are the facts as I know them. The ridge regression estimator, $\hat{\beta}_R$, is given as; \begin{equation} \hat{\beta}_R = \left(X'X + kI \right)^{-1}X'y, \ k \geq 0 \end{equation} and the ...
user avatar
  • 59
0 votes
0 answers
11 views

Avenues for further study of a linear model?

I have been assigned a project where I take a dataset and fit a regression model. I have found that the model I have fitted is poor, even after making several updates to the variables used in order to ...
user avatar
0 votes
0 answers
11 views

Validity of scatter plots for multivariate regression

I was just wondering how reliable scatter plots are in the context of multivariate regression. Say, for example, I want to fit the following model: ...
user avatar
0 votes
0 answers
23 views

Weighted least squares with sample variance

I am taking a look at some practice problems for the weighted least squares estimator. However I encountered a problem where I am second-guessing what my W matrix should be. I know what the other ...
user avatar
  • 473
0 votes
1 answer
46 views

Variance inflation factor with two predictors

I think this may be a simple question, but if we have two predictor variables where our regression model can be expressed by an equation of the form $$Y=\beta_0+\beta_1X_{t1}+\beta_2X_{t2}+\epsilon_t$$...
user avatar
  • 473
0 votes
0 answers
11 views

transformation using Barlett's method

I am working in regression analysis and there was a problem that asked to use Barlett's method to obtain a transformation to make the variance of Poisson approximately constant ($\sigma^2=\Omega(\mu)=\...
user avatar
  • 473
4 votes
0 answers
105 views

finding optimal entry and exit points in a time series of prices

given an initial 1000 dollars cash value, and a time series of bid price, ask price, assuming constraints of transaction costs, e.g. 1 dollar every time we buy or sell, what is the best approach to ...
user avatar
  • 103
0 votes
1 answer
85 views

What makes inequality true in proof of Gauss Markov theorem

Elsewhere on this site, I found a very compact proof of the Gauss-Markov theorem, seen below. I don't understand the justification for the middle step with the inequality. Specifically, what property ...
user avatar
  • 143
0 votes
0 answers
24 views

Show that $𝑌^𝑡𝑌= \hat{𝑌}^{t}\hat{𝑌}+ 𝑒^𝑡𝑒$

I'm trying to solve this property of multiple linear regression $𝑌^𝑡𝑌= \hat{𝑌}^{t}\hat{𝑌}+ 𝑒^𝑡𝑒$ Any suggestions would be great!
user avatar
0 votes
0 answers
36 views

Test which functional form that best explains data

I had this question in an exam lately and I was not sure how to answer it. Now the exam is done, and I can't go back, but it's been in my head ever since and I'm really curious about the answer. ...
user avatar
0 votes
2 answers
80 views

Can I determine if two random variables are independent if I know their expected values and the variances?

I have two random variables $X$ and $Y$. I know the expected values $E\left[X\right]$ and $E\left[Y\right]$, as well as their respective variances $V\left[X\right]$ and $V\left[Y\right]$ (I have them ...
user avatar
2 votes
1 answer
17 views

low $p$-value and low explained variation connection in multiple regression analysis

I've just started studying multiple linear regression and I'm stuck at creating a dataset for which a multiple regression model would have a low $p$-value (the coefficients are non zero) and also low ...
user avatar
0 votes
0 answers
30 views

How to handle an integral or derivative as a collinear variable in regression?

I would like to regress multiple unknowns relating a dependent variable to an second variable both of which are functions of third independent variable. I can relate the integral of the first ...
user avatar
0 votes
1 answer
55 views

what is your opinion for the best function can be fitted to these plots?

I have three sets of data (x,f(x)) as follows: ...
user avatar
  • 375
0 votes
0 answers
23 views

Standard deviation formula for coefficients in multiple regression analysis

I am trying to understand how to calculate the individual deviation in this regression analysis table. $(HH Size)_{se} =$ $stderr \over \sqrt{S_{xx}} $ $=$ $ {0.444400903 \over \sqrt{10}} = 0.14053$ ...
user avatar
  • 195
0 votes
0 answers
33 views

Relationship between correlation coefficient and the agle between two regression lines

From: https://en.wikipedia.org/wiki/Pearson_correlation_coefficient Geometric interpretation Regression lines for y = gX(x) [red] and x = gY(y) [blue] Regression lines for y = gX(x) [red] and x = gY(...
user avatar
  • 499
-1 votes
1 answer
61 views

How do I judge the accuracy of a model with 3 variables?

For my high-school maths exploration, I have chosen to simulate heat flow in a one-dimensional rod using GNU Octave. I will then solve the heat equation and judge the accuracy of the heat equation in ...
user avatar
0 votes
0 answers
29 views

QMLE for correlated Bernouli model parameter

9Consider a binary dependent variable of the form $$y_t=\begin{cases} 1, & \ \text{with probability} \ p(y_t=1|x_t) \\ 0, & \ \text{with probability} \ 1- p(y_t=1 |x_t) \end{cases}$$ Suppose ...
user avatar
1 vote
0 answers
22 views

Analytic solution of regression problem

I have a regression problem as below. $$ A^* = \operatorname*{argmin}_A \| Y - AX \|_F^2 $$ where $Y$ and $X$ are matrices, and $A$ is a mapping matrix between these two matrices. There are analytic ...
user avatar
  • 81
0 votes
0 answers
21 views

In Twisk (2013) on the equation describing a generalized estimating equation (GEE), why is there a subscripted '1' after beta?

In his book on Applied Longitudinal Data Analysis for Epidemiology, page 60 there is an equation that describes a generalized estimating equations (GEE) model. This equation models the relationship ...
user avatar
  • 101
0 votes
0 answers
50 views

Statistical significance in context of financial data?

I understand statistical significance in the general sense: we take a sample from a population and compute some parameter from the sample to infer what is the propulsion parameter to some degree of ...
user avatar
1 vote
1 answer
59 views

On the difference in nonparametric regression models

I recently read the wikipedia article about nonparametric regression. It contains the following quote: In nonparametric regression, we have random variables $X$ and $Y$ and assume the following ...
user avatar
0 votes
1 answer
105 views

Maximum likelihood variance estimator of simple linear regression is biased

Colleagues, I understand that bias is defined as $b(\theta)=\mathbb{E}(\hat{\theta})-\theta$. How can one show that the variance estimator for maximum likelihood estimators of simple simple linear ...
user avatar
0 votes
0 answers
40 views

supervised model for car accidents?

I can't seem to figure out how I would fit a multinomial distribution here? I think I've managed to produce the linear combination as: yi = α0 + α1x 1,i + α2 x2,i I'd be hugely grateful for any ...
user avatar
  • 15
1 vote
1 answer
74 views

Translate my GLM model with coefficients into a formula

I wasn't sure whether to post here or on stats.stackexchange, but trying here first since my goal is to find/derive a math formula for my r generated model. If you look at my post history you can see ...
user avatar
  • 2,226
0 votes
1 answer
58 views

Find Best Fit Equation From 4 Data Points [closed]

I have thousands of rows which each contain 4 different data values. Data example: A = 799, B = 190.68, C = 131.74, Y = 48.65 A = 1209, B = 9.67, C = 9.67, Y = 311.36 A = 932, B = 212.23, C ...
user avatar
1 vote
0 answers
27 views

Solving gradient expression as a differential equation

I study horizontal gravity gradient $\frac{\Delta g}{\Delta l}$ in certain field. We measure $\Delta g$ and $\Delta l$ thhrough several profiles and same direction. Something like that: enter image ...
user avatar
0 votes
1 answer
122 views

Why square of pearson correlation does not match with r2 score?

As per link of correlation and R2 , it is mathematically shown that square of pearson correlation is equal to r2. However, I am trying to replicate these results with my data in python and I do not ...
user avatar
  • 103
0 votes
1 answer
68 views

Regression and stress management

Participants were randomly allocated to one of two stress management therapy groups (group A vs. group B), or a waiting list control group. Their baseline levels of stress were measured before ...
user avatar
0 votes
1 answer
67 views

Why is "over-parameterized linear regression" non-convex?

Consider the real matrices $V, W, X, Y$. Define the function $L(W, V) := \frac 1 2 \| Y - V W X \|_2^2$. How can it be shown that $L$ non-convex, as claimed here? Tnx. I have tried to look first at ...
user avatar
0 votes
1 answer
38 views

How to write an equation where both independent variables and dependent variables are log transformed in a multiple regression?

How to write the multiple regression model when both the dependent variable and independent variables are log-transformed? I know that without any log transformation the linear regression model would ...
user avatar
  • 11

1
2 3 4 5
8