Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression-analysis]

This tag is for questions about regression analysis. In statistical modeling, regression analysis is a statistical process for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors').

0
votes
1answer
20 views

Given data matrix and error matrix, find value of $x,y,z$

In an exercise about multiple regression, I'm given the following data matrix $X$ and error vector $\epsilon$: $$ X = \begin{bmatrix} 1 & 4 & x \\ 1 & -2 & 1\\ 1 & 1 &...
0
votes
0answers
6 views

Cox proportional hazard model with change point for the relative hazard

I have to consider the following simulation scenario: The covariate X~Be(1/3). If X=0, survival time Y=$E_1$~Exponential(1). If X=1, survival time Y=$E_1$ if $E_1$≤$\psi$, and as $\psi$+$E_{2}$ if $...
1
vote
0answers
17 views

Regressions with two independent variables where one is part of another.

I am doing simple regression analysis where I use one-way fixed effect model to estimate the effects of two variables on dependent variable. The question I am asking is how to interpret these two ...
1
vote
0answers
16 views

How to calculate $\text{cov}(\hat{Y}_{ij}, \hat{Y}_{kj})$ if $Y_{ij} = \mu + a_i + b_j + e_{ij}$?

Let's assume we have the model following Two-Factor model without replications : $$Y_{ij} = \mu + a_i + b_j + e_{ij}, \; i=1,\dots,p \; \text{and} \; j=1,\dots, q $$ I am interested in calculating ...
2
votes
0answers
20 views

Covariance of errors $\text{cov}(\hat{e_{ij}},\hat{e_{i\ell}})$ in Two-Way Anova model

Exercise : Consider the Two-Way Anova model $Y_{ij} = \mu + a_i + b_j + e_{ij}$ with $i = 1, \dots, p$ and $j=1,\dots,q$. Show that : $$\text{cov}(\hat{e_{ij}},\hat{e_{i\ell}}) = -\sigma^2\left(\...
0
votes
0answers
7 views

Prediction of time series based on lagged correlations

I have several questions. I will split the text up in one high-level description of the goal of my exercise, a detailed description of my potential solution and finally my actual questions. Please ...
0
votes
1answer
11 views

Ridge Regression Coefficient Estimate is linear

I was able to derive the formula for the Ridge Regression Coefficient Estimate $\hat{\boldsymbol{\beta}}^{ridge}$. However, I am not 100% sure what it means in terms of showing that the Ridge ...
0
votes
0answers
17 views

Formula for ordinal complementary log-log regression

I have a model developed in R using the polr() function in the MASS package. The model is an ordinal regression with a complementary log-log link (method=cloglog). The model uses four predictors ...
0
votes
0answers
15 views

Is it possible to estimate truncated regression model with varying\stochastic upper and lower bound?

Currently, I am estimating the truncated regression model with fixed values of the upper bound and lower bound. However, in my case, the lower bound and upper bound can be varying across each data ...
0
votes
1answer
28 views

Relationship between different types of correlation coefficients

Let, $r_{1(2.34...p)}$ = Correlation between $x_1$ and $x_{2.34...p}$. The latter being the residuals after regressing $x_2$ on $x_3 , x_4 ....x_p$. $r_{1.234..p}$ = Multiple correlation coefficient ...
0
votes
0answers
11 views

how write residual by adding outliers?

I'm reading a paper about outliers and its detection. there they suppose a initial group of observatios and next they add a group of high leverage identical outliers. where to the initial observations ...
1
vote
1answer
37 views

how to bound the hat matrix?

I'm reading a paper about linear regression and in some point they define: $$ w_{ij}=\frac{h^{2}_{ji}}{ph_{ii}(1-h_{jj})^{2}}$$, where $h_{ij}$ are the elements of the hat matrix. The problem is ...
0
votes
0answers
7 views

Logit link function for large $|\eta|$

I am doing a project on beta regression and have run into a problem with my link function $$g(\eta) = log[\frac{\mu}{(1-\mu)}]$$ $\eta = \sum x_i^T\beta$, with $x_i$ being vectors of explanatory ...
0
votes
0answers
13 views

How do I derive a Least Square Estimator of a linear combination of two variables?

I am working on a problem where I have the following model: lm(Y ~ x1 x2) If I have the output of this general model in R, is it possible to derive the LSE of: <...
0
votes
0answers
24 views

The linear mixed effects model

I know it's probably better to post this question on Stack Overflow or Cross Validated, which I did, but there was no response and I post the question here to see whether I could get some luck. ...
0
votes
1answer
17 views

the norm of solution to overdetermined linear equations

For example, Ax=b is an overdetermined linear equations. We want to minimize $||Ax-b||_2$. So the solution is $x = (A^TA)^{-1}A^Tb$. What I want to ask is how does the norm of x , $||x||_2$, change ...
0
votes
1answer
53 views

Fisher Matrix and Hessian matrix

I know that the Fisher matrix is easily obtained from the Hessian matrix $I\left(\hat{\beta}\right)=-H\left(\hat{\beta}\right)$ Why is the covariance variance matrix the inverse of the Fisher ...
0
votes
1answer
22 views

Calculating the var(β) in a least square regression model

The linear model that I'm working with is: $$y_t =α +βx_t + ε_t$$ Based on my Lecture I have: $$Var(\hatβ) = Var(Σw_tε_t)$$ where ε is the error term and $$w_t = \frac{x_t-\overline x}{Σ(x_t-\...
1
vote
0answers
31 views

On a nonlinear regression problem

Consider the function $f\colon \mathbb{R}^2\to \mathbb{R}$, $f(x_1,x_2)=x_1^2 +x_2$. Assume that I don't know the form of $f$ and I only have a set of $N$ independent "input-output" data $\{(x_1^{(i)},...
0
votes
1answer
26 views

Doubt about the role of “equivalence class of functions” in this Least Square example.

I am dealing with a proof from De Boor (1972) about least square approximations using splines. Suppose we have a set of data and we want to estimate the least square approximation. Let $ \$ $ be a ...
0
votes
0answers
7 views

If $x$ is statistically significant using $2$-sigma rule?

I have the covariance matrix (columns $y, x_1, x_2, x_3$): $$ \left[ \begin{matrix} 750 & 240 & 10 & 280 \\ 240 & 110 & 0 & 100 \\ 10 & 0 & 7 & 0 \\ 280 & 100 &...
0
votes
2answers
59 views

How to choose degree for polynomial regression?

I know how to perform polynomial regression. But is there any method to use for estimating the degree of the polynomial that is best suited? Some kind of meta-regression. With best suited I mean the ...
1
vote
1answer
73 views

Conditional Expectation Decomposition in Regression Analysis

I am currently working on my understanding of regression fundamentals and I checked this source (one can find the (even exact) same statement in multiple sources). In Theorem 3.1.1, the author claims ...
0
votes
0answers
39 views

Odds Ratios with independent linear trends

I have a question regarding odds ratios for a project I'm working on. For my analysis to determine if there was a significant change in access to a service at a certain type of facility from one year ...
0
votes
1answer
99 views

Finding the appropriate polynomial fit for set of data

Is there a function or library in Python to automatically compute the best polynomial fit for a set of data points? I am not really interested in the ML use case of generalizing to a set of new data, ...
0
votes
1answer
28 views

What's the Most Appropriate Type of Regression for this Problem?

I have a data set from two groups: firms that use AI and their costs and firms that don't use AI and their costs. Within both groups I have data about their specific costs, e.g. fixed costs, variable ...
0
votes
0answers
9 views

Proofs for Hierarchical model with Normal Response

I need help with this proof for Applied Regression Analysis. The first two screen caps are of the model and the others are the exercise. I don't even know where to start on this proof
0
votes
0answers
20 views

Standard Error Estimate for Beta Coefficients

Suppose I have a linear regression model consisting of $\beta$ estimates, relative to a reference term. Each of these $\beta$s has a $\bar{x}$, a $s_x$, and then $n$, with a calculated $\hat{\sigma}$ ...
0
votes
1answer
23 views

Please correct my thinking about Ridge Regression

If ridge regression biases ALL beta coefficients of a regression model towards zero, wouldn't the model massively mispredict the y-variable? I know my logic must be wrong here, but I'd appreciate if ...
1
vote
1answer
111 views

L2-norm with estimated weights

Suppose I'm performing linear regression. My lecturer said the formula below can be used for estimating the weight vector that is passed to the L2-norm part of the loss function but he didn't ...
1
vote
0answers
35 views

Linear Regression Diagnostics

I am trying to determine if there is a relationship between a dependent variable y and independent variable x by fitting a least squares regression model. Scatterplot of data: Diagnostic plots: ...
1
vote
2answers
36 views

Show whether or not $\hat\beta$ is a consistent estimator

I have the following model: $y_i=\mathbf x_i'\beta+\epsilon_i $ $E(\mathbf x_i\epsilon_i)=0$ Now, assume there is a positive function $g(x)$, let $g_i=g(\mathbf x_i)$. Consider the estimator: $$\...
0
votes
0answers
25 views

basic, clear Book about linear regression with examples?

i'm taking a course about linear models , and the book used is Ravishanker "a first course of linear models", the problem is that the book its so theoretical and my background is so basic, so is very ...
0
votes
0answers
27 views

VIF - Variance Inflation, when to remove the variable

I'm doing a regression analysis on cement mixtures. The goal is obviously to create the mixture with the most strength. Here are the following variables for me to work with: Variables: Strength = ...
0
votes
0answers
35 views

Higher Variance due to Overfitting.

This is a homework question. Consider the following two regression models $$\mathbf y=\mathbf X_1\mathbf {\beta_1} + \mathbf {\varepsilon}$$ $$\mathbf y=\mathbf X_1\mathbf {\beta_1} + \mathbf X_2\...
0
votes
1answer
34 views

Best -least bad- regression for $ (x,y) $ points where $ y=0$ or $y=1 $.

I have a series of $(x,y)$ points where: $ 0 < x < 1 $ and $y=0$ or $y=1$ I want to aproximate $y$ values from a given $x$. I know this is a case where ...
0
votes
0answers
29 views

Fitting a cubic spline model with two knots in $R$

We are given data from a set of data from an example given by: battery voltage drop in a guided missile motor observed over the time of missile flight. The set of data has three columns of data: ...
0
votes
0answers
36 views

Regression Analysis book recommendation?

I'm having a hard time understanding the lectures of the Regression Analysis course. Any suggestions about good Analysis textbook ? Preferably to be a easy to follow book, with detailed explanations ...
1
vote
0answers
42 views

Proof Οf Τhe Variance Οf $\hat y_{x_0}$

EXERCISE Show in a simple linear regression model that the variance of $\hat y_{x_0}=\hat b_0+\hat b_1x$ is: $$V(\hat y_{x_0})=σ^2 \dfrac {\sum_{i=1}^{n} x_i^2}{n \cdot \sum_{i=1}^{n} (x_i-\bar ...
0
votes
1answer
18 views

Regression - comparing two models made for two groups.

I used logistic regression to model a binary dependent variable (depression diagnosis) while age (continous variable) and education level (categorical variable) were independent variables. I have ...
1
vote
0answers
31 views

Showing that $\operatorname{cov}(y_{x_0}, \hat{y_{x_0}}) = 0$

Exercise : For the Simple Linear Model $\mathbb E[y_x] = b_0 + b_1x$, prove that for a newly given $x_0$ and $y_{x_0}$ a new observation while $\hat{y_{x_0}}$ its point estimate, it is : $$\...
0
votes
1answer
132 views

How to prove that $\text{cov}(y_i,\hat{y_i}) = \sigma^2 h_{ii}$

In one part of our notes (and some books), the following expression is used : For the generalized linear model $y=X\beta + \epsilon$, it is : $$\text{cov}(y_i,\hat{y_i})=\sigma^2h_{ii}$$ ...
0
votes
0answers
17 views

Determining confidence intervals and confidence band for a basic polynomial regression problem

I'm working through Elements of Statistical Learning. I'm trying Exercise 3.2 (page 94 in the textbook hardcopy / page 113 in the textbook PDF), a polynomial regression problem which asks me to: make ...
1
vote
0answers
52 views

Prove that $\hat\sigma^{2}$ is an unbiased estimator for $\sigma^{2}.$

Prove that $\hat\sigma^{2}$ is an unbiased estimator for $\sigma^{2}.$ Hint: You can use the result $E(x'Ax)=tr(A\sum)+\theta'A\theta$, where $E(x)=\theta$ and $V(x)=\sum.$ We want $E(\hat\sigma^2)=\...
2
votes
3answers
62 views

Non-linear regression problem

A physical phenomenon I'm studying obeys the simple law (harmonic oscillator): $$\theta=\theta_0\cos(\omega t),$$ where $\theta$ is an angle, $\theta_0$ the amplitude of the oscillation, $\omega$ ...
0
votes
0answers
7 views

Least Angle Regression derivation for two covariates case

LARS builds up estimates $\hat µ = X\beta$, in successive steps, each step adding one covariate to the model. In case of only $2$ covariates the current correlations $c(\hat µ) = X^T(y-\hat µ)$ ...
1
vote
2answers
27 views

Showing that $\hat \beta_1 = S_{xy}/S_{xx}$ for a simple linear regression

To show $\hat \beta_1 = S_{xy}/S_{xx}$ I know I can use $\mathbf{\hat\beta_{2x1} = (X'X)^{-1} X' y}$ However, when I do this problem I only get to this step and I'm unsure if I'm even taking the ...
0
votes
0answers
15 views

Choosing fit range for curve-fitting

I have some time-dependent data $\{y_1, y_2, \ldots, y_n\}$ where I know that for late times, the data becomes pretty noisy. Something I thought of doing is to keep a lower bound of my fit fixed at $...
0
votes
0answers
9 views

Smoothing algorithm for unequal variances in data

I have data in the form of a histogram with bins that each have their own error bars. I'm interested in finding a smoothing algorithm that fits the data while taking the error bars into account (e.g. ...
0
votes
1answer
48 views

Least Squares estimator of $\beta$ using matrices

Show, using matrix notation and staring with the principle of least squares, that the least squares estimator of $\beta$ is given by: $\hat\beta =$ $\frac{\sum_{i=0}^nx_iy_i}{\sum_{i=0}^n x_i^2}$ I'...