Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

0
votes
1answer
12 views

Independence between error and regressor

Let the following classical linear regression: $$y_i = x_i \theta + u_i, \quad E(u_i|x_i) \sim N(0, \sigma^2)$$ Can I conclude that $x$ and $u$ are independent? I would like this because I want to ...
0
votes
1answer
18 views

Forming ANOVA table without observed values.

Need some help in forming the ANOVA table without observed values. I only managed to find SST with the first line of hint, but do not know how to proceed to find SSTreatment or SSError. Any hint/...
1
vote
0answers
24 views

Why is this inequality true in this linear regression problem?

From Understanding Machine Learning: Theory and Algorithms: How was the inequality in the red box below derived? For reference in this learning problem, $L_{D_i}(w) \equiv \Bbb E_{D_i} \space l(w, (...
0
votes
0answers
30 views

Confidence interval for $ \beta $ given only $\hat{\alpha}$ and $\hat{\beta}$ in linear regression

Assuming that there is a correlation between the random variables X and Y, the line of regression was estimated based on 30 values $ (x_{i}, y_{i}) $(where 200 $\leq$ x $\leq$ 250 ) $: $$ \hat{y} = ...
0
votes
1answer
19 views

Relationship between the standard error of the estimators and the standard error of the error term [closed]

I'm trying to solve the following problem: My thinking is that If the Standard error of the error term would then the error term of the regression would also fall (as the regression would start ...
2
votes
2answers
31 views

Relationship between OLS estimates of slope coefficients of simple linear regression Y on X and X on Y

Assume a model $y = \beta_0 + \beta_1x + u$. Given a sample $(x_i, y_i)_{i=1}^n$, we can find the OLS estimates of $\beta_1$, $\hat{\beta_1}$. Then suppose that we assume another model $x = \gamma_0 + ...
0
votes
1answer
25 views

Calculating the var(β) in a least square regression model

The linear model that I'm working with is: $$y_t =α +βx_t + ε_t$$ Based on my Lecture I have: $$Var(\hatβ) = Var(Σw_tε_t)$$ where $ε_t$ is the error term and $$w_t = \frac{x_t-\overline x}{Σ(x_t-\...
0
votes
0answers
18 views

Derive the expression for $ \operatorname{Var}(b_1)$ for the no-intercept linear model y = $b_1$*$x_i$ + $ε_i$

Note: $b_1$ is the estimate for $β_1$ What I tried: $ \operatorname{var}(b_1) = \operatorname{var}( Σ(x_iy_i)/Σx_i^2)$ $ \operatorname{var}(b_1)= (1/(Σx_i^2)^2) * Σx_i * \operatorname{var}(y_i)$ ...
1
vote
0answers
18 views

How to determine where the data is not linear anymore

I have some data. When I plot these data, the graph looks very linear when $x>x_0$ (In the following figure $x_0$ is shown by dashed lines). One way to determine the exact location where the graph ...
0
votes
0answers
18 views

Spurious Regression and Co-integration

I downloaded a ppt file from Spurious Regression and Co-integration On page 3 it says: "In general, regression models for non-stationary variables give spurious results. Only exception is if the ...
6
votes
0answers
71 views

Prime number intercept

Suppose I arrange my (infinite) list of prime numbers in the following way: \begin{array}{c|c}x_i&2&5&11&17&23&31&\cdots\\\hline y_i&3&7&13&19&29&37&...
0
votes
1answer
48 views

Cook's Distance

I have a problem with calculating Cook Distance (I'm trying to understand it). Ok so here is the task and my 'solution'. I'm asking for comment, is it ok, or what do I wrong. We have simple linear ...
0
votes
0answers
76 views

Linear Regression's Expectation of Prediction Error on a given point in the test set

I'm self-learning the book "The Elements of Statistical Learning", and I've got a little problem on deriving equation 2.27 in this book. I would appreciate if anyone could help me with this. In order ...
0
votes
0answers
46 views

What does this absolute value notation mean?

In a regression model function, What does this absolute value notation mean?
0
votes
0answers
14 views

Minimum-volume confidence ellipsoid for regression with nuisance parameters

I have a linear regression problem with Gaussian errors, nuisance variables and the parameter of interest, $\alpha$. I want to find the smallest-volume region containing the true value of $\alpha$ ...
0
votes
0answers
6 views

Derivation for marginal effect sizes' distribution in linear model

Model: $$Y_{N\times 1} = X_{N\times M} \beta_{M\times 1} + \epsilon_{N\times 1}$$ the design matrix $X_{N\times M}$ is known, each column of the design matrix has been standardized to have mean 0 and ...
0
votes
0answers
44 views

Ols estimator with the errors following a bernoulli distribution

I am having trouble understanding how i should approach the following problem: Given 𝑦𝑖 = 𝛼 + 𝛽𝑥𝑖 + 𝜀𝑖 𝑖 = 1, … , N with 𝜀𝑖 𝑖 = 1,2 … , N being a succession of IID Bernoulli ...
0
votes
2answers
78 views

Derivation of Linear Regression using Normal Equations

I was going through Andrew Ng's course on ML and had a doubt regarding one of the steps while deriving the solution for linear regression using normal equations. Normal equation: $\theta=(X^TX)^{-1}X^...
1
vote
0answers
24 views

Linear Regression: value for slope

I'm sorry, I'm new to linear regression so I had a very stupid question. The slope of the best-fit line is defined as the value that minimizes the sum of the squared deviations from each point. Is ...
1
vote
1answer
101 views

Conditional Expectation Decomposition in Regression Analysis

I am currently working on my understanding of regression fundamentals and I checked this source (one can find the (even exact) same statement in multiple sources). In Theorem 3.1.1, the author claims ...
0
votes
0answers
17 views

How to reduce credible sets in over-specified linear regression while maintaining global coverage probability?

Vectors $A, B$ and covariance matrix $C$ are fixed and known. I have a vector of measurements, $Y\in\mathbb{R}^n$, sampled from $$ M_1: Y \sim N(A\alpha_* + B\beta_*, C) $$ My goal, roughly speaking,...
1
vote
0answers
38 views

Linear regression of basis functions for multivariate inputs

Background - My current understanding of linear regression of basis functions: Given an input domain $\mathcal{X}$, target domain $\mathcal{Y}$, and a data set $S=\left\{ \left(x_{i},y_{i}\right)\...
0
votes
0answers
25 views

What is the formula of the linear regression with an error propagation

I am in Physics Licenciature and a day the teacher showed me a formula for the linear regression with error propagation, and time after, I was searching this formula and I didn't find it. Then I am ...
0
votes
1answer
29 views

What's the Most Appropriate Type of Regression for this Problem?

I have a data set from two groups: firms that use AI and their costs and firms that don't use AI and their costs. Within both groups I have data about their specific costs, e.g. fixed costs, variable ...
0
votes
0answers
13 views

Linear regression with dependent variables: express prediction with dot products

When dealing with Linear regressin with dependent variables, one can consider the optimization problem: $$\arg \min_w 0.5\lVert{w}\rVert^2 \\ s.t. Xw=y$$ Where $X\in\mathbb{R}^{n,d}$ is the data ...
0
votes
0answers
39 views

ESL - Linear Model where output is a vector. Ch 2 Pg 11

In the terminology used in ESL, a vector is a column vector. Let output be a $k$-vector, i.e.$$Y=(Y_1,Y_2,\cdots,Y_K)^T$$Now please refer to following line on pg 12. In general $\hat{Y}$ can be a $K$-...
0
votes
1answer
52 views

Linear regression analysis where the fit values must be greater than the observed values?

Long story short, I would like to efficiently:Minimize ||bX-y||2 subject to X ≥ 0 and bX ≥ y I have an observation that is a single curve (y) in the form of signal intensity vs. frequency. I ...
0
votes
1answer
66 views

Equivalent formulation of LASSO?

I am currently trying to tell wheter or not those two problems are equivalent : $$\min_x \|x\|_1 \text { s.t. } \|Ax-y\|^2_2 \le \varepsilon.$$ And $$\min_x \|Ax-y\|^2_2 \text { s.t. } \|x\|_1\le ...
1
vote
1answer
26 views

How to approximate prediction interval in linear regression

Suppose we have a linear regression model of the following format : $$ y(x) = \beta_0 + \beta_1 x_1+ \beta_2x_2+\beta_3x_3+\epsilon$$ We know that the prediction interval associated with a level $\...
1
vote
1answer
37 views

Linear regression model with 2 categorical variables

Let's consider the following problem : We want to predict a variable $y$ and we have two categorical variables : $A$ that can take 3 different values and $B$ than can take 2 different values. A ...
1
vote
1answer
42 views

How to interpret regression equation?

I am trying to understand how to interpret the regression line given: $y = -5.18 + 1.94x$ (regression line) where $y$ is number of cold drinks sold and where $x$ is temperature Interpret values of $...
0
votes
0answers
25 views

Total Least Square fitting

Say I want to fit a straight line using Total Least Square (as opposed to Least Square), which is to minimize the sum of (yi-k*xi-b)^2/(k^2+1) over all xi's and yi's, where xi's and yi's are training ...
0
votes
0answers
30 views

Two dimensional Linear Regression Hat Matrix

Let X=(X1, X2)nxp where X1 (nxq) with rank=q and X2 (nx(p-q)) with rank=(p-q). Let H and H1 be hat matrix of X and X1. 1) Prove that HH1=H1 and H1H=H1 2) Prove that (H-H1) is idempotent
1
vote
1answer
24 views

What is J in while calculating SST in multiple regression?

I am little confused what actually is the J in the formula of the SST and SSR for multiple regression SST= $Y^T\left[ 1-\frac{1}{n}J\right]Y$ SSR=$Y^T\left[ H-\frac{1}{n}J\right]Y$
1
vote
0answers
39 views

Elliptical Confidence Set Calculation

I got stuck in a homework question: In Linear regression model with assumption $\varepsilon_{i} \sim \cal{N}(0, \sigma^{2})$, iid. $$Y_{i} = X_{i}^{\intercal}\theta^{*} + \varepsilon_{i}, ~ i = 1, \...
0
votes
0answers
15 views

Why residuals are a good estimator of random disturbance?

Let a linear OLS model: $$Y= X \beta + u$$ Where $u$ is a random disturbance. If we define the residual of the regression as $$e = Y - X \widehat{\beta}$$ where $\widehat{\beta}$ is the OLS vector of ...
0
votes
0answers
23 views

Standard Error Estimate for Beta Coefficients

Suppose I have a linear regression model consisting of $\beta$ estimates, relative to a reference term. Each of these $\beta$s has a $\bar{x}$, a $s_x$, and then $n$, with a calculated $\hat{\sigma}$ ...
0
votes
0answers
11 views

Find $V(\tilde\beta|X)$

In the linear regression $Y=X\beta+\epsilon$, with $E(\epsilon_i|x_i)=0$, it is known that the true $\beta$ satisfies the restriction $M\beta=0$, where $M$ is a $q \times k$ matrix with $q<k$. $\...
0
votes
1answer
25 views

Please correct my thinking about Ridge Regression

If ridge regression biases ALL beta coefficients of a regression model towards zero, wouldn't the model massively mispredict the y-variable? I know my logic must be wrong here, but I'd appreciate if ...
0
votes
0answers
15 views

Error term too big/can in improve my multiple regression?

I have made regression selecting the best features (p-val<0.05). I now have the model with a 0.85 R-squared and residual of 2.64. Well, sometimes when I try to predict some new instances, my ...
-3
votes
1answer
27 views

What is the t-score for the relationship between X & Y

The police chief is concerned about alarm systems degrading in city buildings and failing to operate. Using a sample of 100 alarms in operation the previous year, the police department regresses ...
0
votes
0answers
13 views

Multiple regression - positive/negative linear relationship

Is there a way other than from the x values to tell from the regression printout whether there is an evidence of a negative or positive linear relationship? Would the values in the Coefficients ...
1
vote
1answer
147 views

L2-norm with estimated weights

Suppose I'm performing linear regression. My lecturer said the formula below can be used for estimating the weight vector that is passed to the L2-norm part of the loss function but he didn't ...
1
vote
0answers
53 views

What does it essentially mean if the neural network has convex error surface?

Suppose if I am building a Linear Regression model with one fully connected layer and a sigmoid with minimizing mean squared error as objective. I understand that this network has a convex error ...
1
vote
0answers
37 views

Linear Regression Diagnostics

I am trying to determine if there is a relationship between a dependent variable y and independent variable x by fitting a least squares regression model. Scatterplot of data: Diagnostic plots: ...
1
vote
0answers
80 views

Proving that $\mathbf{(H-\frac{1}{n}J_n)}$ is indempotent

I am trying to show that the matrix $\mathbf{(H-\frac{1}{n}J_n)}$ is idempotent where $\mathbf{H}$ is the Hat-matrix (Projection matrix) of linear regression and $J_n$ is the $n\times n$ matrix with $...
0
votes
0answers
28 views

basic, clear Book about linear regression with examples?

i'm taking a course about linear models , and the book used is Ravishanker "a first course of linear models", the problem is that the book its so theoretical and my background is so basic, so is very ...
0
votes
0answers
14 views

how affect the variance and the expected value add a variable in a linear model

if the correct model is $Y=X_{1}\beta_{1} + X_{2}\beta_{2} + \varepsilon$, with variance $Var(Y)=\sigma^{2}I$.how change the variance and the expected value of $\beta^{*}_{1}$, in $Y=X_{1}\beta^{*}_{1}...
0
votes
0answers
19 views

Variance vs. asymptotic variance of OLS estimators?

I immediately assumed that all three were true. Which one is false, and why?
1
vote
0answers
15 views

Confidence interval of dependant variable individual value by R-squared value

From the physical experiment I've got a sample collection of values (unique Y for each unique X). The problem is to find the ...