Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

Filter by
Sorted by
Tagged with
1
vote
2answers
17 views

Is there a direct interpretation of the least squares solution based on composition of linear operators?

The least squares solution to the problem $$\min_x \|Ax - b\|_2^2$$ is $x = (A^\top A)^{-1}A^\top b$. Is there an interpretation of this solution by directly interpretating $x$ as the output of a ...
0
votes
0answers
4 views

What are the properties of data we have to have in order to do least square regression with just matrix multiplication?

In the problem of linear regression, we are given $n$ observations $\{ (x_1, y_1),\dots,(x_n, y_n)\}$, where each input $x_i$ is a $d$-dimensional vector. Our goal is to estimate a linear predictor $f(...
0
votes
0answers
3 views

Whats the difference between B0 B1 and b0 b1 in the regression model

I understand that b0 and b1 are coefficients one gets for the regression model and B0 and B1 are parameters. Im not quite sure what the difference between these two variables are. Could i get an ...
0
votes
1answer
15 views

Question on predicting values from linear regression

I have this set of values and Table 1 shows the average driving distance (in yards) for players on the men's (PGA) and women's (LPGA) professional golf tours from 1992 through 2003. I need to answer ...
0
votes
1answer
16 views

Why is the decision making for a hypothesis test opposite when testing for a slope and a mean?

When conducting a hypothesis test for a sample mean, we reject the null hypothesis when the test statistic is less than the critical value, or the P-value is less than the rejection region. Why is ...
0
votes
0answers
8 views

Total derivative of linear regression

I am trying to reconcile how the error term in linear regression behaves upon an infitismal change. $$ y = \alpha + \beta x + \varepsilon $$ $$ \Rightarrow y - \alpha + \beta x + \varepsilon = 0 $$ ...
0
votes
1answer
27 views

Least squares estimator for different model

I have learned in my class that for the usual simple linear regression model $Y_i=\beta_0 + \beta_1x_i+\epsilon_i$, the estimators are $b_1 = \frac{S_{xy}}{S_{xx}}$ and $b_0 = \bar{y}-b_1\bar{x}$. ...
0
votes
0answers
18 views

Convergence rate of linear regression without chain rule

To minimize an MSE, a common method is to perform a gradient descent on the objective. For example, the derivative is: $\frac{d}{dw} \sum_{i=1}^n (t_i - w x_i)^2 = \sum_{i=1}^n 2 (t_i - w x_i) x_i$. ...
2
votes
2answers
61 views

Linear regression model with a known parameter

Question: Consider the following linear regression with one parameter (intercept $\beta_{0}$ known). $ y_{i} = \beta_{0}^{*} + \beta_{1}x_{i} + \epsilon_{i} $ for i = 1,....n a) Compute the LSE , ...
0
votes
0answers
16 views

Statistic. Bayesian Linear Regression.

I am trying to solve the following problem. If $y | \beta \sim N(X \beta, I_n)$ and $\beta \sim N(0, g^{-1}(X^t X)^{-1})$ for $g>0$. Find $ \pi(\beta|y)$ and show that $E(\beta|y)$ is a function ...
2
votes
0answers
21 views

Find X matrix in the Linear Model

If $y=Xb+e$ is a (General) linear model with n=p=3 Given that $b_3$ is non estimable, and $b_1+ 2b_2 + 2b_3$ is estimable . How can I find an X which is consistent with the above. I ...
0
votes
0answers
6 views

How to prioritize predictions on K nearest neighbors

I found multiple observations with the same (smallest) Euclidean distance, but now I don't know which one to pick for K=1. Anyone knows how would that work?
0
votes
1answer
16 views

Is this a typo in this econometric exercise?

I'm trying to solve an exercise in OLS estimator: I'm not sure if $\mathrm{E}(u | \boldsymbol{x}, q)=0$ is a typo. Moreover, $\mathrm{E}(u | \boldsymbol{x})=0$ is used in the solution. Should it ...
0
votes
0answers
8 views

comparing effect of dummies to continuous explanatory variables in linear regression

I have a 30 explanatory variables, among which are some dummy variables. The dependent variable is in log format and so are almost all the explanatory variables. So my model looks like: ...
0
votes
2answers
30 views

Differentiating $RSS(\beta) = (y-X\beta)^T(y-X\beta)$ with respect to $\beta$

Suppose I have a vector $y$ of dimension $N \times 1$, and a matrix $X$ of dimension $N \times p$ and a vector $\beta$ of dimension $p \times 1$. Then I wish to differentiate the matrix equation : $...
1
vote
1answer
23 views

How does it follow that $\operatorname{Var} (g({X}) {\epsilon} | {X}) = (g({X}) )\operatorname{Var} ( {\epsilon} | {X}) (g({X}) )^{\prime}$?

Good morning, I'm reading lecture slides bout the BLUE properties of OLS estimator. Conditional unbiasedness Conditional variance My question: I have two equalities from the two slides: $$...
0
votes
0answers
18 views

Question regarding linear regression weighting matrix

Consider the linear regression model $$b = Xy + e, \quad E[e] = 0, \quad E[ee'] = V$$ Assume that the matrix $X$ has linearly independent columns. It is well known that the minimum variance affine ...
0
votes
0answers
8 views

How does Locally Weighted Regression work for test sets far outside the train set bound

I was following CS229 machine learning course where I came across the Locally Weighted Regression algorithm. We have to minimize $\sum\limits_{i}w^i(y^i - \theta^T x^i)$ and output $\theta^Tx$ ...
0
votes
1answer
28 views

Prove this formula about residuals in case there is intercept in the OLS estimator

I'm learning OLS estimator with difficulty with computing the $R^2$. First are the notations used in my lecture note: $X_{i} \equiv\left(\begin{array}{c}{X_{i 1}} \\ {X_{i 2}} \\ {\vdots} \\ {X_{i K}}...
0
votes
0answers
13 views

Prove this formula for $R^2$ in the case there is one explanatory variable in the OLS estimator

I'm learning OLS estimator with difficulty with computing the $R^2$. First are the notations used in my lecture note: $X_{i} \equiv\left(\begin{array}{c}{X_{i 1}} \\ {X_{i 2}} \\ {\vdots} \\ {X_{i K}}...
0
votes
1answer
16 views

Rearrangement of summation definition of variance

I'm following a proof of simple linear regression as detailed in Chapter 24 of The Probability Lifesaver by Stephen J Miller. There's one step involving the variance that the author explains as "...
1
vote
1answer
12 views

Why use residual plots in linear regression for assessing normality?

Let's take the case of condition simple linear regression for example where we are assuming: $$Y|X=x = \beta_0 + \beta_1 x + \epsilon,$$ where $\epsilon$ represent the random noise. In order to ...
0
votes
0answers
10 views

How to compare variable importance between continuous var and categorical var in linear regression?

I know the concept of standardized coefficients, which should be applied to continuous variable only and it is used for comparing variable importance among continuous variables. But what about ...
0
votes
0answers
31 views

How to prove $(X^TX)^{-1}$ has rank p where $X \in R^{n \times p}$ and has $Rank(X) = p$

I was reading Least square solution of Linear Regression. Following is the error function: \begin{equation} J(\theta) = \frac{1}{2}\sum_{i=1}^{n}{\vert{X\theta-y}\vert}^2 \end{equation} where $\...
0
votes
0answers
11 views

partitioned regression problem

I am currently studying linear regression, and I'm stuck with the below problem. I would appreciate very much if someone solves the below problem. Consider the multiple regression model: $ y = X_1\...
0
votes
0answers
34 views

Find Cov$\big(\hat{\beta_0},\hat{\beta_1}\big)$.

The least squares estimator of $\beta_0$ is given as $\hat{\beta_0}=\overline{Y}-\hat{\beta_1}\overline{X}$. Find Cov $\big(\hat{\beta_0},\hat{\beta_1}\big)$. When is Cov $\big(\hat{\beta_0},\hat{\...
0
votes
1answer
19 views

rank of a residual matrix

Hi I am currently studying linear regression. My question is Is it true that $rank(M)=n-k$ if $M = I_n- X(X'X)^{-1}X'$ where M is (n x k) matrix and $rank(M)=k$. I can't solve it. Thanks for your ...
0
votes
0answers
17 views

How to make non-linear regression equation with 4 variables?

If we use only X and y, we can write y = mx + c equation for linear regression. But now I have another 2 variables. So now I have 4 variables. Let's think "x, y, z, w" as my 4 variables. How can I ...
0
votes
0answers
18 views

Multiple Linear Regression, Correlation Coefficients

Show that the value of ry_hat,y = ( ry,x1 + ry,x2 +…+ ry,xp)^1/2 if the predictors are pair-wise uncorrelated and rx,y represents the correlation coefficient of x and y. y_hat is the collection of ...
1
vote
0answers
20 views

Regression for single input, multiple output

I have an equation: Y = R+B+G, where, R, B and G are the independent variables and Y is the dependent variable. I trained a regression based model using tensorflow, that could guess the function and ...
0
votes
1answer
28 views

Invertibility of a matrix that arises from least squares estimation

Consider a linear regression $$ y=X_1\beta_1+X_2\beta_2+u $$ to be estimated by least squares. Here, $y$, $X_1$, and $X_2$ are $n\times 1$, $n\times k_1$ and $n\times k_2$. Let $X=(X_1,X_2)$ to be $n\...
0
votes
0answers
19 views

Linear Regression Model Proof Question.

I have several proof questions for my class assignment. Suppose that we have the liner regression model: $y = X \beta x + \epsilon$. Let $P = X(X'X)^{-1}X'$ and $M = I - P$. Show that $MX = 0$ and $...
1
vote
1answer
30 views

Variable and its dynamics in one multiple regression model

I am trying to find the dependence between default rates in bank and macroeconomic variables with linear regression. To do so I created a code which estimates every possible model - every combination ...
0
votes
1answer
21 views

What's the optimal parameter for vector B given x and y

I need to brush up my Maths, for sure, but, meanwhile I hope you could help me with this. This is a Linear Regression problem. Question: given X and Y compute the optimale B (from what I understand, ...
0
votes
1answer
26 views

How can I show that $ \frac{\sum\limits_i(Y_i-\hat{Y}_i)}{\sigma^2(n-2)}\sim \frac{\chi^2_{(n-2)}}{n-2} $ for simple linear regression.

I am working on simple linear regression with the normal error assumption $Y_i=\beta_0+\beta_1 X_i+\epsilon_i$, $\epsilon_i \sim N(0,\sigma^2)$ where the estimated mean response is $\hat{Y}_i=b_0+b_1 ...
0
votes
0answers
15 views

Least squares coefficient estimate equation given in the book An Introduction to statistical learning

The book "An Introduction to Statistical Learning" states that Let $\hat y_i $ = $\hat \beta_0 + \hat\beta_1x_i $ be the prediction for Y based on the ith value of X. Then $ e_i = y_i - \hat y_i ...
2
votes
0answers
55 views

The fitted values of a simple linear regression are linear combinations of the observed

I have notes which say that the ith fitted value $\hat{Y}_i$ is a linear combination of the response values $$\hat{Y}_i = \sum_{j=1}h_{ij}Y_j$$ where $$h_{ij} = \frac 1 n + \frac{(x_i-\overline{x}...
0
votes
0answers
8 views

Notation for Dyadic Hierarchical Partitions of $[0, 1)$

For a presentation I'm giving, I need to explain how tree-based regression works. I'm not $100\%$ sure of the notation I'm using, and whether it's consistent. So the basic idea is that we write the ...
0
votes
0answers
21 views

Mistakes in calculating SST SSR SSE

Lets say my three measured data was $y= 2,3,4$ for $x= 1,2,3.$ I want to calculate R-square for checking how well the observed data fits to my model, y=x. (or how well my model explains the data.) ...
0
votes
2answers
29 views

Least square estimators for “no slope” and “no intercept” models.

I'm trying to understand simple linear regression. Here is a problem I'm working on, and I'm trying to understand the answers conceptually. I want to find the least square estimators $b_1$ and $b_0$ ...
0
votes
1answer
55 views

Prove that $ \Phi(\Phi^T\Phi)^{-1}\Phi^T=I $ if $\Phi$ has more columns than rows

In linear regression, for a data set $\bar t$, the least-squares solution of the equation $\bar t = \Phi\bar w$ is $$\hat{\bar w} = (\Phi^T\Phi)^{-1}\Phi^T\bar t$$ where $\Phi$ is the design matrix ...
1
vote
0answers
17 views

Bayesian Linear Regression Conjugate Prior

The model is linear regression, that is $$Y=X\beta+\epsilon $$ where $Y$ is the vector of response variable, $X$ is the design matrix, $\beta$ is the vector of regression coefficients and $\epsilon$ ...
1
vote
0answers
21 views

Least squares for an errors-in-variables model

We estimate parameter $\theta$ based on the measurements of two points $(X_1, Y_1)$ and $(X_2, Y_2)$ for the following linear model: $$\begin{cases} \hat{X_1} \theta = \hat{Y_1} \, ,\\ \hat{X_2} \...
1
vote
0answers
23 views

Deriving chi-squared distribution in simple linear regression

Take the linear regression model $y_i=ax_i+b + \varepsilon_i, 1\leq i\leq n$. Let the errors $\varepsilon_i$ follow a normal distribution with mean $0$ and constant variance $\sigma^2$. Let $\hat{a}$...
1
vote
1answer
30 views

Determining the method of calculating p-values for regression

I would like to know any kind of method I can use to find out which distribution was used to calculate the p-values in this regression output table, thanks.
1
vote
1answer
32 views

Output table linear regression

I am unsure of how you would get the missing values denoted "***" in the table using only the values listed in the table. Any help would be greatly appreciated, thanks. Output table of linear ...
2
votes
1answer
110 views

On confidence sets and linear regression

I'm working on Exercise 3.2 from Elements of Statistical Learning. It asks to find a $95\%$ confidence interval for a linear regression prediction (ordinary least squares are used) using two different ...
0
votes
0answers
38 views

For the model $y = \beta_0 + \beta_1 x + e$, does $\beta_1$ reflect the causal effect of $x$ on $y$?

Appraise the statement: "For the model $y = \beta_0 + \beta_1 x + e$, $\beta_1$ reflects the causal effect of $x$ on $y$." This is a question posed for homework. This is about linear regression. I ...
1
vote
0answers
51 views

How to define a performance measure for linear regression with some variable errors?

This is actually an engineering problem. I have a linear regression of the following form: $$ Y = X(p(t),\dot p(t), q) \beta, $$ where the regressor $X$ is a nonlinear function of $p$, $\dot p(t) = \...
3
votes
1answer
83 views

How the α affects the solution path in the elastic net?

The following Quiz is the rough translation (with minor modifications) of Quiz No.10-2 of the exam of the "2018's semi-first grade of Japan Statistical Society Certificate" (see Ref (1)) ". According ...