Questions tagged [linear-regression]
For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.
670
questions
1
vote
2answers
17 views
Is there a direct interpretation of the least squares solution based on composition of linear operators?
The least squares solution to the problem
$$\min_x \|Ax - b\|_2^2$$
is $x = (A^\top A)^{-1}A^\top b$.
Is there an interpretation of this solution by directly interpretating $x$ as the output of a ...
0
votes
0answers
4 views
What are the properties of data we have to have in order to do least square regression with just matrix multiplication?
In the problem of linear regression, we are given $n$ observations $\{ (x_1, y_1),\dots,(x_n, y_n)\}$, where each input $x_i$ is a $d$-dimensional vector. Our goal is to estimate a linear predictor $f(...
0
votes
0answers
3 views
Whats the difference between B0 B1 and b0 b1 in the regression model
I understand that b0 and b1 are coefficients one gets for the regression model and B0 and B1 are parameters. Im not quite sure what the difference between these two variables are. Could i get an ...
0
votes
1answer
15 views
Question on predicting values from linear regression
I have this set of values and Table 1 shows the average driving distance (in yards) for players on the men's (PGA) and women's (LPGA) professional golf tours from 1992 through 2003.
I need to answer ...
0
votes
1answer
16 views
Why is the decision making for a hypothesis test opposite when testing for a slope and a mean?
When conducting a hypothesis test for a sample mean, we reject the null hypothesis when the test statistic is less than the critical value, or the P-value is less than the rejection region. Why is ...
0
votes
0answers
8 views
Total derivative of linear regression
I am trying to reconcile how the error term in linear regression behaves upon an infitismal change.
$$ y = \alpha + \beta x + \varepsilon $$
$$ \Rightarrow y - \alpha + \beta x + \varepsilon = 0 $$
...
0
votes
1answer
27 views
Least squares estimator for different model
I have learned in my class that for the usual simple linear regression model $Y_i=\beta_0 + \beta_1x_i+\epsilon_i$, the estimators are $b_1 = \frac{S_{xy}}{S_{xx}}$ and $b_0 = \bar{y}-b_1\bar{x}$.
...
0
votes
0answers
18 views
Convergence rate of linear regression without chain rule
To minimize an MSE, a common method is to perform a gradient descent on the objective. For example, the derivative is:
$\frac{d}{dw} \sum_{i=1}^n (t_i - w x_i)^2 = \sum_{i=1}^n 2 (t_i - w x_i) x_i$.
...
2
votes
2answers
61 views
Linear regression model with a known parameter
Question:
Consider the following linear regression with one parameter (intercept $\beta_{0}$ known).
$ y_{i} = \beta_{0}^{*} + \beta_{1}x_{i} + \epsilon_{i} $ for i = 1,....n
a) Compute the LSE , ...
0
votes
0answers
16 views
Statistic. Bayesian Linear Regression.
I am trying to solve the following problem.
If $y | \beta \sim N(X \beta, I_n)$ and $\beta \sim N(0, g^{-1}(X^t X)^{-1})$ for $g>0$. Find $ \pi(\beta|y)$ and show that $E(\beta|y)$ is a function ...
2
votes
0answers
21 views
Find X matrix in the Linear Model
If $y=Xb+e$ is a (General) linear model with n=p=3
Given that $b_3$ is non estimable, and $b_1+ 2b_2 + 2b_3$ is estimable
.
How can I find an X which is consistent with the above.
I ...
0
votes
0answers
6 views
How to prioritize predictions on K nearest neighbors
I found multiple observations with the same (smallest) Euclidean distance, but now I don't know which one to pick for K=1. Anyone knows how would that work?
0
votes
1answer
16 views
Is this a typo in this econometric exercise?
I'm trying to solve an exercise in OLS estimator:
I'm not sure if $\mathrm{E}(u | \boldsymbol{x}, q)=0$ is a typo. Moreover, $\mathrm{E}(u | \boldsymbol{x})=0$ is used in the solution. Should it ...
0
votes
0answers
8 views
comparing effect of dummies to continuous explanatory variables in linear regression
I have a 30 explanatory variables, among which are some dummy variables. The dependent variable is in log format and so are almost all the explanatory variables. So my model looks like:
...
0
votes
2answers
30 views
Differentiating $RSS(\beta) = (y-X\beta)^T(y-X\beta)$ with respect to $\beta$
Suppose I have a vector $y$ of dimension $N \times 1$, and a matrix $X$ of dimension $N \times p$ and a vector $\beta$ of dimension $p \times 1$. Then I wish to differentiate the matrix equation :
$...
1
vote
1answer
23 views
How does it follow that $\operatorname{Var} (g({X}) {\epsilon} | {X}) = (g({X}) )\operatorname{Var} ( {\epsilon} | {X}) (g({X}) )^{\prime}$?
Good morning, I'm reading lecture slides bout the BLUE properties of OLS estimator.
Conditional unbiasedness
Conditional variance
My question:
I have two equalities from the two slides:
$$...
0
votes
0answers
18 views
Question regarding linear regression weighting matrix
Consider the linear regression model
$$b = Xy + e, \quad E[e] = 0, \quad E[ee'] = V$$
Assume that the matrix $X$ has linearly independent columns. It is well known that the minimum variance affine ...
0
votes
0answers
8 views
How does Locally Weighted Regression work for test sets far outside the train set bound
I was following CS229 machine learning course where I came across the Locally Weighted Regression algorithm.
We have to minimize
$\sum\limits_{i}w^i(y^i - \theta^T x^i)$
and output $\theta^Tx$ ...
0
votes
1answer
28 views
Prove this formula about residuals in case there is intercept in the OLS estimator
I'm learning OLS estimator with difficulty with computing the $R^2$. First are the notations used in my lecture note:
$X_{i} \equiv\left(\begin{array}{c}{X_{i 1}} \\ {X_{i 2}} \\ {\vdots} \\ {X_{i K}}...
0
votes
0answers
13 views
Prove this formula for $R^2$ in the case there is one explanatory variable in the OLS estimator
I'm learning OLS estimator with difficulty with computing the $R^2$. First are the notations used in my lecture note:
$X_{i} \equiv\left(\begin{array}{c}{X_{i 1}} \\ {X_{i 2}} \\ {\vdots} \\ {X_{i K}}...
0
votes
1answer
16 views
Rearrangement of summation definition of variance
I'm following a proof of simple linear regression as detailed in Chapter 24 of The Probability Lifesaver by Stephen J Miller. There's one step involving the variance that the author explains as "...
1
vote
1answer
12 views
Why use residual plots in linear regression for assessing normality?
Let's take the case of condition simple linear regression for example where we are assuming:
$$Y|X=x = \beta_0 + \beta_1 x + \epsilon,$$
where $\epsilon$ represent the random noise.
In order to ...
0
votes
0answers
10 views
How to compare variable importance between continuous var and categorical var in linear regression?
I know the concept of standardized coefficients, which should be applied to continuous variable only and it is used for comparing variable importance among continuous variables. But what about ...
0
votes
0answers
31 views
How to prove $(X^TX)^{-1}$ has rank p where $X \in R^{n \times p}$ and has $Rank(X) = p$
I was reading Least square solution of Linear Regression. Following is the error function:
\begin{equation}
J(\theta) = \frac{1}{2}\sum_{i=1}^{n}{\vert{X\theta-y}\vert}^2
\end{equation}
where $\...
0
votes
0answers
11 views
partitioned regression problem
I am currently studying linear regression, and I'm stuck with the below problem. I would appreciate very much if someone solves the below problem.
Consider the multiple regression model:
$ y = X_1\...
0
votes
0answers
34 views
Find Cov$\big(\hat{\beta_0},\hat{\beta_1}\big)$.
The least squares estimator of $\beta_0$ is given as $\hat{\beta_0}=\overline{Y}-\hat{\beta_1}\overline{X}$. Find Cov
$\big(\hat{\beta_0},\hat{\beta_1}\big)$. When is Cov $\big(\hat{\beta_0},\hat{\...
0
votes
1answer
19 views
rank of a residual matrix
Hi I am currently studying linear regression. My question is
Is it true that
$rank(M)=n-k$ if $M = I_n- X(X'X)^{-1}X'$ where M is (n x k) matrix and $rank(M)=k$.
I can't solve it. Thanks for your ...
0
votes
0answers
17 views
How to make non-linear regression equation with 4 variables?
If we use only X and y, we can write y = mx + c equation for linear regression.
But now I have another 2 variables. So now I have 4 variables. Let's think "x, y, z, w" as my 4 variables. How can I ...
0
votes
0answers
18 views
Multiple Linear Regression, Correlation Coefficients
Show that the value of ry_hat,y = ( ry,x1 + ry,x2 +…+ ry,xp)^1/2 if the predictors are pair-wise uncorrelated and rx,y represents the correlation coefficient of x and y.
y_hat is the collection of ...
1
vote
0answers
20 views
Regression for single input, multiple output
I have an equation:
Y = R+B+G, where, R, B and G are the independent variables and Y is the dependent variable.
I trained a regression based model using tensorflow, that could guess the function and ...
0
votes
1answer
28 views
Invertibility of a matrix that arises from least squares estimation
Consider a linear regression
$$
y=X_1\beta_1+X_2\beta_2+u
$$
to be estimated by least squares. Here, $y$, $X_1$, and $X_2$ are $n\times 1$, $n\times k_1$ and $n\times k_2$. Let $X=(X_1,X_2)$ to be $n\...
0
votes
0answers
19 views
Linear Regression Model Proof Question.
I have several proof questions for my class assignment.
Suppose that we have the liner regression model: $y = X \beta x + \epsilon$. Let $P = X(X'X)^{-1}X'$ and $M = I - P$. Show that $MX = 0$ and $...
1
vote
1answer
30 views
Variable and its dynamics in one multiple regression model
I am trying to find the dependence between default rates in bank and macroeconomic variables with linear regression.
To do so I created a code which estimates every possible model - every combination ...
0
votes
1answer
21 views
What's the optimal parameter for vector B given x and y
I need to brush up my Maths, for sure, but, meanwhile I hope you could help me with this.
This is a Linear Regression problem.
Question: given X and Y compute the optimale B (from what I understand, ...
0
votes
1answer
26 views
How can I show that $ \frac{\sum\limits_i(Y_i-\hat{Y}_i)}{\sigma^2(n-2)}\sim \frac{\chi^2_{(n-2)}}{n-2} $ for simple linear regression.
I am working on simple linear regression with the normal error assumption $Y_i=\beta_0+\beta_1 X_i+\epsilon_i$, $\epsilon_i \sim N(0,\sigma^2)$ where the estimated mean response is $\hat{Y}_i=b_0+b_1 ...
0
votes
0answers
15 views
Least squares coefficient estimate equation given in the book An Introduction to statistical learning
The book "An Introduction to Statistical Learning" states that
Let $\hat y_i $ = $\hat \beta_0 + \hat\beta_1x_i $ be the prediction for Y based on the ith value of X. Then $ e_i = y_i - \hat y_i ...
2
votes
0answers
55 views
The fitted values of a simple linear regression are linear combinations of the observed
I have notes which say that the ith fitted value $\hat{Y}_i$ is a linear combination of the response values
$$\hat{Y}_i = \sum_{j=1}h_{ij}Y_j$$
where
$$h_{ij} = \frac 1 n + \frac{(x_i-\overline{x}...
0
votes
0answers
8 views
Notation for Dyadic Hierarchical Partitions of $[0, 1)$
For a presentation I'm giving, I need to explain how tree-based regression works.
I'm not $100\%$ sure of the notation I'm using, and whether it's consistent.
So the basic idea is that we write the ...
0
votes
0answers
21 views
Mistakes in calculating SST SSR SSE
Lets say my three measured data was $y= 2,3,4$ for $x= 1,2,3.$
I want to calculate R-square for checking how well the observed data fits to my model, y=x. (or how well my model explains the data.)
...
0
votes
2answers
29 views
Least square estimators for “no slope” and “no intercept” models.
I'm trying to understand simple linear regression. Here is a problem I'm working on, and I'm trying to understand the answers conceptually.
I want to find the least square estimators $b_1$ and $b_0$ ...
0
votes
1answer
55 views
Prove that $ \Phi(\Phi^T\Phi)^{-1}\Phi^T=I $ if $\Phi$ has more columns than rows
In linear regression, for a data set $\bar t$, the least-squares solution of the equation $\bar t = \Phi\bar w$ is $$\hat{\bar w} = (\Phi^T\Phi)^{-1}\Phi^T\bar t$$ where $\Phi$ is the design matrix ...
1
vote
0answers
17 views
Bayesian Linear Regression Conjugate Prior
The model is linear regression, that is
$$Y=X\beta+\epsilon $$
where $Y$ is the vector of response variable, $X$ is the design matrix, $\beta$ is the vector of regression coefficients and $\epsilon$ ...
1
vote
0answers
21 views
Least squares for an errors-in-variables model
We estimate parameter $\theta$ based on the measurements of two points $(X_1, Y_1)$ and $(X_2, Y_2)$ for the following linear model:
$$\begin{cases} \hat{X_1} \theta = \hat{Y_1} \, ,\\ \hat{X_2} \...
1
vote
0answers
23 views
Deriving chi-squared distribution in simple linear regression
Take the linear regression model $y_i=ax_i+b + \varepsilon_i, 1\leq i\leq n$.
Let the errors $\varepsilon_i$ follow a normal distribution with mean $0$ and constant variance $\sigma^2$. Let $\hat{a}$...
1
vote
1answer
30 views
Determining the method of calculating p-values for regression
I would like to know any kind of method I can use to find out which distribution was used to calculate the p-values in this regression output table, thanks.
1
vote
1answer
32 views
Output table linear regression
I am unsure of how you would get the missing values denoted "***" in the table using only the values listed in the table. Any help would be greatly appreciated, thanks.
Output table of linear ...
2
votes
1answer
110 views
On confidence sets and linear regression
I'm working on Exercise 3.2 from Elements of Statistical Learning. It asks to find a $95\%$ confidence interval for a linear regression prediction (ordinary least squares are used) using two different ...
0
votes
0answers
38 views
For the model $y = \beta_0 + \beta_1 x + e$, does $\beta_1$ reflect the causal effect of $x$ on $y$?
Appraise the statement: "For the model $y = \beta_0 + \beta_1 x + e$, $\beta_1$ reflects the causal effect of $x$ on $y$."
This is a question posed for homework. This is about linear regression. I ...
1
vote
0answers
51 views
How to define a performance measure for linear regression with some variable errors?
This is actually an engineering problem. I have a linear regression of the following form:
$$
Y = X(p(t),\dot p(t), q) \beta,
$$
where the regressor $X$ is a nonlinear function of $p$, $\dot p(t) = \...
3
votes
1answer
83 views
How the α affects the solution path in the elastic net?
The following Quiz is the rough translation (with minor modifications) of Quiz No.10-2 of the exam of the "2018's semi-first grade of Japan Statistical Society Certificate" (see Ref (1)) ". According ...
