# Tag Info

## New answers tagged regression

0

This question was with respect to linear regression in machine learning class. One of the mentors from my class (Tom Mosher) answered this: When X is the whole matrix of training examples, then h = X * theta. When x is a single training example, then h = theta' * x. Note the use of upper and lower-case letters for x and X. Thanks @martini and @MPW for ...

1

Since you haven't observed $\epsilon$ or $m$, you have the following joint distribution of $m$ and $\epsilon$: $$f(m,\epsilon) = \frac{1}{2\pi}e^{-\frac{\epsilon^2+m^2}{2}}$$ However, you've also observed $x,y$ so we know that $m,\epsilon$ took values such that the linear relationship between $y,x$ holds. To get a likelihood function for $m$, lets ...

0

"Is this your homework, Larry?" I. Look at the scatter plot And find a relation such that $y_i\approx f(x_i), i=1,...,n$ II. "You might want to watch out that front window Larry". Set a family of functions $F$ (ex: linear fonctions) and a cost function$L$ such that: $\sum_{i=1}^{n}L(y-f(x))$ is a minimum for a given function $f\in F$., wher n is the ...

0

$$Y_i=B_0+B_1X_i+\epsilon_i$$ $$\hat{Y_i}=\hat{B_0}+\hat{B_1}X_i$$ a) $$E[MSE]=E[\frac{\sum(Y_i-\hat{Y_i})^2}{n-2}]=\sigma^2=0.6^2$$ $$E[MSR]=E[\sum(\hat{Y_i}-\overline{Y})^2]=\sigma^2+B_1\sum(X_i-\overline{X})^2=1026.36$$ b) $$\sigma(\hat{B_1})=\sqrt{\frac{\sigma^2}{\sum(X_i-\overline{X})^2}}=\frac{0.6}{\sqrt{\sum(X_i-\overline{X})^2}}$$ for the case ...

1

No, since the coefficient is the slope of line which minimizes the sum of squared residuals. In that sense it cannot be interpreted as the as the mean, you should rather interpret it as the marginal impact of X on Y.

0

I tried working out the same as you using Excel. My results agree with yours up to $d=11$. From that point on we begin to differ. The reason is that the difference in size of the values becomes so great that no meaningful analysis is possible. In Excel, by the time you are trying to work with $d=11$ you have to find the 11th power of each x-value and these ...

2

Let $S$ be the space spanned by the columns of $X$. Then $H$ is an orthogonal projection onto $S$ and $I - H$ is an orthogonal projection onto $S^\perp$. Since $$y - \hat{y} = (I - H)y \in S^\perp$$ and $$\hat{y}-\bar{y} = Hy - (1/n)\mathbb{I}y$$ we have that $(y - \hat{y})^T(\hat{y}-\bar{y}) = (1/n)y^T(I-H)\mathbb{I}y$. By the definition of ...

0

The common explanation for this (I don't know if there is a better one) is that (in linear regression) error components from random variables are assumed to be linearly independent from each other, that is, they are also random and one cannot be used to estimate the other. With this assumption you will get that the expected sum of their products to have a ...

1

A point about the formula for $Y_i$: $u_i$ is the residual for case $i$ and is the difference between the observed and estimated values. (Diagnostics can be performed on the residuals to verify that a linear multiple regression is appropriate). For the estimated value, $$\tilde{y}=\beta_0+\beta_1 x_1 + \beta_2 x_2$$ and this equation is linear, so for ...

1

Y is basically just a function at this point, and X1 and X2 are its input parameters. You can then treat the changes in X1 and X2 as numerical differences. For the first part, you could say Y = B0 + B1*(X1+3) + B2*X2 (ignoring the error term, because we can never know what the error term will be). Compared to the original value, this new Y is 3*B1 larger. ...

1

The estimate $\hat Y_k$ estimates the height of the regression line at the value $X_k$ of the predictor variable. Its variance is given your second displayed equation. The predicted value $\hat Y_p = b_0 + b_1X_p$ corresponds to a new value $x_p,$ which is not part of the dataset used to find the estimated regression coefficients $b_0$ and $b_1.$ Its ...

0

The difficult part in this sort of fit is assigning the points to the two lines – such assigments can often (and I suspect also in this case) not be made analytically. You need to either apply some heuristic to classify the points (e.g. by clustering), or try out the $2^{N-1}$ partitions. You might also be able to start with some guessed partition, do ...

0

How about this? Fitting one line is easy: shift the coordinates so that the average is the origin; treating the new coordinates as complex numbers, the desired line passes through the square roots of the sum of their squares. You could then partition the sample points according to their signed distance from that line, and shift the first line by the ...

0

Here's a plot that shows two extremes: make $\sum{ (x_i - \bar{x})^2 }$ as big as possible, $x_i = \pm$ 1 (top) as small as possible, $x_i$ near 0 (bottom). Lines are fit to $x_i, y_i$ with 5 Monte Carlo runs, $y_i \sim \mathbb{N}$ . Obviously, the top lines are pretty nearly horizontal, but lines fit to [all $x_i$ near 0, $y_i$] are nonsense. Line ...

Top 50 recent answers are included