Tag Info

Hot answers tagged

2

You should not use very highly correlated variables in your model, as the result of the regression will be meaningless (Imagine regressing Y on $X_1,...,X_n$ with $X_1=...=X_n$, you're obviously losing the unicity of the coefficient). Try to perform a principal component analysis and drop the variables that have no explanatory power in sample.


2

Regression models are not created to do what you want. For example, if you had each of those 4 variables in the model like this: $$y=b0+b1*x1+b2*x2+b3*x3+b4*x4$$ then we can increase the predicted y to infinity by increasing any x variable that has a positive coefficient. Furthermore, we cannot fit the model in one range of observed x values and then use x ...


1

The normal equation $(A^TA)\vec{\beta} = A^T\vec{y}$ actually gives you two equations: $\displaystyle\underbrace{\begin{pmatrix}\sum_{i=1}^{n}x_i^2 & \sum_{i=1}^{n}x_i \\ \sum_{i=1}^{n}x_i & n\end{pmatrix}}_{A^TA} \underbrace{\begin{pmatrix} \beta_1 \\ \beta_0 \end{pmatrix}}_{\vec{\beta}} = \underbrace{\begin{pmatrix} \sum_{i=1}^{n}x_iy_i \\ ...


1

The two ways you propose are not equivalent in an OLS regression. You should stick to the version where every fruit has a dummy. A generic OLS model is of the form $$y_i=\alpha+\beta_1 orange_i+\beta_2 apple_i+X\gamma+\epsilon_i.$$ So you need 2 dummies $orange,apple$, and if both are zero the constant $\alpha$ is the unconditional mean of pears. in general, ...


1

Firstly, please note that your second term in the brackets should be $w^TX^Ty$ (the $y$ is not transposed). Next, I think there are two things going on here: Since the gradient will be taken with respect to $w$, the term $y^Ty$ has been ignored Since $w^TX^Ty$ is a scalar, $(w^TX^Ty)=(w^TX^Ty)^T=y^TXw$. So, ignoring the $y^Ty$ term, your ...


1

People do often use binary categorical variables as 0/1 numeric variables in regression, yes. Note however you are almost certainly violating the i.i.d. gaussian assumption on regression errors in that case. But people still do it. An alternative is to learn two different regression models, one for people under 18 and one for people over 18. However this ...


1

Here I am thinking I should remember something about this, or maybe I shouldn't. We have a matrix "equation": $$ \begin{bmatrix} 1 & h_1 & w_1 \\ \vdots & \vdots & \vdots \\ 1 & h_n & w_n \end{bmatrix} \begin{bmatrix} 7 \\ 0.08 \\ 0.06 \end{bmatrix} \overset{\text{?}} = \begin{bmatrix} a_1 \\ \vdots \\ a_n \end{bmatrix} \tag 1 $$ ...


1

Let us suppose that you fit a model $Z = a +b X+cY$ based on $N$ data points $(X_i,Y_i,Z_i)$. The so-called normal equations are $$\sum _{i=1}^N Z_i= N a + b\sum _{i=1}^N X_i+ c\sum _{i=1}^N Y_i$$ $$\sum _{i=1}^N X_iZ_i= a\sum _{i=1}^N X_i + b\sum _{i=1}^N X_i^2+ c\sum _{i=1}^N X_iY_i$$ $$\sum _{i=1}^N Y_iZ_i= a\sum _{i=1}^N Y_i + b\sum _{i=1}^N X_iY_i+ ...


1

If your sample size remains constant, you will have $k>n$ where $i=1,...,n$ so your matrix will not be invertible and an infinite (continuum) betas will be able to minimize the square errors.


1

I think to resolve $\theta$ by gradient will be hard way (or impossible??). Because it different with linear classfication, it will not has close form. So i suggest you can use other method example Newton's method. BTW, do you find $\theta$ using above way?



Only top voted, non community-wiki answers of a minimum length are eligible