0
$\begingroup$

I am trying to wrap my head around Arg Min, and I think I may have grasped it, so I want to discuss the interpretation in a specific example from a LASSO regression model $$ \hat{\beta_h} = \arg \min_{\beta_h} \quad RSS + \lambda \sum_{j=1}^{247} \vert \beta_h \vert $$ From what I understand, this example says that $\hat{\beta_h}$, should be equal to the ${\beta_h}$ which minimizes the function $\quad RSS + \lambda \sum_{j=1}^{247} \vert \beta_h \vert$.

I am not sure if this interpretation is correct.

Bu this brings another questions, shouldn't also $\lambda$ should be chosen, together with $\beta_h$ to minimize $\quad RSS + \lambda \sum_{j=1}^{247} \vert \beta_h \vert$. Where does this happen?

Thanks.

$\endgroup$

1 Answer 1

0
$\begingroup$

The sum $\sum_{j=1}^n |\beta_j|$ imposes a restriction to your parametric set in the form of $n$-dimensional diamond. Therefore, the arg min of your function, i.e., the minimal $\beta$, should be inside this diamond. Clearly, the borders of this diamond are determined by $\lambda$, and if the arg min is further away from $0$, your answer may be very affected by the size of $\lambda$. Therefore, $\lambda$ is usually varied on a set of predetermined values, and then the optimal $\lambda$ is chosen according to some criteria, e.g., $\lambda$ that minimizes the mean squared error.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .