0
$\begingroup$

Suppose $f(\beta)$ is strictly convex. Show that $f(\hat{\beta}_k^*) < f(\hat{\beta}_k)$, where $\hat{\beta}, \hat{\beta}^* \in \mathbb{R}^n$ and

$\hat{\beta}_k^* = \begin{cases} \hfill \frac{1}{2}(\hat{\beta}_i+\hat{\beta}_j) \hfill & \text{ if $k \in \{i,j\}$} \\ \hfill \hat{\beta}_k \hfill & \text{ otherwise.} \\ \end{cases}$

This is a part of the proof to Lemma 2 (a) in [1]. The proof to this Lemma can be found in the Appendix of [1], but it includes this step, which I unfortunately do not understand. If I understand it correctly, it is also given that $f(\beta) >0$ $\forall \beta \neq 0$.

UPDATE: Any help would be appreciated. I might need to provide you with more information, but if you tell me what other information you need, then let me know. I attempted to solve this question several times and only am able to start with $f(\hat{\beta}_k^*)$ and separate this into $f(\frac{1}{2} \hat{\beta}+\frac{1}{2}\tilde{\beta})$, where $\tilde{\beta}$ is a vector, whose elements are $\{\hat{\beta}_1, \dotsc, \hat{\beta}_j, \dotsc, \hat{\beta}_i, \dotsc, \hat{\beta}_n\}$, where $n$ is the last element and I simply assume $i<j$. Then I can make use of the strict convexity, but that's how far I get.

[1] Zou, Hui, and Trevor Hastie. "Regularization and variable selection via the elastic net." Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67.2 (2005): 301-320.

$\endgroup$
  • $\begingroup$ just added my new information inside the post. $\endgroup$ – pthesling Aug 31 '15 at 15:20
0
$\begingroup$

I think that the claim is false. Consider $X = \mathrm{diag}(1, 0, 0)$, $y = 0$ and the strictly convex function $J(x) = ||x - (0, 1, 2)^t||_2^2$. Then the minimizer of $|y - X\beta|^2 + \lambda J(\beta)$ is clearly $\hat{\beta} = (0, 1, 2)^t$.

Or is there another assumption that I read over?

$\endgroup$
  • $\begingroup$ well it is given in the paper like this. But maybe I didn't see one of the assumptions. $\endgroup$ – pthesling Sep 1 '15 at 17:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.