3 votes

Local optima of non-negative-least-squares?

Broadly speaking, with convex problems, necessary conditions are sufficient. A nice characterisation of an optimal solution is: If the cost $f$ is convex & differentiable and the constraint set $C$...
  • 163k
2 votes

Least squares with equality constraint

If your vector $x$ has $n$ components, then simply set all $x_i:=\frac{1}{n}$. Then $\bar{x}=\frac{1}{n}$, each $x_i$ is equal to this mean, your sum of squares is zero (so it is minimal, because ...
1 vote

Prove that the sample covariance between observation and OLS fittings are nonnegative

According to the wiki page you linked, $C$ is PSD, and $X(X^TX)^{-1}X^T$ is also PSD since for any $u \in \mathbb{R}^n$, $$ u^T X(X^TX)^{-1}X^T u = y^T S^{-1} y = \|y\|^2_{S} \ge 0 $$ where $y := X^T ...
  • 4,782
1 vote

Difference in Tikhonov regularization for linear and non-linear case?

I think you're confused by the notation. Lets say we have some transformation $f$ acting on $x$, i.e. $f(x)$. The Problem can be stated as: $$ \min_x \,\{\lVert f(x)-y\rVert^2 + \lambda\lVert x\rVert^...
  • 153

Only top scored, non community-wiki answers of a minimum length are eligible