Reputation
Top tag
Next privilege 15,000 Rep.
Protect questions
Badges
2 11 39
Newest
 Excavator
Impact
~149k people reached

8h
comment Next book in learning Differential Geometry
Many people recommend Introduction to Smooth Manifolds by Lee.
1d
comment Proof of convergence for the proximal point algorithm
There's this whole topic called monotone operator theory where this kind of terminology is standard. It comes up in convex optimization theory. I'm probably omitting a ton of details in these comments.
1d
comment Proof of convergence for the proximal point algorithm
Haha. So $\partial f$ is a set-valued mapping, and $I + t \partial f$ is a set-valued mapping. And if $F$ is a set-valued mapping, then $F^{-1}$ is the set-valued mapping defined by $F^{-1}(w) = \{x \mid w \in F(x) \}$. So it would seem that $(I + t \partial f)^{-1}$ is a set-valued mapping. However, it turns out that $(I + t \partial f)^{-1}(w)$ is a singleton for any $w$, so $(I + t \partial f)^{-1}$ can be viewed as a function that takes a vector as input and returns a single vector as output. And it turns out that $(I + t \partial f)^{-1}$ is the same thing as the prox-operator of $f$.
1d
comment Proof of convergence for the proximal point algorithm
Yes, that's a good way to say it. And also the function $y \mapsto f(y) + \frac{1}{2t} \| y - x \|^2$ is guaranteed to have a unique minimizer (when $f$ is closed and convex), so we could write $x = \text{prox}_{tf}(x)$ rather than $x \in \text{prox}_{tf}(x)$.
1d
comment Proof of convergence for the proximal point algorithm
I certainly didn't downvote you! I just upvoted you now. Your answer is great. In my comment $I$ is the identity operator, $\partial f$ is the set-valued mapping that maps $x$ to the set $\partial f(x)$, which consists of all subgradients of $f$ at $x$. You're right, my proof there is presented too tersely, but it's possible to fill in the details.
1d
comment Proof of convergence for the proximal point algorithm
Note that $x$ minimizes a closed convex function $f$ $\iff 0 \in \partial f(x) \iff x \in (I + t\partial f)(x) \iff x = (I + t \partial f)^{-1}(x) \iff x = \text{prox}_{tf}(x)$. That's a different proof that the minimizers of $f$ are exactly the fixed points of the prox-operator of $f$.
1d
comment Linear Algebra L2 minimization
The gradient is equal to $0$ at a minimizer. So, you can compute the gradient of this function and set it equal to $0$, and you will have your linear system.
2d
comment Help understanding the range and kernel of a linear transformation
To make it concrete, you could try to describe the range and null space (kernel) of the linear transformation $T(x_1,x_2,x_3) = (x_1,0,0)$. Which of the following vectors are in the range: $(0,5,0)$? $(7,0,0)$? $(0,5,7)$? Which of those vectors are in the null space?
2d
comment What is the interpretation of the following optimization problem?
If we replace the positivity constraint with a non negativity constraint, this feasibility problem is a linear program, and linear programs are often solved using interior point methods, or a simplex method. It could also be solved using proximal algorithms.
2d
comment What is the interpretation of the following optimization problem?
Yes, you can replace 1 by any constant. This is called a feasibility problem.
May
26
comment Why imaginary numbers axis is plotted perpendicular to the real numbers axis?
If you do it that way, then addition and multiplication have nice geometric interpretations. For example, to multiply two complex numbers, you add the angles and multiply the lengths. (As a special case, multiplying by $ i $ simply rotates by 90 degrees counter clockwise. )
May
26
comment System of linear equations: get approximate solution with non-negative coefficients
It would also be possible to minimize the $1$-norm of the residual, or the $\infty $ norm, if that would be useful.
May
25
comment System of linear equations: get approximate solution with non-negative coefficients
A keyword is non negative least squares. Matlab has a function for that. You minimize the norm of the residual subject to a non negativity constraint. This optimization problem could be solved using proximal algorithms.
May
25
comment How was the determinant of matrices generalized for matrices bigger than $2 \times 2$?
If you solve a generic 3 by 3 or 4 by 4 linear system by hand, the formula for the determinant when $ n $ is $3$ or $4$ will pop out. At that point, you can see the pattern and guess a formula that holds for any $ n $.
May
22
comment Is this enough to explain why set theory work in real analysis?
I'm by no means an expert on set theory (far from it). But my impression is that any paradoxes in set theory arise from attempting to introduce sets which, if they actually existed, would have to contain themselves. And I think our common sense idea of a set will tell us that a set can't contain itself, any more than say a grocery bag could contain itself. So if you just avoid introducing sets that would have to contain themselves, I think our common sense ideas about sets can be trusted.
May
19
comment Eigenvectors and Eigenvalues PLEASE HELP!
You'll get a better response if you show what you tried.
May
15
comment Books covering the basics of Fourier Transform for image processing
I found it very enlightening to learn the linear algebra viewpoint on the Fourier transform. For example Strang's book Linear Algebra and its Applications has a good discussion of the discrete Fourier transform. The DFT simply changes basis to a special basis, the discrete Fourier basis, which is a basis of eigenvectors for the cyclic shift operator.
May
15
comment Is there any way to make the following function convex?
Note that even if $ P$ were positive definite, your quadratic equality constraint would still be a non convex constraint. (It constrains $ x $ to belong to a non convex set. ) In a convex problem, equality constraints should be linear.
May
15
comment I don't understand/know how to solve this question from MIT ocw Caculus.
You'll get a better response if you show what you tried, what you're stuck on specifically, and what your thoughts are.
May
13
comment Consider I'm a 10 year old kid, explain what “linearly independent” and “basis” means
Should we assume the 10 year old kid knows what vectors are?