The tag has no wiki summary.

learn more… | top users | synonyms

1
vote
0answers
30 views

Derivation of Steepest Descent Direction used in Line Search Methods

In the numerical optimization text I am reading, the Steepest Descent Direction was derived by considering $$ \min_{||p||_2\leq 1} p^T\nabla f(x_k) $$ This resulted in $$ p_k=-\frac{\nabla ...
1
vote
2answers
18 views

Gradient decent using Taylor Series

I'm reading a book about Gradient methods right now, where the author is using a Taylor series to explain/derive an equation. $$ \mathbf x_a = \mathbf x - \alpha \mathbf{ \nabla f } (\mathbf x ) $$ ...
2
votes
1answer
81 views

How does a section of a stack give a sheaf?

At nLab in the article constant stack and a few other related articles, a pattern is mentioned where a section of a constant sheaf is a locally constant function, a section of a constant stack is a ...
2
votes
0answers
36 views

When is the map “attaching irreducible components” an effective isomorphism?

Let $\mathcal C$ be a category with fiber products. We say that a morphism $X \to Y$ in $\mathcal C$ is an effective epimorphism if the sequence of sets $$ \text{Hom}(Y,S) \to \text{Hom}(X,S) ...
2
votes
0answers
13 views

Does the method of steepest decent always move in an orthogonal direction between iterations?

I understand everything, I think, about the method but the result (or requirement) that successive steps are orthogonal to each other. SO, with the formula for this algorithm as: ...
3
votes
4answers
133 views

Tricky descent proof

EDIT: Please see EDIT(2) below, thanks very much. I want to prove by infinite descent that the only positive integer factors of integers of the form $a^2+3b^2$ have the same form. For example, ...
2
votes
0answers
35 views

G-equivariant invertible sheaves on affine curves

Let $A$ be a Noetherian integral domain, and $G$ a finite group of automorphisms acting on $A$. Let $B = A^G$, the ring of invariants. The inclusion $B \hookrightarrow A$ induces a surjective morphism ...
0
votes
0answers
63 views

Can't find gradient for MLE for mult-class logistic regression

$$P(k | x_i;w)= \frac{exp(w_k^tx_i)}{\sum_{j=1}^K exp(w_k^tx_i)}$$ $y_i^k$ is a vector that uses 1-of-k encoding. Thus, if $y_i=k$, then the vector $y_i$ has a 1 in the kth spot and a 0 everywhere ...
0
votes
0answers
29 views

As a beginner, I would like to solve convex quadratic maximization problem with a gradient descent variant in probability simplex?

I know the basics of gradient approaches to optimize the function iteratively, but for this case have have a equality constraint as $\sum_{i=1}^Nx_i = 1$ where each $x_i \geq 0$ with the objective ...
3
votes
1answer
154 views

SGA 4.5 proof of Hilbert 90 and semilinear Galois action

In SGA 4.5's proof of Hilbert 90, proposition 1.5.2(that the inclusion $V'^G \otimes_k k' \rightarrow V'$ is an isomorphism) is deduced from faithfully flat descent as stated in 1.4.5. The way that I ...
3
votes
1answer
108 views

Using faithfully flat descent to prove representability of a functor in a simple case

Let $k$ be a field with a fixed separable closure $k_s$ and $G$ a finite type $k$-group scheme. Assume $F:(\mathrm{Sch}/k)^{opp}\rightarrow\mathrm{Set}$ is a contravariant functor whose restriction ...
2
votes
0answers
32 views

Optimization, descent direction, neccessary condition

I'm learning about nonlinear, unconstrained optimization. In my book it says that a descent direction $p_k$ must satisfy: $$p_k\nabla f(x_k)^T < 0$$ This seems to mean that $p_k$ must be obtuse to ...
1
vote
1answer
58 views

What is the difference between projected gradient descent and ordinary gradient descent?

I just read about projected gradient descent but I did not see the intuition to use Projected one instead of normal gradient descent. Would you tell me the reason and preferable situations of ...
1
vote
0answers
107 views

Intuition about multiplicative gradient descent

Suppose we want to minimize a function $f(x)$ wrt $x$, i.e., we want to solve, $$x^* = \arg \min_x f(x)$$ One method to solve such problems is gradient descent. In gradient descent, one uses the ...
5
votes
1answer
112 views

Can I check smoothness after a base-change

Let $X\to S$ be a flat morphism of noetherian schemes. I know that I can check smoothness on the geometric fibers to see whether $X\to S$ is smooth. Let $T\to S$ be a surjective morphism. Under what ...
1
vote
1answer
160 views

Machine Learning, why not use matrix multiplication instead of gradient descent?

If we want to minimize our Cost function for a given set of data, why do we use gradient descent and continually guess values until we find a min value for theta when when can just use matrix ...
5
votes
2answers
182 views

combinatorial descents finding the number of permutations with criteria

I need help with the following: Define a descent of a permutation to be $j$ when $p_{j+1} < p_j$. Then the descent set of a permutation is the set of all descents. For example, the ...
0
votes
1answer
169 views

Finding descent direction of quadratic function

I have a quadratic function: $f(x) = 24x_1+14x_2+x_1x_2$ and point $x_0 = (2,10)^T$ with $f(x_0) = 208$ And the first question is "give descent direction r in $x_0$" The second question "is f convex ...
1
vote
0answers
58 views

Rate of convergence of a single-neuron Perceptron network

I'm implementing a Perceptron network which basically consists of a single neuron in a single layer, trying to learn an OR logic port (linearly separable), but using the sigmoid function as ...
3
votes
2answers
320 views

Solving Linearly Constrained Quadratic Programming with Coordinate Descent

Does anybody have any idea about how to solve the following problem with Coordinate Descent? \begin{align} \min &\quad \mathbf{x}^{\top}P\mathbf{x} + b^{\top}\mathbf{x}\\ \text{Subject to}& ...
2
votes
1answer
132 views

Fréchet mean between points in $\mathbb{R}^3$

Let $X$ be a set of $n$ points in $\mathbb{R}^3$ and $f_m$ be the Fréchet mean, i.e.: $$ f_m= \arg \min_{p \in M} \sum_{i=1}^n w_id^2(p,x_i) $$ where $(\mathbb{R}^3,d)$ is a complete metric space, ...
1
vote
0answers
63 views

Can Fermats descent be interpreted on a conic?

Fermat proved the Diophantine equation $$(x^2)^2 + (y^2)^2 = z^2$$ has only solutions $(0,0,0)$, $(0,\pm 1,\pm 1)$ and $(\pm 1,0,\pm 1)$ using "infinite descent". The conic $C : X^2 + Y^2 - 1$ has a ...
11
votes
1answer
266 views

How to get Fermat descent working on other conics?

Fermat solved the Diophantine equation $(x^2)^2 + (y^2)^2 = z^2$ using descent, the key step was using the Pythagorean triples: $x^2 = u^2 - v^2$ $y^2 = 2 u v$ $z = u^2 + v^2$ but then it is seen ...
1
vote
1answer
475 views

How to prove the Energy function of a Hopfield net is monotonically decreasing?

How to prove the Energy function of a Hopfield net is monotonically decreasing? $E = -1/2 \sum_{i,j} {w_{ij}}{s_i}{s_j} + \sum_{i}^{}s_{i} \theta_i$ I'll assume a proof involves the standard ...
5
votes
0answers
190 views

Do cokernels in RingSpc automatically lead to descent?

I'm currently interested in the following result: Let $f: X \to Y$ be a fpqc morphism of schemes. Then there is an equivalence of categories between quasi-coherent sheaves on $Y$ and "descent data" ...