The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
1answer
9 views

Parameterise Doubly Stochastic Matrices

Given the set of doubly stochastic matrices of dimension $n$, $D$, is it possible to find a continuous bijectice mapping $f: \mathbb{R}^i \rightarrow D$ for $i \leq n^2$. The motivation is to be able ...
0
votes
0answers
26 views

Asymptotic expansion using method of steepest descents

I am trying to find the first term in the asymptotic expansion of $$\frac{1}{2\pi i}\int_{c-i\infty}^{c+i\infty}\frac{1}{s^2}e^{t(s-m\sqrt{s^2-1})} ds $$ where $0<m<1$, $c<1$, as $t$ ...
1
vote
1answer
20 views

why use a small learning rate in gradient descent

I am new to neural networks and recently found out about gradient descent. Something does not sit right with me. x←x−λ∇fk(x) Why does this formula work? ...
3
votes
0answers
25 views

(Relatively) simple applications of descent theory?

There are some questions on descent theory scattered throughout MSE and MO. The answers to these questions are generally speaking way over my head. In volume two of Borceux, there is a short chapter ...
1
vote
1answer
27 views

Reference request: convergence property of continuous gradient descent?

Does anyone know of a text that treats the problem of gradient descent from a continuous perspective instead of a discretized perspective? For example, most text investigates the numerical properties ...
1
vote
0answers
28 views

Does every Young diagram have a unique minimal major index?

Given a Young diagram, $Y_\rho$, corresponding to an irreducible complex representation $\rho$ of the symmetric group $S_n$, we can associate a set of major indices $\{ ...
1
vote
0answers
47 views

Can every iterative algorithm be viewed as gradient descent over some objective?

In Algorithms for Non-negative Matrix Factorization, Lee and Seung give multiplicative algorithms derived from gradient descent on the Frobenius norm to find a non-negative matrix factorization. ...
3
votes
1answer
60 views

Is there a particularly simple example of geometric descent?

I'm looking for a particularly simple and familiar example of descent in geometry or topology in order to motivate the general definition. I'm not counting the definition of the arrow category ...
1
vote
0answers
61 views

Does faithfully flat descent work using restriction of scalars rather than extension?

Vistoli's notes on fibred categories and descent - http://homepage.sns.it/vistoli/descent.pdf - introduce (section 4.2.1) descent on modules over a commutative ring. The idea is as follows: ...
0
votes
1answer
44 views

Steepest Descent Sequence

How can I compute the first three iterates for the steepest descent sequence $f(x_1,x_2) = \frac{(x_1^2+3x_2^2)}{2}$ beginning at $x_0 = (\frac{\sqrt{3}}{2}, \frac{1}{2 \sqrt{3}})^T$ $\nabla ...
4
votes
0answers
99 views

Unwinding descent via Barr-Beck

Let $f: U \rightarrow X$ be a faithfully flat morphism of nice schemes (quasiseparated, quasicompact, and anything else I might have forgotten). One can understand descent in quasicoherent sheaves ...
2
votes
0answers
115 views

Derivation of Steepest Descent Direction used in Line Search Methods

In the numerical optimization text I am reading, the Steepest Descent Direction was derived by considering $$ \min_{||p||_2\leq 1} p^T\nabla f(x_k) $$ This resulted in $$ p_k=-\frac{\nabla ...
1
vote
2answers
232 views

Gradient decent using Taylor Series

I'm reading a book about Gradient methods right now, where the author is using a Taylor series to explain/derive an equation. $$ \mathbf x_a = \mathbf x - \alpha \mathbf{ \nabla f } (\mathbf x ) $$ ...
2
votes
1answer
100 views

How does a section of a stack give a sheaf?

At nLab in the article constant stack and a few other related articles, a pattern is mentioned where a section of a constant sheaf is a locally constant function, a section of a constant stack is a ...
2
votes
0answers
46 views

When is the map “attaching irreducible components” an effective isomorphism?

Let $\mathcal C$ be a category with fiber products. We say that a morphism $X \to Y$ in $\mathcal C$ is an effective epimorphism if the sequence of sets $$ \text{Hom}(Y,S) \to \text{Hom}(X,S) ...
2
votes
0answers
30 views

Does the method of steepest decent always move in an orthogonal direction between iterations?

I understand everything, I think, about the method but the result (or requirement) that successive steps are orthogonal to each other. SO, with the formula for this algorithm as: ...
2
votes
0answers
51 views

G-equivariant invertible sheaves on affine curves

Let $A$ be a Noetherian integral domain, and $G$ a finite group of automorphisms acting on $A$. Let $B = A^G$, the ring of invariants. The inclusion $B \hookrightarrow A$ induces a surjective morphism ...
4
votes
1answer
244 views

SGA 4.5 proof of Hilbert 90 and semilinear Galois action

In SGA 4.5's proof of Hilbert 90, proposition 1.5.2(that the inclusion $V'^G \otimes_k k' \rightarrow V'$ is an isomorphism) is deduced from faithfully flat descent as stated in 1.4.5. The way that I ...
3
votes
1answer
188 views

Using faithfully flat descent to prove representability of a functor in a simple case

Let $k$ be a field with a fixed separable closure $k_s$ and $G$ a finite type $k$-group scheme. Assume $F:(\mathrm{Sch}/k)^{opp}\rightarrow\mathrm{Set}$ is a contravariant functor whose restriction ...
2
votes
0answers
55 views

Optimization, descent direction, neccessary condition

I'm learning about nonlinear, unconstrained optimization. In my book it says that a descent direction $p_k$ must satisfy: $$p_k\nabla f(x_k)^T < 0$$ This seems to mean that $p_k$ must be obtuse to ...
1
vote
1answer
127 views

What is the difference between projected gradient descent and ordinary gradient descent?

I just read about projected gradient descent but I did not see the intuition to use Projected one instead of normal gradient descent. Would you tell me the reason and preferable situations of ...
1
vote
0answers
181 views

Intuition about multiplicative gradient descent

Suppose we want to minimize a function $f(x)$ wrt $x$, i.e., we want to solve, $$x^* = \arg \min_x f(x)$$ One method to solve such problems is gradient descent. In gradient descent, one uses the ...
5
votes
1answer
182 views

Can I check smoothness after a base-change

Let $X\to S$ be a flat morphism of noetherian schemes. I know that I can check smoothness on the geometric fibers to see whether $X\to S$ is smooth. Let $T\to S$ be a surjective morphism. Under what ...
1
vote
1answer
380 views

Machine Learning, why not use matrix multiplication instead of gradient descent?

If we want to minimize our Cost function for a given set of data, why do we use gradient descent and continually guess values until we find a min value for theta when when can just use matrix ...
5
votes
2answers
283 views

combinatorial descents finding the number of permutations with criteria

I need help with the following: Define a descent of a permutation to be $j$ when $p_{j+1} < p_j$. Then the descent set of a permutation is the set of all descents. For example, the ...
0
votes
1answer
399 views

Finding descent direction of quadratic function

I have a quadratic function: $f(x) = 24x_1+14x_2+x_1x_2$ and point $x_0 = (2,10)^T$ with $f(x_0) = 208$ And the first question is "give descent direction r in $x_0$" The second question "is f convex ...
1
vote
0answers
76 views

Rate of convergence of a single-neuron Perceptron network

I'm implementing a Perceptron network which basically consists of a single neuron in a single layer, trying to learn an OR logic port (linearly separable), but using the sigmoid function as ...
3
votes
2answers
503 views

Solving Linearly Constrained Quadratic Programming with Coordinate Descent

Does anybody have any idea about how to solve the following problem with Coordinate Descent? \begin{align} \min &\quad \mathbf{x}^{\top}P\mathbf{x} + b^{\top}\mathbf{x}\\ \text{Subject to}& ...
2
votes
1answer
146 views

Fréchet mean between points in $\mathbb{R}^3$

Let $X$ be a set of $n$ points in $\mathbb{R}^3$ and $f_m$ be the Fréchet mean, i.e.: $$ f_m= \arg \min_{p \in M} \sum_{i=1}^n w_id^2(p,x_i) $$ where $(\mathbb{R}^3,d)$ is a complete metric space, ...
1
vote
1answer
581 views

How to prove the Energy function of a Hopfield net is monotonically decreasing?

How to prove the Energy function of a Hopfield net is monotonically decreasing? $E = -1/2 \sum_{i,j} {w_{ij}}{s_i}{s_j} + \sum_{i}^{}s_{i} \theta_i$ I'll assume a proof involves the standard ...
7
votes
0answers
208 views

Do cokernels in RingSpc automatically lead to descent?

I'm currently interested in the following result: Let $f: X \to Y$ be a fpqc morphism of schemes. Then there is an equivalence of categories between quasi-coherent sheaves on $Y$ and "descent data" ...