# Tag Info

### Mathematics (provable properties) of autoencoder neural networks

As a starter, how about this reference: Pierre Baldi and Kurt Hornik. “Neural networks and principal component analysis: Learning from examples without local minima”. In: Neural Networks 2 (1989), pp....

### Likelihood of two dependent variables

Your $p(x\mid z,\lambda)=e^{-\lambda tx}(1-e^{-\lambda t})^{1-x}$ is wrong. It should not depend on $\lambda$ ($X$ is determined by $Z$ and $t$) and should be $1$ in any possible case $(x=1, z\ge t$ ...

### exp concavity of hellinger loss

I am assuming $l \in [0,1]$. Break the functions up: let $$f(z) = e^{\eta z/2} \qquad g(z) = (\sqrt{z} - \sqrt{l})^2 + (\sqrt{1-z} - \sqrt{1-l})^2 \qquad h(z) = f(g(z)).$$ Use the chain rule to ...

### Bishop - Pattern Recognition & Machine Learning, Exercise 1.4

The reason for the term involving $|g(y)|$ originates from the change of variables from $x$ to $y$. When we change from $p_X(x)$ to $p_Y(y)$, we have to ensure that the integral over the probability ...
1 vote

### Are Graphs with "Edge Weights" more popular than Graphs with "Node Weights"?

In my understanding, if your hypothesis is, e..g, "people of similar heights tend to be friends with people of similar heights", then you should just do normal clustering without node ...

### Understanding the rewritten form of Nesterov Accelerated Gradient used to derive Nadam

Is there any proof that these two are the same or at least do the same thing? Answer : YES, these two formulas are equilavent (proof below). Rather, the "rewritten NAG" seems as if its ...

### How to discover a recursive relation in a data set?

There are fairly simple and fast methods for finding polynomials that match data exactly (As long as the data set size doesn't exceed 10^6, it'll work) (Newton-Gregory Interpolation, and much more, ...

### how to transform the gradient update to a argmin expression

I have found the literature answering the first transformation: (https://www.eecis.udel.edu/~xwu/class/ELEG867/Lecture10.pdf) Specifically, for the gradient descent w^{t+1}=w^{t}-\eta \nabla f(w^{t})...
1 vote

### Computing $(A^TA)A^T$ versus $A^T(AA^T)$ on a Computer with Limited Memory where $A$ is $1000000 \times 2$

The product of an $m\times n$ and an $n \times k$ matrix is an $m \times k$ matrix, and each entry requires $n$ multiplications to compute, for a total of $mnk$ multiplications (assuming the naive ...

1 vote

### How do we know that some System of Equations doesn't have an "Analytical Solution"?

When you cannot solve a problem by hand, an analytical solution does not exist. Numerical solution by the aid of a computer or a calculator could be possible, if an answer exists.
1 vote

### How do we know that some System of Equations doesn't have an "Analytical Solution"?

Let $f:A\to B$ be a mapping, then an “equation” can be written as $f(a)-b=0$ with $a \in A$ and $b\in B.$ From here, you can wind up in almost every branch of mathematics. With neural networks, $A$ is ...

### How do we know that some System of Equations doesn't have an "Analytical Solution"?

In my opinion, your question is verbose and comprehends some (very) anxious attempt to delegate to a numerical instance-dependent method the task of finding the roots of some given statement. We may ...
Accepted

### Derive the derivative of cost function of logistic regression.

\$ \def\b{\omega_0}\def\s{\sigma}\def\o{{\tt1}}\def\p{\partial} \def\L{{\cal L}} \def\LR#1{\left(#1\right)} \def\diag#1{\operatorname{diag}\LR{#1}} \def\Diag#1{\operatorname{Diag}\LR{#1}} \def\trace#1{\...