New answers tagged

0 votes

Mathematics (provable properties) of autoencoder neural networks

As a starter, how about this reference: Pierre Baldi and Kurt Hornik. “Neural networks and principal component analysis: Learning from examples without local minima”. In: Neural Networks 2 (1989), pp....
user avatar
  • 550
0 votes

Likelihood of two dependent variables

Your $p(x\mid z,\lambda)=e^{-\lambda tx}(1-e^{-\lambda t})^{1-x}$ is wrong. It should not depend on $\lambda$ ($X$ is determined by $Z$ and $t$) and should be $1$ in any possible case $(x=1, z\ge t$ ...
user avatar
  • 144k
3 votes

exp concavity of hellinger loss

I am assuming $l \in [0,1]$. Break the functions up: let $$f(z) = e^{\eta z/2} \qquad g(z) = (\sqrt{z} - \sqrt{l})^2 + (\sqrt{1-z} - \sqrt{1-l})^2 \qquad h(z) = f(g(z)).$$ Use the chain rule to ...
user avatar
0 votes

Bishop - Pattern Recognition & Machine Learning, Exercise 1.4

The reason for the term involving $|g(y)|$ originates from the change of variables from $x$ to $y$. When we change from $p_X(x)$ to $p_Y(y)$, we have to ensure that the integral over the probability ...
user avatar
  • 7,964
1 vote

Are Graphs with "Edge Weights" more popular than Graphs with "Node Weights"?

In my understanding, if your hypothesis is, e..g, "people of similar heights tend to be friends with people of similar heights", then you should just do normal clustering without node ...
user avatar
  • 763
0 votes

Understanding the rewritten form of Nesterov Accelerated Gradient used to derive Nadam

Is there any proof that these two are the same or at least do the same thing? Answer : YES, these two formulas are equilavent (proof below). Rather, the "rewritten NAG" seems as if its ...
user avatar
  • 8,153
0 votes

How to discover a recursive relation in a data set?

There are fairly simple and fast methods for finding polynomials that match data exactly (As long as the data set size doesn't exceed 10^6, it'll work) (Newton-Gregory Interpolation, and much more, ...
user avatar
0 votes

how to transform the gradient update to a argmin expression

I have found the literature answering the first transformation: (https://www.eecis.udel.edu/~xwu/class/ELEG867/Lecture10.pdf) Specifically, for the gradient descent $$w^{t+1}=w^{t}-\eta \nabla f(w^{t})...
user avatar
1 vote

Computing $(A^TA)A^T$ versus $A^T(AA^T)$ on a Computer with Limited Memory where $A$ is $1000000 \times 2$

The product of an $m\times n$ and an $n \times k$ matrix is an $m \times k$ matrix, and each entry requires $n$ multiplications to compute, for a total of $mnk$ multiplications (assuming the naive ...
user avatar
  • 22.1k
2 votes

Computing $(A^TA)A^T$ versus $A^T(AA^T)$ on a Computer with Limited Memory where $A$ is $1000000 \times 2$

In the context of someone who only knows about the definition of matrix multiplication, would this be correct? $A^TA$ is a $2 \times 2$ matrix (4 total cells), while $AA^T$ is a $1000000 \times ...
user avatar
  • 9,528
1 vote
Accepted

Explicit relation between the regularization parameters in Ivanov and Tikhonov regularization

Assume that for $\lambda>0$ problem (2) is solvable with solution $w_\lambda$. If $\hat L$ is convex, then a solution exists and is unique. Now take $\lambda<\lambda'$, and let $w$ and $w'$ be ...
user avatar
  • 42.9k
2 votes
Accepted

Proof of the variance of one-dimensional projections

The mean of the data $\overline{\mathbf{x}}={1\over N}\sum_{n=1}^N \mathbf{x}_n$ is a vector averaging over all the values of $\mathbf{x}_n$ and $\mathbf{u}_1$ is a unit vector, so $\mathbf{u}_1^T \...
user avatar
  • 7,964
1 vote

How do we know that some System of Equations doesn't have an "Analytical Solution"?

When you cannot solve a problem by hand, an analytical solution does not exist. Numerical solution by the aid of a computer or a calculator could be possible, if an answer exists.
user avatar
  • 39.3k
1 vote

How do we know that some System of Equations doesn't have an "Analytical Solution"?

Let $f:A\to B$ be a mapping, then an “equation” can be written as $f(a)-b=0$ with $a \in A$ and $b\in B.$ From here, you can wind up in almost every branch of mathematics. With neural networks, $A$ is ...
user avatar
  • 706
-2 votes

How do we know that some System of Equations doesn't have an "Analytical Solution"?

In my opinion, your question is verbose and comprehends some (very) anxious attempt to delegate to a numerical instance-dependent method the task of finding the roots of some given statement. We may ...
user avatar
0 votes
Accepted

Derive the derivative of cost function of logistic regression.

$ \def\b{\omega_0}\def\s{\sigma}\def\o{{\tt1}}\def\p{\partial} \def\L{{\cal L}} \def\LR#1{\left(#1\right)} \def\diag#1{\operatorname{diag}\LR{#1}} \def\Diag#1{\operatorname{Diag}\LR{#1}} \def\trace#1{\...
user avatar
  • 28.1k
0 votes

k-fold cross validation

In the train/cross-validate/test paradigm, your aim is to choose and tune your best model and then train it; one of your objectives is to avoid over-fitting. You start by separating a test set from ...
user avatar
  • 144k
1 vote

Calculation of Intrinsic dimension of datasets

0 because there are no dimensions, just integers in a set 5 the dimension is the exponent of epsilon in the general formula for dependence of number of elements on diameter 10 there are 10 ...
user avatar

Top 50 recent answers are included