Distribution divergence and moment distance It is well-known that under some conditions, a distribution can be uniquely determined by its moments. What I'm curious about, is that suppose two distributions $P_1$, $P_2$ have very similar moments of all orders, namely
$|m_1^1 - m_1^2| < \delta_1$
$|m_2^1 - m_2^2| < \delta_2$
...
$|m_n^1 - m_n^2| < \delta_n$
where $m_a^b$ means the $a^{th}$ moment of the $b^{th}$ distribution, and $\delta_k$ is a small quantity. Then intuitively $P_1$ and $P_2$ should be very "close" to each other. So is there any theorem that states some divergence (say KL or Hellinger divergence) between $P_1$ and $P_2$ can be upper-bounded by a function of $\delta_k, k=1\ldots...n$? Thanks in advance!
 A: A lower bound which couples the moments of $P_2$ with the mean of $P_1$ is Kullback's inequality for KL divergence (but not in a clean way in terms of the $\delta$'s - there are other simple lower bounds for KL divergence such as in terms of TV distance ). However, you can find distributions where the moments are arbitrarily close and the K-L divergence $D(P_1 || P_2)$ is infinite. 
Let $P_1$ take on the value $0$ with probability $1$, and $P_2$ take on the value $0< c << 1$ with probability $1$. Then the KL divergence is infinite, but the $\delta_k$'s can be chosen to be small for appropriate values of $c$ (you could use scaled and shifted Bernoulli's or something else if you want a less trivial example - if the supports are disjoint, the KL divergence will always be infinite (or if the support of $P_1$ strictly contains the support of $P_2$). 
Likely, you want to impose some additional constraints, which can give nice bounds. For example, the Hellinger distance is easily bounded in terms of the TV distance. A neat example (along with a link to Verdu's recent ITA paper) is here for the case of Gaussians where the covariance matrix differs entrywise by at most $\delta$. 
