# Selecting the best unknown probability distribution given constrains

I'm not sure how to pose the question, so I'll just try to explain. It's inspired by reading lecture notes on statistical mechanics (section 2 on page 3).

Let's say I have an unknown probability distribution $$\{p_i\}_{i=1}^N, \sum_i^N p_i = 1, 0 \le p_i \le 1$$ (for simplicity assuming finite number of possible outcomes). All I know are some averages obtained with these $$p_i$$s, e.g. $$\sum_{i=1}^N p_i x_i = X$$

$$\sum_{i=1}^N p_i y_i = Y$$

etc. All $$x_i$$, $$y_i$$, $$X$$, $$Y$$, ... are a given, but the number of such equations is less than $$N$$, so it's not possible to uniquely determine all $$p_i$$s. What I want to find is "the best" solution for $$\{p_i\}_{i=1}^N$$. Let's say that it should deviate from uniform distribution ($$p_i = 1/N$$ for all $$i$$) "as little as possible". Is there any rigor concept for such a task in mathematics?

E.g. in statistical mechanics, one should find a set of $$p_i$$s that maximizes entropy $$S = -\sum_{i=1}^N p_i \ln p_i$$ given the constraints (this way you can obtain Gibbs distribution if average energy $$U$$ of the system is a given: $$\sum_i p_i = 1, \sum_i p_i E_i = U \implies p_i = \exp(-E_i/T)/Z$$).

On the other hand, without reading the lecture notes above and knowing nothing about the entropy, I'd just try to minimize $$\chi^2$$ instead: $$\chi^2 = \sum_{i=1}^N \left( p_i - 1/N \right)^2$$ given the same constraints, this way I would get a different set of $$p_i$$

Obviously, there are many more ways to do it.

But is there any rigor concept that I could use to compare the results? Maybe a way to define how "far" one distribution is from another, so that I could say that the one obtained by maximizing entropy is "closer" to a uniform distribution than the one obtained by minimizing $$\chi^2$$?