# How to construct a joint distribution given a continuous space?

I need to "construct a distribution" that I can run in a computer program, in a particular way that is not clearly specified.

Say I have the following function,

$$f(x) = (4-2.1x^2 + \frac{y^4}{3})x^2 + xy + (-4 + 4x^2)y^2$$

And I am going to explore the space $x \in [-3,3], y \in [-2,2]$ in search of the global min.

Say I have $n$ samples, $x\in[-3,3],y\in[-2,2]$:

$$(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)$$

These samples evaluate to values that are higher and lower. I throw away the bottom half of the values, and keep the top half.

In order to continue, I need to estimate the mutual information gain between the $y$'s and the $x's$, and then produce samples according to that distribution (eventually, I need to scale up to more dimensions). To do that, I need the conditional distribution, the joint distribution, etc.

Mutual Information Gain: $$I[X;Y] = \int_Y \int_X p(x,y) log(\frac{p(x,y)}{p(x)p(y)})dxdy$$

How do I "estimate" the conditional distribution of all remaining $x$'s, given, say, all $y$'s, using the $\frac{n}{2}$ $x$'s remaining, relative to all $n$ $y$'s?

• Do you mean after you observed $n$ pairs of them, you sort $X_i$ and $Y_i$ independently, and discarded all the bottom half data? And you do not know if there is any pair remaining? – BGM Mar 3 '17 at 4:05
• @BGM I have a set of samples, $\lbrace [x_1,y_1], \ldots, [x_n,y_n]\rbrace$ and each sample evaluates to a value. I take the best half of the samples (maxing the value, maybe). Then, I need to somehow find the "mutual information gain" of $x$ relative to $y$. In the end, this will be run in say, $d$ dimensional space..and there will be more to the mutual information gain calc – donlan Mar 3 '17 at 21:17