I need to "construct a distribution" that I can run in a computer program, in a particular way that is not clearly specified.

Say I have the following function,

$$ f(x) = (4-2.1x^2 + \frac{y^4}{3})x^2 + xy + (-4 + 4x^2)y^2 $$

And I am going to explore the space $ x \in [-3,3], y \in [-2,2]$ in search of the global min.

Say I have $n$ samples, $x\in[-3,3],y\in[-2,2]$:

$$ (x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n) $$

These samples evaluate to values that are higher and lower. I throw away the bottom half of the values, and keep the top half.

In order to continue, I need to estimate the mutual information gain between the $y$'s and the $x's$, and then produce samples according to that distribution (eventually, I need to scale up to more dimensions). To do that, I need the conditional distribution, the joint distribution, etc.

Mutual Information Gain: $$ I[X;Y] = \int_Y \int_X p(x,y) log(\frac{p(x,y)}{p(x)p(y)})dxdy $$

How do I "estimate" the conditional distribution of all remaining $x$'s, given, say, all $y$'s, using the $\frac{n}{2}$ $x$'s remaining, relative to all $n$ $y$'s?

  • $\begingroup$ Do you mean after you observed $n$ pairs of them, you sort $X_i$ and $Y_i$ independently, and discarded all the bottom half data? And you do not know if there is any pair remaining? $\endgroup$ – BGM Mar 3 '17 at 4:05
  • $\begingroup$ @BGM I have a set of samples, $\lbrace [x_1,y_1], \ldots, [x_n,y_n]\rbrace$ and each sample evaluates to a value. I take the best half of the samples (maxing the value, maybe). Then, I need to somehow find the "mutual information gain" of $x$ relative to $y$. In the end, this will be run in say, $d$ dimensional space..and there will be more to the mutual information gain calc $\endgroup$ – donlan Mar 3 '17 at 21:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.