Let us say we have three random variables $X_1$, $X_2$, and $X_3$ with joint distribution $P(X_1, X_2, X_3)$. I want to find the best (non-random) function $\theta_P (x_1, x_2, x_3)$ that minimises the expected value $$ \mathbb E \left[ \left( (\theta_P(X_1, X_2, X_3) - X_M \right)^2\right] \to \min, $$ where $M$ is uniform over $\{1,2,3\}$ and independent of $X_1, X_2, X_3$. Expectation is taken over the distribution $P$ and the distribution of $M$.

Also, I am interested in another function, $\xi_P(x_1,x_2,x_3)$ that minimises the following expected value: $$ \mathbb E \left[ \max_i \left( \xi_P(X_1, X_2, X_3) - X_i \right)^2 \right] \to \min. $$ Expected value is taken over the distribution $P$. Most probably, both $\theta_P$ and $\xi_P$ will depend on the distribution $P$ (that's why the subscript). One of the candidates for both cases can be $$\theta(x_1, x_2, x_3) = \xi(x_1, x_2, x_3) = \frac{x_1 + x_2 + x_3}{3}$$ but I am not at all sure this give the minimum expectations.

In some sense, both $\theta_P$ and $\xi_P$ are the closest (on average) points from $X_1, X_2, X_3$. The distance is a square of the Euclidean distance but, for minimisation problem, this distance will be equivalent to the Euclidean. (Right?)


I think I have an answer.

First case, $\theta_P$.

The first observation is that since $M$ is uniform and independent over $\{1,2,3\}$, we have that $$ \mathbb E[((\theta_P(X_1, X_2, X_3) - X_M)^2] = \frac 13 \mathbb E \left[\sum_{m=1}^3 (\theta_P(X_1, X_2, X_3) - X_m)^2 \right], $$ where the latter does not involve random $M$ any more.

The second fact (very obvious!) is that if $A \le B$ then $\mathbb E[A] \le \mathbb E[B]$. Hence, we can try to find the solution as $$ \theta_P(x_1, x_2, x_3) = \arg \min_t \left( \sum_{m=1}^3 (t-x_m)^2 \right) $$ where $x_1, x_2, x_3$ are treated as constants. In other words, we try to minimise $\theta_P$ in each of the points. By using basic calculus, the optimal $t$ is then $$ t = \frac{x_1 + x_2 + x_3}3, $$ which is also the expression for the optimal $\theta_P(x_1, x_2, x_3)$.

Surprisingly enough, the solution does not depend on the distribution of $(X_1, X_2, X_3)$.

Second case, $\xi_P$.

Again, we do optimisation for each triple of fixed values $(X_1, X_2, X_3)$ separately, i.e. $$ \xi_P(x_1, x_2, x_3) := \arg \min_t \left( \max \left( (t-x_1)^2, (t-x_2)^2, (t-x_3)^2 \right) \right). $$

We observe that $$ \max \left( (t-x_1)^2, (t-x_2)^2, (t-x_3)^2 \right) = \max \left( (t-x_{\min})^2, (t-x_{\max})^2 \right), $$ where $x_{\min} = \min(x_1, x_2, x_3)$ and $x_{\max} = \max(x_1, x_2, x_3)$. Finally, $$ \max( (t-x_\min)^2, (t-x_\max)^2) = \begin{cases} (t-x_\max)^2, & t < \frac{x_\min + x_\max}{2},\\ (t-x_\min)^2, & t \ge \frac{x_\min + x_\max}{2}, \end{cases} $$ and the global minimum is achieved at $t = \frac{x_\min + x_\max}{2}$. Altogether, $$ \xi_P(x_1, x_2, x_3) = \frac{\min(x_1, x_2, x_3) + \max(x_1, x_2, x_3)}2. $$ Again, it doesn't depend on distribution of $X_1, X_2, X_3$.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.