I am learning stats myself, I learned about unbiased estimators yesterday. This is the definition of unbiased estimators:
Let $\theta$ be a fixed parameter for a probability distribution $P$ and let $X_1,X_2,\ldots, X_n$ be samples drawn from $P$, then a statistic $u(X_1,X_2,\ldots,X_n)$ is called an unbiased estimator for $\theta$ if $E(u(X_1,X_2,\ldots,X_n)=\theta$
These are some of my confusions:
1) Is this definition complete and correct?
2) The condition that the expectation of statistic should equal the parameter seems completely arbitrary to me. I mean to ask what does this imply and why is it a desirable property?
For example, supposed I have $n$ iid samples $x_1,x_2,\ldots,x_n$ ($x_i$ are real numbers) from an unknown distribution $P$. I want to estimate a fixed parameter $m$ (like median/variance etc) of the distribution $P$ and I have a statistic whose expectation equals $m$. Why should I expect $u(x_1,x_2,\ldots,x_n)$ to approximate $\theta$ and how close is this approximation supposed to be?