How are Tr(AB) results restricted?

In quantum mechanics one postulates that for each state $i$ there is a matrix $A_i$ and for each measureable dynamical variable $j$ (velocity etc.) there is a matrix $B_j$. Both are Hermitian matrices over complex numbers.

The experimental average of the dynamical variable B is postulated to be

$\text{'average of variable j with state i'}=\operatorname{Tr}(AB)$

(additionally some restrictions are placed on the state density matrix $A\geq 0$ and $\operatorname{Tr}A=1$)

Does someone have an idea how this postulates restricts possible outcomes for the average values? Or can completely general systems from probability theory always be stated with this trace notation? Are there mathematically implicit correlations between different states or variables due to this postulate? Is there a similar framework in probability theory unrelated to QM?

All these questions are meant to be purely mathematically derived from the form of the equation.

-
I think that matrix $A$ is diagonal. Isn't it? –  userNaN Feb 27 '12 at 10:32
My vague recollection of QM is that if $A$ is a pure state, then it is an eigenvector of $B$, i.e. $AB=\lambda A$, where $\lambda$ is the value of the said dynamical variable (in this state). If we normalize $tr(A)=1$, then $$tr(AB)=tr(\lambda A)=\lambda tr(A)=\lambda.$$ For a mixed state, the trace gives you a weighted linear combination. I'm not sure this is heading in the direction of your question, but I think that the discreteness of the set of values comes from the discreteness of the set of eigenvalues. –  Jyrki Lahtonen Feb 27 '12 at 10:32
In general it is not restricted to diagonal matrices or pure states. I basically wonder if I can set up an arbitrary probabilistic thought experiment and translate that to the above matrix/trace formalism, or if the formulism implicitly includes contraints on thought experiments. Discreteness of results could be one constraint. Is there a constraint on possible pure state outcomes? –  Gerenuk Feb 27 '12 at 10:42
I'm not sure I understand the question. This expression is usually derived by assuming an arbitrary mixed state and showing that the expectation value is given by that trace. If that is so, how could there possibly be any restrictions inherent in this expression that are incompatible with the correct expectation values for arbitrary mixed states? –  joriki Feb 27 '12 at 11:10
Then I think you'd have to define "arbitrary probabilistic experiments". If this means anything like what I can imagine it might mean, then all discrete probabilistic experiments are covered by this formalism simply because you can associate every possible outcome with an eigenvector of $A$ and then consider only mixed states formed using eigenvectors of $A$ to get a classical probability distribution. If that's not what you mean, then I think you'll have to say more about the "arbitrary probabilistic experiments". –  joriki Feb 27 '12 at 11:26

The direct analogue in classical probability is that $A$ becomes an arbitrary probability measure $\mu$, and $B$ some random variable $X$. The trace operator is then the expectation value of $X$ relative to measure $\mu$.

That the two coincide is easily seen in the case where your state space is finite dimensional, so $A$ is actually a matrix. The underlying probability space is a finite set $\Omega$ on which we take the sigma algebra $2^X$. Given an arbitrary probability measure $\mu$ on $\Omega$ (which we identify with a function on the finite set) you can extend it to a probability measure on $\Omega^2$ by setting

$$\tilde{\mu}(x,y) = \mu(x) \delta(x,y)$$

This naturally can be seen also as the diagonal matrix corresponding to the Hermitian matrix $A$.

Then a random variable on $\Omega^2$ is just a scalar function over $\Omega^2$, which we can easily identify with a matrix $B$, and the expectation value can be computed to be the same as the trace of $AB$.

If you want to encode finite probability spaces in the "quantum" language, just restrict your random variables to those defined on $\Omega$ (so you are in fact looking $\Omega$ included in $\Omega^2$ as the diagonal subset). If you want to encode finite-dimensional quantum mechanics as probability, then after diagonalising $A$ you can extract $\mu$, and in this basis $B$ is just some matrix on $\Omega^2$ that you can identify with $X$.

For the infinite dimensional case some more care is needed: the main idea is to go through the Borel functional calculus; see also this Wikipedia entry on spectral measure.

-
This may be a good answer, however, I don't understand the details :) Can you help me find references, where it is shown how to encode probabilities and random variables in matrices (without tricks?) and where logically the trace is the required operation to get averages. Maybe even a very simple example. In the comments you see how I understand the problem with an easier notation. –  Gerenuk Feb 27 '12 at 16:23