Kalman filter, intuitively I am currently working my way through the Kalman filter equations. Be warned, I think do have a solid understanding of math, yet I am just an engineer.
So first of all there is this excellent website that provides quite an easy to follow introduction to the matter. I get along with it really well but there is one caveat that really gives me headaches. Consider the multidimensional expression for the covariance extrapolation:
$$ \mathbf P_{n+1,n} = \mathbf F \mathbf P_{n,n} \mathbf F^\top + \mathbf Q $$
Note the scheme $ \mathbf A \mathbf B \mathbf A^\top $ that occurs in other occasions again.
How could one describe this intuitively? For comparison, in $ \mathbf A \vec x = \vec b $ my mind intuitively grasps $\mathbf A$ as a recipe to make the components for $\vec b$ from the components of $\vec a$ on the lowest level. A little elevated from that I see $\mathbf A$ as a function/projection/...
But I just cannot come up with something mnemonic for the above scheme, and I'd be thankful if someone could shine a light.
 A: This scheme in the particular context of Kalman filter comes from the Gaussian assumption and linearity of the system. Basically you know your previous state's distribution, i.e. $p(x_{n,n}) = N(\hat{x}_{n,n}, P_{n,n})$, where $N(m,P)$ represents Gaussian distribution with mean $m$ and covariance $P$, and you want to "predict" the next state using your dynamical system model.
The system model describes a conditional probability with PDF $p(x_{n+1,n}|x_{n,n})=N(A x_{n,n}, Q)$. This means if you knew the previous state exactly, you could have predicted the distribution of the next state since you know the statistical properties of the model error.
Now to calculate the distribution of the predicted state you need to multiply the probability of being in a state with the conditional probability for every possible previous state and sum all the results, i.e.
$$ p(x_{n+1,n}) = \int_{-\infty}^{\infty} p(x_{n+1,n}|x_{n,n}) p(x_{n,n}) d x_{n,n} = \int_{-\infty}^{\infty} N(A x_{n,n}, Q) N(\hat{x}_{n,n}, P_{n,n}) d x_{n,n}$$
If you work out this multiplication using the Multivariate normal distribution density function, it is not hard to get the result:
$$ p(x_{n+1,n}) = N(A \hat{x}_{n,n}, A P_{n,n} A^T + Q)$$
which is precisely the prediction step equations of the Kalman filter.
As for the intuition for the scheme $APA^T$ in this context, it is just square nature of the covariance, e.g.
$$E[(Ax)(Ax)^T] = E[A xx^T A^T] = A E[xx^T] A^T = APA^T$$
The same goes for also the discrete Lyapunov function, LQR problem and related Riccati function and so on.
