# Kalman Filter -- handling large covariance matrices with principal-component-like structures

I understand that the estimation of the covariance matrices are the important part of the Kalman filter. However in my use case my covariance matrices are really big but with a pretty neat factor model (PCA-like) structure. Borrowing from Wikipedia convention, what I mean is that our $$Q$$ and $$R$$ matrices can be written as some sort of reduced form $$Q =B\Sigma_{Q}B^{T}+\Sigma_{qq}$$ where $$\Sigma_{qq}$$ is a diagonal matrix and the dimension of $$\Sigma_{Q}$$ is significantly smaller than the dimension of $$Q$$ and similarly $$R =B\Sigma_{R} B^{T}+\Sigma_{rr}$$

For simplicity we can even assume $$\Sigma_{Q} = c \Sigma_{R}$$, $$\Sigma_{qq} = c \Sigma_{rr}$$. i.e they are scaled by a constant.

Different from PCA or SVD, $$\Sigma_Q$$ or $$\Sigma_R$$ are not diagonal matrices, as in different "factors/components" have correlations, but it's enough to reduce the complexity of $$R$$ and $$Q$$ by quite a bit.

Now, can the "reduced form" of the covariance matrices be carried into the rest of the update steps (again borrowed from wikipedia), such as $$P$$? If I still have to operate on a full $$P$$, it'd defeat the purposes of the reduced forms I developed for $$R$$ and $$Q$$.

We can further assume $$F=I$$ in my use case, if that helps, but not much we can assume about $$H$$. If necessarily we can also assume I have no observation noise (i.e. $$R$$ is 0). I'd like to have a more general case without making assumption on $$F$$ or $$R$$ if possible for my own intellectual curiosity.

\begin{aligned} \mathbf {P} _{k\mid k-1}&=\mathbf {F} _{k}\mathbf {P} _{k-1\mid k-1}\mathbf {F} _{k}^{\textsf {T}}+\mathbf {Q} _{k},\\ \mathbf {S} _{k}&=\mathbf {H} _{k}\mathbf {P} _{k\mid k-1}\mathbf {H} _{k}^{\textsf {T}}+\mathbf {R} _{k},\\ \mathbf {K} _{k}&=\mathbf {P} _{k\mid k-1}\mathbf {H} _{k}^{\textsf {T}}\mathbf {S} _{k}^{-1},\\ \mathbf {P} _{k|k}&=\left(\mathbf {I} -\mathbf {K} _{k}\mathbf {H} _{k}\right)\mathbf {P} _{k|k-1}. \end{aligned}

• Are you assuming nothing about the system dynamics? Specifically, are there no assumptions on the jacobians $F_k$ and $H_k$? Commented May 16 at 18:26
• @RollenS.D'Souza added an edit. We can assume $F=I$ in my use case, but not much we can assume about $H$. Commented May 16 at 18:40
• If $F=I$, then you need the rows of $H_k$ form a linearly independent basis for $\mathbb{R}^n$ for some time interval $k=r,r+1,\dots,f$. Otherwise, you will lose observability in this time interval and the state estimation will not be unique. Also, it can cause all sorts of numerical issues with the Kalman filter if not implemented properly. The formulas given in Wikipedia are not numerically stable. Commented May 17 at 9:12
• You might want to look at ensemble Kalman filter or local ensemble transform Kalman filter. Commented May 20 at 16:16