# Sensor fusioning in Kalman filter

I'm interested, how is the dual input in a sensor fusioning setup in a Kalman filter modeled?

Say for instance that you have an accelerometer and a gyro and want to present the "horizon level", like in an airplane, a good demo of something like this here.

How do you actually harvest the two sensors positive properties and minimize the negative?

Is this modeled in the Observation Model matrix (usually symbolized by capital H)?

-

The Kalman filter can be used as is for two separate sensors, just consider a measurement at time $t$ to be

$$y_t = [y^\mathrm{accel}_t, y^\mathrm{gyro}_t]'$$

and write the appropriate sensor noise covariance matrix ($H$). Your levels of confidence of the "good and bad" parts of the sensors, encoded in $H$, will automatically "fuse" the sensor measurements in the sense the Kalman filter will give the optimal linear estimator of the state, given your measurements.

Of course, the farther your measurement model deviates from a linear function with white Gaussian noise, the less successful the Kalman filter will be.

-

I'm not entirely sure of what parameters you're taking as an input, but as far as my experience goes with the H matrix, it is used to focus on a particular input, irrespective of the weightage you intend to give it (I've seen the H matrix in other forms too, but do not have the knowledge to comment on it). What you seem to be looking for, is an algorithm to fuse the inputs of two sensors. The best algorithm available for it is IMM (interacting multiple model). It internally uses Kalman filters, and has steps for estimate fusion and mixing probabilities. A transition probability matrix will help you decide on emphasizing the positive properties and minimising the negative. The models correct each other, which plays a large role in getting a more accurate estimate.

-

Horizon line is $G' * (u, v, f)=0$, where $G$ is a gravity vector, $u$ and $v$ image centered coordinates and $f$ focal length. Now pros and cons of sensors: gyro is super fast and accurate but drifts, accelerometer is less accurate but (if calibrated) has zero bias and doesn't drift (given no acceleration except gravity). They measure different things - accelerometer measures acceleration and thus orientation relative to the gravity vector while gyro measures rotation speed and thus the change in orientation. To convert it to orientation one has to integrate its values (thankfully it can be sampled at high fps like 100-200). Thus Kalman filter that supposed to be linear is not applicable to gyro. for now we can just simplify sensor fusion as a weighted sum of readings and predictions.

You can combine two readings - accelerometer and integrated gyro and model prediction using weights that are inversely proportional to data variances. You will also have to use compass occasionally since accelerometer doesn't tell you much about the azimuth but I guess it is irrelevant for calculation of a horizon line. The system should be responsive and accurate and for this purpose whenever orientation changes fast the weights for gyro should be large; when the system settles down and rotation stops the weights for accelerometer will go up allowing more integration of zero bias readings and killing the drift from gyro.

Finally the Observation Model is typically a model of noise and for gyro it will have little variance and large bias while for accelerometer is would have zero bias and larger variance.

-