I recently updated the homepage of my Kalman Filter tutorial with a new example based on a simple radar tracking problem. The goal was to make the Kalman Filter understandable to anyone with basic knowledge of statistics and linear algebra, without requiring advanced mathematics.
The example starts with a radar measuring the distance to a moving object and gradually builds intuition around noisy measurements, prediction using a motion model, and how the Kalman Filter combines both. I also tried to keep the math minimal while still showing where the equations come from.
I would really appreciate feedback on clarity. Which parts are intuitive? Which parts are confusing? Is the math level appropriate?
If you have used Kalman Filters in practice, I would also be interested to hear whether this explanation aligns with your intuition.
The derivation of the Q matrix is a separate topic and requires additional assumptions about the motion model and noise characteristics, which would have made the example significantly longer. I cover this topic in detail in the book.
I'll consider adding a brief explanation or reference to make that step clearer. Thanks for pointing this out.
Your early explanation of the filter (as a method for estimating the state of a system under uncertainty) was great but (unless I missed it) when you introduced the equations I wasn't clear that was the filter. I hope that makes sense.
In Kalman filter theory there are two different components:
- The system model
- The Kalman filter (the algorithm)
The state transition and measurement equations belong to the system model. They describe the physics of the system and can vary from one application to another.
The Kalman filter is the algorithm that uses this model to estimate the current state and predict the future state.
I'll consider making that distinction more explicit when introducing the equations. Thanks for pointing this out.
I've been in the process of writing a tutorial on how PID filters work for a much younger audience. As a result, I've been looking back at the original tutorials that made stuff click for me. I had several engineers try to explain PID control to me over the course of about a year, but I don't think I really got it until I ended up watching Terry Davis (yeah, the TempleOS guy) show off how to use PID control in SimStructure using a hovering rocket as an example.
The way he built the concept up was to take each component and build on the control system until he had something that worked. He started off with a simple proportional controller that ended up having a steady state error with the rocket hovering beneath the target height. Once he had that and pointed out the steady state error, he implemented the integral term showed off how it resulted in overshoot. Once that was working, he implemented the derivative control to back the overshoot off until he had something that settled pretty quickly.
I'm not sure how you could do something similar for a Kalman Filter, but I did find it genuinely constructive to see the thought process behind adding each component of the equation.
So it is right to say that the implementation of the KF is tightly coupled to the system. Getting that part right is usually the hardest step.
You're right that the term can feel vague without that context. I’ll consider adding a short clarification earlier in the introduction to make this clearer before diving into the math. Thanks for the suggestion.
The challenge would be to keep it intuitive and accessible without oversimplifying. Still, it could be an interesting direction to explore.
1. understand weighted least squares and how you can update an initial estimate (prior mean and variance) with a new measurement and its uncertainty (i.e. inverse variance weighted least squares)
2. this works because the true mean hasn't changed between measurements. What if it did?
3. KF uses a model of how the mean changes to predict what it should be now based on the past, including an inflation factor on the uncertainty since predictions aren't perfect
4. after the prediction, it becomes the same problem as (1) except you use the predicted values as the initial estimate
There are some details about the measurement matrix (when your measurement is a linear combination of the true value -- the state) and the Kalman gain, but these all come from the least squares formulation.
Least squares is the key and you can prove it's optimal under certain assumptions (e.g. Bayesian MMSE).
Open the Sendspin live demo in your browser: https://www.sendspin-audio.com/#live-demo
Some more info on Kalman implementation here https://github.com/Sendspin/time-filter/blob/main/docs%2Fthe...
1. Model of system
2. Internal state
3. How is optimal estimation defined
4. Covariance (statistics)
Kalman filter is optimal estimation of internal state and covariance of system based on measurements so far.
Kalman process/filter is mathematical solution to this problem as the system is evolving based on input and observable measurements. Turns out that internal state that includes both estimated value and covariance is all that is needed to fully capture internal state for such model.
It is important to undrstand, that having different model for what is optimum, uncertenty or system model, compared to what Rudolf Kalman presented, gives just different mathematical solution for this problem. Examples of different optimal solutions for different estimation models are nonlinear Kalman filters and Wiener filter.
---
I think that book on this topic from author Alex Becker is great and possibly best introduction into this topic. It has lot of examples and builds requred intuition really well. All I was missing is little more emphasis into mathematical rigor and chapter about LQG regulator, but you can find both of this in original paper by Rudolf Kalman.
Basically, a Kalman filter is part of a larger class of "estimators", which take the input data, and run additional processing on top of it to figure out the true measurement.
The very basic estimator a low pass filter is also an "estimator" - it rejects high frequency noise, and gives you essentially a moving average. But is a static filter that assumes that your process has noise of a certain frequency, and anything below that is actual changes in the measured variable.
You can make the estimator better. Say you have some idea of how the process variable should behave.For a very simple case, say you are measuring temperature, and you have a current measurement, and you know that change in temperature is related to current being put through a winding. You can capture that relationship in a model of the process, which runs along side the measurement of the actual temperature. Now you have the noisy temperature reading, the predicted reading (which acts like a mean), and you can compute the covariance of the noise, which then you can use to tune the parameter of low pass filter. So if your noise changes in frequency for some reason, the filter will adjust and take care of it.
The Kalman filter is an enhanced version of above, with the added feature of capturing correlation between process variables and using the measurement to update variables that are not directly measurement. For example, if position and velocity are correlated, a refined measurement on the position from gps, will also correct a refined measurement on velocity even if you are not measuring velocity (since you are computing velocity based of an internal model)
The reason it can be kind of confusing is because it basically operates in the matrix linear space, by design to work with other tools that let you do further analysis. So with restriction to linear algebra, you have to assume gaussian noise profile, and estimate process dependence as a covariance measure.
But Kalman filter isnt the end/all be all for noise rejection. You can do any sort of estimation in non linear ways. For example, I designed an automated braking system for an aircraft that tracks a certain brake force command, by commanding a servo to basically press on a brake pedal. Instead of a Kalman filter, I basically ran tests on the system and got a 4d map of (position, pressure, servo_velocity)-> new_pressure, which then I inverted to get the required velocity for target new pressure. So the process estimation was basically commanding the servo to move at a certain speed, getting the pressure, then using position, existing pressure, and pressure error to compute a new velocity, and so on.
Yet virtually all tutorials stick to single-input examples, which is really an edge case. This site is no exception.
Before I got into control theory, I've read a lot of HN posts about kalman filters being the "sensor fusion" algorithm, which is the wrong mental model. You can do sensor fusion with state estimation, but you can't do state estimation with sensor fusion.
Higher sampling rates can help in some cases, especially when tracking fast dynamics or reducing measurement noise through repeated updates. However, the main strength of the Kalman filter is combining a model with noisy measurements, not necessarily relying on high sampling rates.
In practice, Kalman filters can work well even with relatively low-rate measurements, as long as the model captures the system dynamics reasonably well.
I also agree that it's often something you design into the system rather than applying as a post-processing step.
All of that stuff is used in industry because a lot of regulation (for things like aircraft) basically requires your control laws to be linear so that you can prove stability.
In reality, when you get into non linear control, you can do a lot more stuff. I did a research project in college where we had an autonomous underwater glider that could only get gps lock when it surfaced, and had to rely on shitty MEMS imu control under water. I actually proposed doing a neural network for control, but it got shot down because "neural nets are black boxes" lol.
See for example: https://rlabbe.github.io/Kalman-and-Bayesian-Filters-in-Pyth...
Is there something in this particular resource that makes it worth buying?
The book goes further into topics like tuning, practical design considerations, common pitfalls, and additional examples. But there are definitely many good free resources out there, including the one you linked.
if you dont want to buy the book, most of the linear kalman filter stuff is available for free: https://kalmanfilter.net/kalman-filter-tutorial.html