This week we continued system development, with a special focus on solving a particular problem. Within our system architecture, we have implemented a deep learning model that detects and tracks small fast-moving objects like a tennis ball.

However, this model is not perfect, and the task of tracking a tennis ball is quite challenging. Frequently, a player’s bright-colored shoes, or the corner of the court lines can be detected as the ball, which throws outliers into the data for ball coordinates at multiple time steps. This type of outlier removal can be resolved in many ways. We decided to leverage my mechanical engineering background to implement a Kalman filter for smoothing the coordinates. A Kalman filter uses a physics based model to predict the location at the next time step for an object that is being tracked. We hoped that this would allow us to remove ball coordinates that would be improbable. After setting up the equations and model and coding it, I tested the model on our ball coordinates. While the model did a fantastic job of smoothing, we encountered a new problem. The TrackNet model would sometimes misdetect the ball for 5-10 timestamps in a row consistently. The Kalman filter assumes that the first timestamp is low probability, but after 2-3 timestamps at the misdetected location, the Kalman filter assumes this location to be the ground truth rather than a consistent misdetection, and thus, the smoothing performance decreases.

As a result, we opted to replace the Kalman Filter for a more traditional moving window smoothing, which had higher performance, as it does not seek any particular physics-based motion, and rather, smooths everything equally. With a large enough window, we found this to perform quite well for our task. We compared the performance of the Kalman Filter and the window smoothing by producing three sets of coordinates, one set that was smoothed by the Kalman Filter, one set that was smoothed by the moving-window, and one set that remained unsmoothed. We pushed these coordinates into our hit-detection algorithm and found that the rolling-window most consistently predicted hits correctly, and thus we selected this smoothing method for implementation within the system.