Bundle adjustment

Bundle adjustment is an optimization technique used in computer vision and photogrammetry, which are two fields that are fundamental for the functioning of rescue robots.

You can think of rescue robots as needing to understand the world around them, just like you and me. When these robots look at a scene through their sensors, which typically include cameras, they need to determine the 3D positions of objects in the scene, as well as the position and orientation of the cameras themselves. This process is a part of what we call "reconstruction."

Now, imagine the robot takes several pictures of a scene from different angles. Each picture gives it a slightly different view. It's like when you move around a room, observing a table from different positions. The shape and location of the table might seem a bit different from each viewpoint, right? This is a challenge for the robot because it needs to reconcile these different views to accurately understand the scene.

This is where bundle adjustment comes in. Bundle adjustment is a mathematical process that tries to minimize the errors in the 3D reconstruction of the scene and the camera positions. It takes all the images and the initial guesses of the camera positions and 3D scene points, and refines these guesses by minimizing the difference between the observed image points and the projected 3D points onto the image planes of the cameras.

So, in a nutshell, bundle adjustment is a sort of "global refinement method" that improves the accuracy of our robot's understanding of its environment, helping it to make better decisions and take more precise actions, which is absolutely crucial in rescue missions.