What is visual SLAM? Visual SLAM is a method that lets a robot estimate its camera pose and build a map of its surroundings at once from camera images, which makes it useful indoors where GPS is unreliable.
It is best understood as a whole pipeline rather than a single algorithm. This guide sits within our wider coverage of robot vision systems and walks through the stages, the sensor choices and a sensible way to start experimenting.
What visual SLAM does and why robots need it
Visual SLAM solves a chicken-and-egg problem. A robot needs a map to know where it is, and it needs to know where it is to build the map. Camera-based systems tackle both together, which is especially useful where GPS is unreliable or unavailable, such as indoors. The same source names ORB-SLAM and DTAM as commonly used visual SLAM algorithms.
The camera is the defining sensor. A survey in the peer-reviewed literature reports that visual SLAM covers monocular, stereo, event-based, omnidirectional and RGB-D cameras. The same survey reviews fifty impactful articles, so the field is broad, but for an indoor wheeled robot the practical choice usually comes down to a monocular camera, a stereo camera or an RGB-D camera.
One reason to choose a camera is hardware economics. The survey notes that visual data brings cheaper hardware requirements. That advantage matters most on small indoor platforms with tight cost and weight budgets.
How the pipeline works, stage by stage
Visual SLAM runs as a loop of capture, tracking, mapping and correction. The stages below follow the order in which a frame moves through a typical system.
- Capture images. As the robot moves, it keeps capturing images and updating the map.
- Extract keypoint features. Indirect methods extract feature points, or keypoints, from textures in the scene. These points are matched from frame to frame so the system can see how the camera moved.
- Track and map in parallel. Surveyed SLAM approaches mainly use two threads executed in parallel, tracking and mapping. Tracking estimates the current camera pose, while mapping grows and refines the 3D structure those keypoints describe.
- Close loops. Loop closure means recognizing previously visited locations so tracking and map reconstruction become more precise. Without it, small errors accumulate as the robot drives around a building.
ORB-SLAM is a clear example of this design. According to a university write-up, it uses the same ORB features for all three threads it initializes, and it restricts the number of map points to allow real-time computation. Sharing one feature type keeps the system compact, and capping map points keeps it fast enough to run live.
Where visual odometry fits
Visual odometry and visual SLAM share the same front end, which is why they are often confused. The survey puts the primary distinction as whether the global consistency of the map is taken into account. If you only need frame-to-frame motion, read what is visual odometry. If the robot must return to places and keep a consistent map, you need the full SLAM pipeline with loop closure.
Monocular versus stereo SLAM
The main difference is depth. A monocular camera is simple and cost-effective but, as a sensor overview puts it, lacks scale information. It also requires movement to estimate depth. Stereo adds a second view, but at a price in setup effort and computing.
| Criterion | Monocular camera | Stereo camera | Planning use |
|---|---|---|---|
| Depth and scale | Lacks scale information; needs movement to estimate depth | Depth range limited by baseline and resolution | Prefer stereo if the robot must know true distances from the first frames |
| Setup effort | Simple and cost-effective | Complex calibration required | Prefer monocular for a first prototype or a tight budget |
| Computing load | Lower than stereo | High computational power | Check your onboard computer before committing to stereo |
These comparisons come from the visual SLAM overview by MRDVS.
Monocular suits you when cost and simplicity matter and the robot moves enough to let the system recover depth. Stereo suits you when scale matters and you can afford careful calibration and a stronger processor. Because monocular scale is ambiguous, a map from a single camera is not in real-world units unless something else supplies the scale. A stereo rig or an RGB-D camera avoids that, at the costs shown above.
Visual SLAM versus lidar SLAM indoors
Visual SLAM is the lighter and cheaper option, while lidar SLAM remains the other main route for indoor robots. The survey says visual SLAM is sometimes preferred to LiDAR due to lighter weight, lower acquisition costs and richer environment representation. Those three points drive most small-robot decisions.
There is a limit on the camera side, though. The same overview cautions that purely visual navigation systems are not feasible at present and need to be combined with other sensors. In practice, treat a camera-only design as a starting point, not a finished navigation stack.
- Choose visual SLAM when payload weight and sensor cost are the binding limits, or when you want images that carry richer information than range scans.
- Add or choose lidar when you expect to rely on one sensor for navigation, since camera-only systems usually need a partner sensor.
- Combine them when the robot is more than a prototype. The comparison is less either-or than it first appears.
Richer camera data also opens the door to understanding what is in view, not only where surfaces are. Our piece on semantic segmentation robotics is a natural next step once mapping works.
Benchmark datasets for testing
Public datasets let you judge a visual SLAM system without building your own test route. The survey notes that VSLAM methods are tested on various datasets, for example TUM RGB-D or EuRoC. TUM RGB-D fits indoor RGB-D work, which matches the typical indoor robot. KITTI also appears in practice: a collaborative ORB-SLAM2 repository configures its example around KITTI sequences.
Pick the dataset to match your sensor. Evaluating a monocular pipeline on RGB-D data, or the reverse, tells you little about the robot you intend to build.
Getting started with ORB-SLAM
ORB-SLAM is the usual place to begin. The original paper is ORB-SLAM: A Versatile and Accurate Monocular SLAM System, published in IEEE Transactions on Robotics, 31(5), 1147–1163, 2015, and described by a university write-up as state of the art at the time for its use of bundle adjustment. That write-up also describes it as a system for monocular cameras that generates entire 3D maps from basic video footage.
Later ORB-SLAM2 code extends this. The repository mentions standalone monocular, stereo and depth examples, so one codebase lets you compare sensor modes on your own data. A sensible path looks like this:
- Read the paper first. It explains the design choices before you meet them as build errors.
- Pick a repository and read its notes. The collaborative fork states that the code comes without any warranty and no guarantee to work in your setup, so treat it as research code.
- Match the operating system and ROS version. The Carleton write-up observes that newer distributions such as Ubuntu 24 only support ROS2, which can rule out older ROS-based ports.
- Build with the provided script. The repository supplies build.sh to build the Thirdparty libraries and ORB-SLAM2.
- Run a public dataset before your own robot. A known sequence tells you whether problems come from your build or from your camera.
Frequently asked questions
Why does ORB-SLAM limit the number of map points?
It restricts the number of points so the system can keep up in real-time computation, according to a university analysis of the algorithm.
Which Ubuntu version should I use for the collaborative ORB-SLAM2 fork?
The fork’s authors report testing on Ubuntu 14.04 and 16.04. Newer releases may need adjustments, and ROS2-only distributions such as Ubuntu 24 can complicate ROS-based ports.
Can ORB-SLAM work from ordinary video?
The original system targets monocular cameras and is described as generating entire 3D maps from basic video footage. Scale remains ambiguous with a single camera, so the map is not in real-world units by itself.
Do published demos match what ORB-SLAM claims?
A Carleton student project reproducing the demo footage reported similar results to the claimed performance. That is encouraging, but results on your own camera and environment may differ.
