Why lidar is so important to self-driving cars

 I asked Gemini (Pro 2.5) "Under what conditions is lidar better than cameras for self-driving cars?"

Of course. Lidar is better than cameras for self-driving cars under specific conditions where its core strengths—direct distance measurement and immunity to lighting issues—outweigh a camera’s advantages in interpretation and cost.

The fundamental difference to remember is:

• Lidar is an active sensor: It emits its own light (lasers) and measures the reflection. This makes it a direct 3D measuring device.
• Cameras are passive sensors: They rely on ambient light (from the sun or streetlights) to capture 2D images. They infer depth and distance using complex software.

Here are the key conditions where lidar is superior:

---

1. Low Light and Complete Darkness

This is lidar’s most significant advantage.

• Why Lidar is Better: Since lidar provides its own light source, its performance is nearly identical whether it’s noon or a moonless midnight. It can build a precise 3D map of its surroundings, detecting pedestrians, curbs, and other vehicles with high accuracy, regardless of the time of day.
• Camera Limitation: Standard cameras struggle in the dark. While they can be improved with software (like long exposure or ISO gain), this often introduces image noise, motion blur, and reduced detail, making it harder to reliably detect unlit objects.

Example: A person wearing dark clothing walking on an unlit road at night is a classic nightmare scenario for a camera-only system. Lidar will detect them as a clear object at a precise distance.

2. Direct Sun and Sudden Glare

Cameras can be easily blinded, just like the human eye.

• Why Lidar is Better: Lidar operates on a very specific, narrow wavelength of light (usually in the infrared spectrum) and uses filters to block out all other light, including sunlight. It is therefore almost completely immune to glare from the sun rising or setting, or reflections off wet roads or other cars.
• Camera Limitation: Driving directly into a low sun can “white out” a camera’s sensor, making it temporarily blind to everything in that direction, including brake lights or a stopped car.

Example: Exiting a dark tunnel into bright daylight. A camera’s aperture needs time to adjust, during which its perception is compromised. Lidar’s performance is unaffected.

3. When Unambiguous, Precise Distance is Critical

Lidar’s primary function is to measure distance directly and accurately.

• Why Lidar is Better: Lidar measures the “time-of-flight” of a laser pulse to calculate distance. This is a direct physical measurement, not an inference. This allows the car to know with centimeter-level accuracy exactly how far away an object is and its precise shape. This is crucial for calculating “time-to-collision” and making split-second decisions.
• Camera Limitation: Cameras must estimate distance. Monocular (single) cameras do this by analyzing the apparent size of known objects (e.g., “that car looks small, so it must be far away”), which can be fooled. Stereo (dual) cameras are better, but they still struggle with textureless surfaces (like a white truck trailer) and have lower accuracy at longer ranges.

Example: A flatbed truck is carrying long pipes that extend past the end of the bed. A camera might only identify the truck bed, failing to correctly perceive the distance to the pipe tips. Lidar’s point cloud would clearly map the pipes as the nearest obstacle, preventing the car from driving underneath them.

4. Detecting Unfamiliar or Unusually Shaped Objects

Lidar creates a “geometric truth” model of the world.

• Why Lidar is Better: Lidar doesn’t need to know what an object is to know that it’s there. It simply maps a collection of points in 3D space. A fallen tree, a construction barrier, or a deer on the road are all detected as solid objects to be avoided, even if the AI has never been trained on that specific thing.
• Camera Limitation: A camera-based system relies heavily on its training data to classify objects. An object it has never seen before might be misidentified or missed entirely.

Example: Debris on the highway after an accident. Lidar will see a cluster of points to navigate around, whereas a camera’s AI might struggle to classify the mangled objects.

5. In Some Adverse Weather (with a caveat)

Lidar can often see through certain types of weather better than a camera.

• Why Lidar is Better: The infrared laser used by lidar can penetrate through light fog, dust, and rain more effectively than visible light can. It provides a more usable signal in conditions where a camera image would be a blurry or obscured mess.
• Camera Limitation: A camera lens covered in raindrops or looking into thick fog sees very little, severely degrading its perception capabilities.
• The Caveat: Lidar is not immune to weather. Heavy snow, dense fog, or torrential rain can scatter the laser beams, creating noise and reducing its effective range. However, it often degrades more gracefully than a camera, still providing some useful data when a camera provides none.

---

Summary Table: Lidar vs. Camera

Condition	Lidar (Better)	Camera (Worse)
Darkness/Night	Unaffected; provides its own light.	Severely limited; relies on ambient light.
Sun Glare	Immune; uses specific laser wavelength.	Can be blinded, causing temporary failure.
Distance Accuracy	Direct, centimeter-level measurement.	Inferred/estimated, less reliable.
Object Shape	Precisely maps 3D geometry.	Can misjudge complex shapes and overhangs.
Novel Objects	Detects anything with physical form.	May fail to identify untrained objects.
Light Fog/Dust	Can often penetrate for a clearer view.	Vision is easily obscured.

Conclusion: The Power of Sensor Fusion

It’s important to note that the debate is not strictly about one being “better” than the other overall. The safest and most robust self-driving systems (like those from Waymo and Cruise) use sensor fusion.

They combine Lidar (for geometric truth), Cameras (for context, like reading signs and traffic lights), and Radar (for excellent velocity detection, even in heavy rain/snow) to build a comprehensive, redundant model of the world. Each sensor covers for the weaknesses of the others, creating a system that is far safer than any single sensor could be alone.