How Google's self-driving cars see the world

Google's self-driving cars are traveling more naturally than they ever have before.

Advertisement

Decked out with GPS, sensors, cameras, radar, and lasers, Alphabet’s (Alphabet is the new parent company of Google) cars are capable of gathering tons of data about their environment from a 360 degree perspective so that they can seamlessly operate in a constantly changing environment.

Google self driving car
Google's self-driving cars are already in a few cities.
Google

According to Google's Self-Driving Car Project website, sensors on the car can detect objects up to two football fields away, including people, vehicles, construction zones, birds, cyclists, and more.

But the data collected by each vehicle does more than allow it to respond in the moment. All of the data each car collects is used to constantly improve the software, so that all cars can learn from one vehicle's experience. 

Given that Google's self-driving cars have driven more than 1.2 million miles in autonomous mode since 2009, the software knows how to react in a lot of different situations.

Advertisement

Chris Urmson, the head of technical development for the project, gave a thorough look at how its cars are operating in real-life scenarios in June during a Ted Talk presentation.

"We can take all of the data that the vehicles see overtime, the hundreds and thousands of pedestrians, cyclists, and vehicles that have been out there and understand what they look like and use that to understand infer what other vehicles should look like and other pedestrians should look like,” Urmson said. 

Here’s a breakdown of how Google's self-driving cars see the world around them and how they are using real-time data to respond to a wide range of scenarios.

Advertisement

Google's self-driving vehicles first establish their location by using mapping and sensor data.

Alphabet Google self-driving car
YouTube/TED Talks

Then it uses its sensor data to understand what it sees in the moment. The software processes all of the data and classifies objects based on size, shape, and movement patterns.

Google self-driving car
Google
Advertisement

In this particular image, the car sees other vehicles, which are represented by the purple figures, a cyclist, which is outlined in red, and some orange cones in the top left corner.

Google Alphabet Self-driving car
YouTube/Ted Talks
Advertisement

But the vehicle doesn’t just need to know about its surroundings, it also needs to make predictions about how other vehicles, people, and objects around it will move. Especially on city streets where there are a lot more pedestrians and there is significantly more traffic.

 

Advertisement

So the vehicle takes its collected data and determine how to make momentary decisions, like the correct trajectory, or whether it should slow down or speed up.

 

Advertisement

The car is programmed to understand and respond to things like the flashing lights of a police car or when a school bus is stopped in front of it.

 

Advertisement

It can also determine when a police officer has halted traffic or when the car is being signaled to move forward.

 

Advertisement

When a cyclist is trying to merge into a lane, the vehicle also knows to slow down and let the cyclist enter.

 

Advertisement

While Google’s cars can anticipate a lot of things by using this collected data, there are still going to be situations that arise that have never happened before.

Alphabet Google self-driving car
YouTube/Ted Talk
Advertisement

For example, Urmson said that one time when one of its test cars was driving through Mountain View, it came across a woman in an electric chair chasing a duck in circles on the road. But the vehicle was still able to adjust and slow down to avoid her.

 

Advertisement

Check out more of what Google's self-driving car sees and Urmson's full presentation below.

 

On February 28, Axel Springer, Business Insider's parent company, joined 31 other media groups and filed a $2.3 billion suit against Google in Dutch court, alleging losses suffered due to the company's advertising practices.

Alphabet Google Self-Driving Car
Advertisement
Close icon Two crossed lines that form an 'X'. It indicates a way to close an interaction, or dismiss a notification.