Research & Insights

By Lukas Bergstrom, December 19, 2012

The Accuracy of Apple Maps Listings: Reality Check

Update: this piece has been featured on Mashable.

map pin

Apple Maps, which replaced Google Maps for all iPhone and iPad users in the most recent version of iOS, has been receiving a lot of attention. Things died down after a while after Tim Cook’s apology, but flared up again last week when travelers in Australia were stranded when they were directed into the Outback. While we all enjoy taking potshots at the cool kids, as data nerds, it bothers us that all we’ve had to go on in the Apple Maps discussion is anecdotal evidence. So we decided to assemble a dataset that would let us take potshots with authority give some solid answers.

Apple Maps has been criticized for a few things, from how it interprets your search to the directions it gives. We asked a simpler question: when you ask for a restaurant, hotel or other business, how often do you get the right location?

Extremely Brief Summary

Apple Maps in the US is bad enough to be noticeable: you probably won’t throw away your iPhone, although you may miss a dinner reservation. Those of you using Apple Maps in the UK, however, might want to keep emergency food and water in the car.

For more detail and statistics, read on …

How We Did It

We started with a list of 1,000 US businesses in our database, then added 100 UK businesses to give some idea of international differences. We had our crowd find the official websites of these businesses and extract current address information from the website. This was our reference data.

We had the crowd pull the same information from Apple Maps, Google Maps, and Bing. In order to replicate the search experience of a typical user, we had people search for “business name, city” first before trying different variations.

Then, we compared the results. In the case of major errors we investigated carefully – many UK addresses lack street numbers and can be described multiple ways.

Results

Our results have two parts: first, if I search for a business, will I get a result? Second, will that result be accurate?

Percentage of businesses found

Percent of businesses found in the databases of three search providers

First we tested how many of the businesses we could find on each service. In results that will surprise no one, Google Maps has listings for the most businesses in the US and UK, with 89% coverage in the US and 91% coverage in the UK. Apple Maps is credible in the US with 74% of businesses found, but with 47% coverage in the UK the phone book starts to look like a real option. Bing is somewhat better than Apple but not great, with 79% coverage in the US and 57% coverage in the UK.

Not being able to find a business is one thing. Getting an incorrect listing might be even worse.

Percentage of listings with major errors

Graph of error rate of business listings for three map providers

We consider a major error to be anything that puts you a block or more from your intended destination. With a 3.4% major error rate in the US (compared to 1.1% for Google Maps and 1.3% for Bing) there’s a decent chance Apple Maps will send you in the wrong direction. Even small errors can be frustrating: witness the difference between the Apple Maps result for Nick’s Crossroad Cafe in Albuquerque with the actual result. (We’re using a Google Maps screenshot for convenience, but our address data comes from Nick’s official website.)

The difference between Central Ave NW and Central Ave NE
Map showing distance between Apple Maps result and actual result

In the UK, not only are the more incorrect listings, but you might find yourself even further off-course. Here’s the Apple Maps result for Advance Gym in Reading, UK compared to the actual result. You’ll also notice that the Apple Maps version is missing a lot of detail compared to the Google Maps version.

About four miles

Map example showing distance between Apple Maps result and actual result

Note that we tried to be forgiving in making our evaluation: for example, if Apple Maps had multiple listings and at least one was valid, we accepted that as valid.

Will I get where I want to go?

So when you search for a business, what are the chances that you’ll find the business you want AND the address will be accurate?

With Apple maps you have a 71% shot, compared to 88% for Google Maps and 78% for Bing. But with a 3.4% error rate, you’re three times as likely to be sent on a wild goose chase with Apple Maps.

In the UK, the situation is dicier: you’ll get a good listing 33% for Apple Maps, compared to 88% for Google Maps and 55% for Bing. And 30% of the time, you’ll get a listing, but it’ll be incorrect.

Local data is hard. Small businesses are closing and starting up all the time. Streets are being re-routed. And Apple is just getting into this game. We’ll be very interested to see how Apple’s data improves over time.

Want to know more about how CrowdFlower handles business data? Get in touch or read about how we provide high-quality SMB data for sales and marketing.