The Wisdom of the Crowd: Oscar Edition (Part 1)

February 23, 2013

, ,

[Image courtesy of Adarsh Upadhyay]


With just two weeks until the 85th Annual Academy Awards, we wanted to leapfrog off some of the recent Oscar buzz and make some predictions of our own. Unlike film critics, at CrowdFlower we prefer not to demonstrate any semblance of independent thought, opting always for the aggregate un-validated opinion of a lot of people we don’t know. It’s that whole ‘wisdom of the crowd’ mentality, we have and considering we have hundreds of thousands of people ready to provide their opinion at a moment’s notice I’d say you can’t blame us.

To make our predictions we created a task in which we asked people in the United States to a) guess how many Oscar winners they would correctly predict and then b) have them make their predictions. Sure enough, in less than two hours, we had collected over 500 responses from people hailing from 46 of the 50 states. In short, the people had spoken.

Below you can see the aggregate crowd wisdom that comprises CrowdFlower’s official Oscar predictions. Items at the top of the list had more crowd agreement. Our Best Picture pick is toward the bottom of the list, with only 26% of the vote (still more than any other contender.)



Looks like Lincoln is going home on top with five awards, including Best Picture, Best Director, and Best Actor. This, of course, is not surprising as we collected the predictions on Lincoln’s birthday and last Monday was President’s Day. Anything else would be unpatriotic. We were a bit concerned, however, by Anne Hathaway’s clear victory over Sally Field (Lincoln) for Best Supporting Actress in her role as Fantine in Les Miserables. Let us not forget that the French got that whole democratic revolution idea from us.

In order to make sure we were putting our best hand forward, once our Contributors made their initial selections we informed them that they would be bonused for any correct predictions they made – but only if they correctly predicted at least as many as they thought they would. This ultimately didn’t result in any change in the aggregate response, but it did highlight some interesting social dynamics.

Men initially had more confidence than woman in their responses, believing they’d correctly predict on average 1 more winner than women. Men, however, were twice as likely to doubt themselves and change their answers once they found out bonuses were involved. Perhaps women are just more honest from the start or men just more greedy.

And speaking of honesty, it turns out that nearly 30% of our respondents admitted they haven’t even watched a single Oscar nominated film, which begs the question of what wisdom the crowd is really working with in this case with so many seemingly picking at random. (We could have limited our results to people that had seen the movies, but wanted to see if seeing the films would affect people’s accuracy.)

So, as we often do, we’ve decided to put the crowd to the test. Can the crowd beat an expert, as the old saying suggests? And even more simply, can the crowd even beat a selection of winners that were randomly generated? We’ll see after the big show.

Brian Reavey

Brian Reavey

Brian is CrowdFlower's Director of Contributor Operations and Channel Partnerships. From managing the Contributor Support Team to establishing partnerships that expand CrowdFlower’s Contributor base into new geographies, new skill sets, or new languages, Brian is in charge of making sure that the CrowdFlower Contributor community continues to be a growing, happy, and productive part of our data enrichment platform.