By Ari Klein, March 26, 2014

When Should You Trust a Tweet?

I’m constantly floored by the innovative applications that CrowdFlower can be used for, here’s one in particular I want to share.

CERC logo


Assessing the Credibility of Tweets

I recently had the chance to speak with Aditi Gupta, a Ph.D. student at the Indraprastha Institute of Information Technology (IIIT), Delhi. She is conducting research as part of the Cybersercurity Education and Research Centre (CERC). Aditi’s research is aimed at assessing the credibility of tweets (more here).

Accurate Information During a Crisis

Of particular interest to the research is the credibility of tweets during a crisis – when real-time updates can often be the only way of knowing what the latest situation is on the ground and if people’s loved ones are safe. When there’s a new trending topic on Twitter, newsjackers and opportunists often crawl out of the woodwork looking for a few more followers. These, and other less trustworthy sources, add noise that can drown out the credible information.

Using Machine Learning to Determine Credibility

It’s this challenge that Aditi is tackling. She wants to help the world understand how much weight to put behind a particular tweet during a crisis situation. She uses CrowdFlower and our global network of workers to create huge volumes of training data for her machine learning model. In short, she is training her algorithm using the ground-truth credibility ratings of real people en masse. The idea is to use this algorithm as a basis for a plugin that will automatically provide a credibility score for any tweet.

Real World Applications – Discrediting Misinformation

With this sort of credibility checking tool, it’s easy to imagine the misinformation of Sunil Tripathi’s involvement in last year’s Boston Marathon Bombing falling flat. Instead, it cascaded quickly across in the Internet making an already difficult situation notably worse. And there are many other examples of misinformation spreading quickly, where a credibility ranking could be useful (Hurricane Sandy and the UK Riots). Of course, I would be remiss not to mention that there are times when even those most credible sources and institutions we trust to report the facts get it wrong too.

You can find Aditi on Twitter here: @aditigupta2010.