Research & Insights

By David Lacy, May 22, 2013

Radiology and Crowdsourcing

Recently, I’ve witnessed a rise in the number of crowdsourcing jobs targeted at accomplishing tasks for the medical industry. Every once in a while I’ll be fortunate enough to run across an exceptionally intriguing job that makes use of crowdsourcing in a really unique way.

Antonio Foncubierta’s crowdsourcing job is one of those unique tasks I happened to stumble upon. Antonio works for the Business Informatics Institute at the University of Applied Sciences Western Switzerland (HES-SO). The job he created gives the crowd the task of categorizing medical images, such as x-rays, PET scans and CAT scans. Here’s how Antonio describes the problem:

The problem with medical images is that they are produced in vast quantities everyday30% of the world storage capacity is used by medical images. Retrieval and analysis are quite challenging. In order to train our models and computer-based systems, ground truth is necessary, but it also requires a lot of manual work and time to obtain. Therefore, we thought of using crowdsourcing as a way to obtain quickly basic ground truth.

The job first displays a link to a page, educating the contributor by clearly distinguishing between different types of medical images as an example. The crowd contributor will then view a number of images on the task, similar to the one below.


Example images from Antonio’s radiology job. Starting from the top left and moving clockwise, this page shows Magnetic Resonance, Ultrasound, 2D Radiography, and Computer Tomography.

So what are some of Antonio’s tips on conducting a successful crowdsourced job?

I think that crowdsourcing is a huge opportunity for researchers, when repetitive tasks need to be performed. However, it is extremely important to have good methods for assessing the quality of the judgments in order to use all the potential of crowdsourcing.

It’s a creative approach to a pressing problem. This job provides insight into the benefits that can be derived from crowdsourcing, while also demonstrating the close and often intertwined relationship technology and medicine share. With the exponential increase in medical data being generated, crowdsourcing is poised to be the next step in overcoming some of the most crucial obstacles in the medical domain.

You can read more about Antonio Foncubierta’s research findings at the links below:

  • Antonio Foncubierta -Rodríguez and Henning Müller, Ground truth generation in medical imaging: A crowdsourcing based iterative approach, in: Workshop on Crowdsourcing for Multimedia, ACM Multimedia, Nara, Japan, 2012
  • Antonio Foncubierta -Rodríguez and Henning Müller, Crowdsourcing opportunities in medical imaging (2013), in: IEEE Communication Society letter