By Ram Rampalli, August 2, 2011

Crowdsourcing Thought Leadership: Building a successful portfolio of crowdsourcing projects (Part 4)


KPIs Crowdsourcing

This is part of a series of guest posts by Ram Rampalli, our crowdsourcing partner at eBay.
Part I – Assessment Stage
Part II – Pilot Stage
Part III – Analysis Stage
Part IV – Production Stage

About the author: Ram Rampalli created and leads the crowdsourcing program within the Selling & Catalogs team at eBay Inc. You can follow him on Twitter (@ramrampalli)

Building a successful portfolio of crowdsourcing projects – Part 4

In the first three parts of this series, we discussed the Assessment, Pilot, and Analysis & Optimization stages. Now that the task is moved to production, what steps can you take to manage this effectively?

1. Leverage the CrowdFlower API

When you move to production, the task is already designed, but you still need to provide input data. If you want the crowd to perform a product categorization tasks, you need to provide the crowd with the product and category information.

At this point, while it is not required, I recommend integrating with the CrowdFlower API. Integrating with the CrowdFlower API allows you to send and receive data seamlessly. This allows you to get your data back as it’s completed without waiting for batch results in the form of a CSV.

2. Identify the KPIs

With the success of our early projects here at eBay, we quickly added new projects and soon had seven concurrent projects running with CrowdFlower. With this type of volume, it is impossible to track every key metric for each project all the time.

I recommend ranking the top three key metrics and identifying them to the CrowdFlower team. This will give the CrowdFlower team targeted metrics for which they can optimize.

Closing Remarks

Crowdsourcing is a relatively new paradigm. Different companies have implemented this paradigm in different ways, focusing on different ways of structuring their interactions with the crowd. So just because a project did not work under one model does not necessarily mean that the project will never work.

These companies are also constantly innovating, coming up with new products and solutions. Over the last 18 months we’ve worked closely with the CrowdFlower team to develop features that are now staples of their platform.

For me, every project has been a learning experience, and as I work on newer projects, I have lots to learn. I hope you enjoyed this four-part series. I would love to hear your thoughts and comments. Happy crowdsourcing!