Five years ago to the day, I committed the first lines of code for the CrowdFlower platform. Back then we were a ragtag team of three, some 20k+ commits later we’ve grown to a team of 65 employees, collected over 1 billion judgments, and have had over 5 million contributors complete tasks on our platform.
Today I’m proud to usher in a new era for the CrowdFlower platform. We’ve taken much of what we’ve learned over the last five years and completely re-thought what a modern distributed work platform should look like. The first thing we did was completely overhaul the site’s navigation. Over the years, the number of options, settings, and features has grown. We wanted to introduce a more intuitive flow and smarter default settings.
The primary navigation now lives on the left side of the layout. We’ve separated the various actions into 3 sections: Design Job, Manage Quality, and Get Results. There’s a natural flow for newly created jobs in the system. We’ve also introduced status indicators to the navigation. If we detect a problem we’ll inform you with a red indicator.
Another big change is the organization of the various settings we provide to customize your job. Previously all settings were in a modal on the job edit action, now they’ve been placed in the appropriate sections and given sensible defaults.
Behind the scenes, we implemented a granular permissions system. This allows us to offer advanced functionality to our Pro and Enterprise users on a feature by feature basis. Speaking of advanced features, we’ve introduced job alerts for our Pro and Enterprise users.
Job alerts allow our users to be notified or have actions taken when certain events occur. For instance, if you need to have your job completed by a certain time, we can notify you if the estimated time of completion exceeds your expectation.
One of the key components of the CrowdFlower platform are test questions. We use test questions to educate the crowd as well as to measure their accuracy.
Test questions are now presented in an interactive grid. You can sort your test questions by how often contributors are missing them or by the total number of judgments. You can also filter test questions by the data they contain or their current state. The platform will automatically flag problematic test questions for review, allowing you to disable or modify them. Additionally, a summary of your overall test question health is now a part of our brand new monitoring dashboard.
The new dashboard provides real time insight into the current state of your job. We provide a time series graph of hourly trusted and untrusted judgments. Another new addition is the contributor funnel. This chart shows the total number of contributors that have attempted your task, which have given up or failed, and those currently active. Results from our contributor satisfaction survey are displayed on the dashboard as well. When a contributor exits a task they are given the opportunity to answer a quick survey. This feedback gives task creators insights into how their task is being received and how it can improved.
Last but not least, we’ve created a stellar new support site. The site is packed with great resources such as templates, API libraries, tutorials, and a forum to ensure you’re experience with CrowdFlower is a success.
The last five years have been an amazing journey. We’re just getting started and have many more exciting features and improvements planned for the coming months. None of this would be possible without our amazing team in San Francisco, the tens of thousands of contributors that complete our tasks everyday, and our customers that trust CrowdFlower to deliver them high quality results. Thanks for an amazing five years and I’ve never been more excited for the next five!