Contributors who’ve tasked on CrowdFlower are familiar with this experience (which has stayed the same for the last few years):
At the beginning of 2014, we decided to enhance the taskwall by making it easier for contributors to find tasks that match their interests and by showing them all of the exciting jobs they would be able to access if they level-up in our system.
Because it’s so heavily trafficked, we knew we couldn’t merely throw the switch on a new design overnight. Instead, I introduced a first at CrowdFlower: use of A/B Testing to determine a data-driven design that would perform as well as (if not better than) the original.
Today we’re proud to announce that the new taskwall experience has been rolled out:
As you can see, we’ve added:
– sorting-by-rewards/satisfaction so that contributors can more easily find the best tasks
– skills information per job via Level badges
– filtering by potential jobs to emphasize to contributors that if they level-up they get access to better jobs
A Behind-the-Scenes Look
Following, I’ll walk you through a high-level overview around the roll-out.
We knew we wanted the revised experience to be snappy, but completely replacing what was a server-side rendition with a client-side Single Page Application was far out of the scope to begin, particularly as there were substantial infrastructure changes necessary to retrofit the taskwall for A/B Testing.
I did due diligence in vetting several A/B Testing options – coming up with a review of frameworks for Ruby-on-Rails – and finally made a choice for Vanity. Once that decision was made and the technology in place, we could begin iterating on the design, knowing with (measurable) confidence how we were impacting the user experience.
In our first test, we pitted control against a bare-bones improvement.
|control||‘new’ : first iteration, rendered server-side|
The new version out-performed control (in terms of clicks) 21.3% vs 20.3% (at 95% confidence) so I continued to iterate, coming up with the next several versions:
After several more iterations, we found that the new version was consistently out-performing control 22.2% vs 20.7% (at 99% confidence) so a decision was made to move full-steam ahead. We’ve now rolled out the new experience – below with welcome modal – to 100% of all contributors.
our new taskwall
A Final Word
It’s wonderful to be part of an initiative that is going to impact so many contributors lives in such a positive fashion, especially as we get them access to better jobs and help them establish their reputations in the CrowdFlower community.
As I wrote before, we at CrowdFlower are scratching the surface as how works gets done now and in the future as we empower contributors with great interfaces.