Last week we did a quick survey of Amazon Mechanical Turk workers (colloquially known as Turkers). Next it seemed natural to do the same survey of workers on Gambit, another workforce to which we channel work. This analysis of Gambit is interesting in and of itself because it permits comparison between our two largest workforces. Our work on Gambit, however, is especially fascinating for a number of reasons. The volume and completion rates are comparable to those of Turk, but workers on Gambit are working for virtual currency in online social games like Facebook's "SportsBets." Virtual currency can be redeemed in games and on sites like Swag Bucks. Rarely does virtual currency translate into actual money. This phenomenon is amazing, and we are consistently struck by the innovative nature of our partnership with Gambit.
Another interesting thing about the Gambit workforce is how similar its composition is to the Turk workforce of two years ago, in which 80 percent of workers were from North America (the United States and Canada). (The exact results and methods for determining this is discussed below). This similarity points to how labor workforces are globalizing very quickly with Turk (one of the first marketplaces for online work) at the forefront of this growth. Gambit could possibly be moving in this direction as well. For now we'll leave this discussion to a later blog post, as first we must see the results to this survey.
Survey Methodology and Results
Though we ran the same survey, asking the same questions, we reworded some questions for the Gambit workforce, specifically:
- Educational level
- Income level
- Marital status
- Questions about their engagement on Gambit
- How often they do tasks
- Income from tasks
- Why they do tasks/other comments
Again, we ran the survey over a 24 hour period and used the same script to make surveys available uniformly. This worked well—responses came in at about the same rate, 50 responses an hour, as in our Turk survey. As we did in our Turk survey, we must note possible confounding factors and selection bias, i.e. groups who work on weekends as opposed to during the week, workers who do not participate in surveys. In fact, because Gambit provides offers through online games, a strong selection bias against surveys exists, as opposed to other tasks, because they may be more closely associated with product spam, which is universally abhorred.
As we did with our Turk survey, we will focus on where Gambit workers come from, but because scarcely anything is written on who Gambit workers are, here are some graphs.
A very large majority of Gambit workers self-reported that they were from the US, 83.27 percent, while another 6.57 percent said they were from North/South America. When geocoding the users' IP addresses we found that 85.23 percent of responses came from the US and 8.53 percent from Canada. The other countries represented in this geocoding process were Australia, England, Indonesia, Singapore, Malaysia, and the Philippines. In contrast, on Turk we saw a much larger set of countries represented in our survey. It is interesting that the Gambit distribution resembles that of Turk from two years ago, where 80 percent of workers were from the US, UK, and Canada.
We might expect a skew towards English speakers because they are one of the largest target audiences for virtual currency offers. However, this survey is in English and, hence, self-selects people from large English speaking countries. We have run non-English language tasks before on Gambit and have noticed a majority of those workers came from regions which spoke the language, particularly France. This confounding factor exists for our Turk survey as well, as we have noticed that our German Turk tasks almost exclusively consists of workers from Germany. These tasks are run without restrictions so anybody can do the work, but we only considered workers who met our quality standards and were not rejected.
The best way to find out worker locales' may be to ask Gambit and Amazon to track and share this information directly. Barring this, running a "free money" task is our best bet to attract as wide an audience as possible and then geocode IP addresses to determine workforce distribution across countries.
Here is the gender breakdown:
As we can see Gambit workers are mostly female. This may be due to where these tasks presented i.e. next to advertisements and offers in online games. However, we cannot say for sure that this is the reason.
This next graph shows distribution of education level.
Lastly, we also asked the workers why they do our tasks and their responses fell into a number of categories:
- Boredom, to kill time
- To earn extra money for hobbies, fun days, etc.
- Earn online currencies (e.g. Swag Bucks)
- More productive use of time as opposed to say TV, video games
- They are fun to do
Gambit workers shared the following comments about being bored and wanting to feel productive with their time:
- "My husband and I work at the same place. I just answer phones and babysit kids when their parents are buying a piano from my husband. I do these to keep from being bored to death."
- "I figure that if Im just sitting around the house watching TV, I might as well try to do something productive"
- "I find this a fun way to kill time - there interesting and i love earning swagbucks"
- "i just like to keep bussy all the time!"
Users really love their Swag Bucks, too:
- "I get bored and I have nothing else to do, or I would like to earn some more Swagbucks."
- "These tasks help me to add swagbucks to my account and I actually get credited for them whereas the special offers never seem to acknowledge that I've completed an offer and will not give me my earned bucks."
- "Swagbucks offers some nice magazine subscriptions and I love summer reading for free."
- "I like to see my swagbucks account growing"
- "I do tasks as a way to try and earn extra Swag Bucks."
- "I do these tasks for Swag Bucks. So, I receive no actual compensation."
- "i do them to earn swagbucks. sometimes i try to get as many SBs in one day as i can. tasks make a great filler. i never do the grouping tasks. i think they need better instructions, or maybe it's just one of those things that other people are better at than me."
Earning Swag Bucks is a huge motivating factor which we cannot underestimate. In fact when we compare the quality of work from Gambit to work from Turk on HITs where we know the answer, we get about the same ratio of correct answers to wrong answers (about four to one in both cases). We use mistakes to help train workers by telling them why they got something wrong so they can avoid making such mistakes on future HITs. For Swag Bucks, people do correct work so they do not get kicked out of our system. As Labor-on-Demand continues to grow we need to understand how to best incentivize workers, possibly with Swag Bucks, so we can get more work done for meaningful tasks.
Lastly, we noticed how similar Gambit's workforce was to Turk's workforce from a two years ago (based on Panos's previous survey). These trends in the evolution of workforces point to the rapid growth and globalization of the industry. We'll have to watch these trends closely to see how crowdsourcing/Labor-on-Demand/Labor-as-a-Service continues to evolve/develop.
In a later blog post, we'll also take a look at Samasource's workforce.