By Alexander Sorokin, May 12, 2010

Crowdemployees or Crowdcontractors?

The meeting was extremely interesting. The audio didn’t work, so we had a “not so much Distributed Distributed Work meetup” said Chris. The beer and mingling played substitute for the first talks. I very much hope the talks and discussions were recorded, because ours was so interesting.

After we won an indecisive victory over the projector, Alek Felstiner entertained us with an incredible presentation. He walked us through his thinking of how the outdated law designed for factories and workers could be applied to paid crowdsourcing. The main question is “Who are crowdworkers: crowdemployees or crowdcontractors?”

The laws designed to protect the workers from the power of the factory-like employer have little resemblance to the reality of the emerging crowdsourcing industry. The relation between the requester and provider is highly intermittent. The providers are free to work whenever they like, switch between the requesters at will and choose the type of tasks they like to perform. Many providers participate because they have fun or acquire crucial skills (e.g. learn English).

Alek explained to us quite a list of criteria for employee v.s. contractor determination. There’s no clear cut on those, but some of the criteria made me think where we stand and where we want to stand. In my view we’d want the crowdworkers to be crowdcontractors rather than crowdemployees. As application and platform developers, we only need our tasks done with quality results. We don’t want to deal with employment of thousands of people. In fact, we have neither skill nor resources to manage the employment relations with hundreds of thousands of people.

Here’s the list and a mixture of the points in the discussion and my thoughts:

1) The extent to which the services rendered are an integral part of the principal’s business.

The paid crowdsourcing is a relatively new phenomena, so most of the businesses use it occasionally for very specific data collection projects (+). There are two cases where the existence of the business depends on crowdworkers: platforms (AMT, CrowdFlower, HitBuilder, etc) and crowd-powered vertical applications (e.g. castingwords). The platforms strive to be out of the relationship. They provide services for requesters (who really need the work done) to connect to the providers (who can get the work done) (+). The platforms often provide additional services: task design interfaces, quality control tools, etc. The platforms don’t determine how much the workers should be paid (+). The platforms leave the price bargaining to the requesters and providers (+) and only charge a commission fees.

The second case is vertical services, which provide a specific product, like audio transcription. In this case, the crowd is obviously essential for the service to exist and creates the bulk of the value of the company (-). At the same time, many of these products are infeasible without the present relationship. If one were to require treating the crowdcontractors as crowdemployees, all these companies will promptly shut down (weak +).

2) The permanency of the relationship.

Most of the crowdworkers have no permanent relationships whatsoever (+). They switch between tasks, requesters and maybe even platforms in a mouse click (+). Very few providers strongly prefer a single task from a particular requester(-). Such scenario is more common for dedicated worker or team models (e.g. Samasource). In that case there will be a local partner who employs the actual workers. That setup falls cleanly in the existing framework with clear designation of the provider as the contractor to the requester (+).

3) The amount of the alleged contractor’s investment in facilities and equipment.

What makes crowdsourcing attractive is the lack of barriers to entry(btw, there are no barriers for requester as well). No investment is needed beyond a laptop and an Internet connection (-) (in some places that’s a huge investment).

As the industry is in early development, there was little opportunity for the crowdcontractors to invest in facilities, equipment and better tools (-). That does happen. As we have no industry-wide standards for tasks, work, quality control or user interactions, it’s hard to develop useful interfaces. It will hopefully change in the future as the ecosystem matures.

Some platform policies expressly prohibit using any 3rd party software (-). These policies are strongly motivated by the technological realities of managing hundreds of thousands of work providers. The main use of automated tools would be to spam the system. By blocking the automation one can cut the easiest path for fraud. In my view, we should allow the crowdcontractors to build and refine the interfaces to their liking. As I said before, we don’t care, how the work is done. We just need good results.

4) The nature and degree of control by the principal.

Micro-tasked crowdsourcing has very strong control over how the result should look like, but we absolutely don’t care how the results are obtained (+). The workers can do one task, they can do a hundred. They can open multiple tasks or they can do them one at a time (+). Unfortunately, the workers have little choice over the presentation of the task and the work user interface. The user interface is usually developed by the requester or the platform and the workers have no option to choose a different interface for the task (-).

It is unclear if the dynamic nature of crowdsourcing tasks make it feasible to build custom interfaces for specific tasks. If a particular task requires a couple thousand judgements, the investment of effort into a custom interface won’t pay off (+). For the tasks where the volumes hit hundreds of thousands of judgements, a JavaScript-savvy keyboard-loving worker could add a couple shortcuts and blast through the tasks at 4-5 times the speed of the default interface. He might even sell his interface and work know-how to others (+).

Although this is clearly attractive to the requesters and platform developers, the technological challenges are enormous. First, we’ll need to standardize the work and agree on a common representations of tasks. Second, we’ll need to run untrusted code or direct the work in some common format to the new interface (these are actually easier than they sound). Thirds, we’ll need to manage fraud and make sure that the 3rd party interfaces don’t compromise the overall system security and reliability. We actually depend quite a lot on the independence of individual workers and the difficulty of random collusion. If all workers were to go through a single 3rd party interface, we’d have to be extra careful.

5) The alleged contractor’s opportunities for profit and loss.

As of now, there are virtually none (-). The crowdworkers risk with their work only to the extent that the requesters can reject poor work(-). Some platforms tie the quality of the work to the pay level (+). The nature of pay-per-unit interaction makes it difficult for the requesters to setup profit-or-loss agreements. I’m sure requesters would love to have turn-key interactions, where they hand off all their data in large chunks to crowdcontractors and pay per chunk. These contractors would be financially responsible for the timely results, for the quality of the work and so on. They would be free to crowd-sub-contact the work and perform the necessary managerial functions. Unfortunately, we don’t presently have the technology to do this.

6) The amount of initiative, judgment, or foresight in open market competition with others required for the success of the claimed independent contractor.

In the current model, the only judgement the providers can make is whether they want to work on a particular task (-). They do accumulate skill, knowledge and reputation that enables them to do more, but it doesn’t look like they can make strategic decisions (-).

7) The degree of independent business organization and operation.

The crowdcontractors are absolutely independent and perform their services at the time they choose (+). They are also responsible for all their costs (i.e. for their laptop and the Internet connection). In some cases (as in Samasource), the providers even manage their space, hire workers and provide them with training (+). Those clearly satisfy the contractor definition.

That was a long list and a lot of text. So where, precisely, do we stand? I think we are planted quite firmly in the gray and it’s well beyond us to decide.

I believe we want to treat the crowd as crowdcontractors. From that we should look at our systems and policies and develop them to support our standing. We should give even more freedom to the crowdcontractors in choosing the means and the ways to complete the work. If they develop more efficient tools to complete the tasks, we win. If they train workers to deliver 100% quality results, we win big.

(+) are positive factors for the treatment of workers as contractors

(-) are the negative factors

Alexander Sorokin,
University of Illinois at Urbana-Champaign