Builder Survey Tutorial – Part II

June 21, 2013

Today, we’re going to go over part two of building a survey on CrowdFlower. We will cover the settings required in Advanced Options, on the calibrations page and on the channels page. Afterwards, we will show you how to download your results in CSV file format. You can view part one here.

3. Adjusting the Settings

After your survey questionnaire has been constructed, you will need to ensure that the settings are
properly adjusted to allow for the best results. To do this, click on the “Advanced Options” under the
“Edit” tab. Ensure the following settings are defined:

  1. Enter your email address in the notify field.
  2. Enter key words relevant to your survey topic.
  3. Set Max. Work Per IP to 1. (This prevents contributors from taking the survey more than once).
  4. If you would like your job to run only in specific geographical locations, you can select these
    locations via the International tab at the top (All jobs are set to run in the US by default).
  5. Don’t forget to set the time limit to a value you feel is more than sufficient for the contributor.
    For instance, if your survey takes 20 minutes on average, it might be good to set your time limit to
    30 minutes just in case.


Figure 3.0 – The Advanced Options View

4. Launching the Job

To get your job up and running, you can click the brown button entitled “Not Ordered” and then select
“Order Judgments.” If it is your first time running a job on CrowdFlower, you will have to have your job
approved by a Crowdsourcing Expert before it can be sent to the crowd.

You will then be directed to the Job Calibration Settings page. Here you will be able to adjust the
pricing of your job (Payment Per Page), as well as the number of participants you wish to gain feedback
from (Judgments Per Unit). Ensure that your “Units Per Page” value is set to 1. When you feel
comfortable with your settings, click the “Save and Continue to Order” button at the bottom right of the
screen (Figure 4.0).


Figure 4.0 – Launching the Job

The next page will allow you to select the channels you wish to send your survey to. Channels are
essentially large pools of contributor communities and sending your job to these channels will mean your
job will be posted and accessible on the Channel’s website or application (See Figure 4.1 Below).

If you haven’t already added credit to your account, you can do so now by clicking on the “Add Money”
link located below the Current Balance figure.

Figure 4.1 – Choosing Channels for your Job

5. Retrieving Results

As soon as the job is complete, the reports will generate automatically. You can access these reports
by clicking on the Reports tab located on the right side of the toolbar.

The Full Report will have every one of the contributors’ submitted answers, as well as a good bit of
other data, such as location, time and IP address. Each contributor’s response will be listed on an
individual row in the data sheet. You will be able to generate this report both while the job is running
and after the job has completed.


Figure 5.0 – Full Report CSV

That’s all for today! Thank you for taking a look and feel free to reach out with any questions at

David Rand