As a psychologist, I spent years of my life running experiments in a lab the traditional way: emailing volunteers one by one, scheduling them for an hour in my calendar, and guiding them through the tasks I want them to complete. In a good month I’d have a dataset with 20 participants and I’d spend weeks analyzing it every possible way before running another experiment.
Now that I work at CrowdFlower, I want to find out whether we can use crowdsourcing to answer psychological questions more efficiently, without lowering data quality. If so, it has the potential to revolutionize the whole field.
I picked a survey-based experiment that I conducted multiple times as a researcher, which centers on the following question: what is the relationship between anxiety and attention?
Trait anxiety is how likely we are to have anxious thoughts and behaviors. It’s different than state anxiety which is how we feel at the present moment.
Attentional control is a technical term that describes our ability to pay attention.
Some psychologists have found that the more anxious you tend to be, the harder it is to pay attention (e.g., Derryberry & Reed, 2002). Stress causes the brain regions that process emotional information to “hijack” the brain regions that control attention. Highly anxious people’s brains incorrectly categorize neutral information as emotional information, which causes them to be easily distracted.
In my own research I had trouble reproducing this result, so I wanted to try the same experiment using CrowdFlower. I surveyed about 1000 people (click here to view a pdf with details on methodology and demographics of our sample).
Higher anxiety was clearly correlated with worse attention with a p-value of less than 0.00001. As you can see from the scatterplot, the correlation is not perfect, so it was necessary to access a large group of people to clearly show a statistically significant connection.
Now, what else did we find out from this large data set? We saw that there is no statistical difference between females and males on this correlation. The regression lines are almost exactly the same for both females and males.
We found that people aged 18 to 22 have significantly less attentional control than people 23 and above. People 40 and above have significantly higher attentional control than people 22 or younger. This makes me think that maybe the best time for higher education is when you’re 23, not 18!
We also found a large increase in anxiety when people turn 18, and then a slow drop off as age increases.
We found that democrats (n=372) and independents (n=304) were each significantly more anxious as a group than republicans (n=237). Although it appears that the Green party has the highest anxiety score, there were only 6 respondents in that group – not enough to draw any conclusions!
Similarly to the above findings, we also found that single people (this does not include cohabiting couples) are significantly more anxious that married people, and atheists are more anxious than Christians.
Most psychological research comes from studying a very narrow slice of the population. Typically, experimental participant pools are confined to the undergraduates at the scientist’s university, or worse, the university’s psychology majors who do experiments for class credit. How can we discover generalities about how the human mind works if we base our conclusions on, for example, the psychology majors at New York University? From my experience at NYU, these students are upper middle class, Caucasian females from Long Island between the ages of 18 and 20. With crowdsourcing, the hope is that we can expand the participant pool to a wider range of geographical locations, ages, income levels, and political and religious affiliations. Moreover, we can collect more data in much less time, for less money.
- The statistical analyses you see in this post were calculated using StatWing, a new and free online tool to make sense of your data! Brought to you by one of CrowdFlower’s old colleagues. Just upload and analyze.
- The pretty graphs were made in R using custom scripts. R is a statistical programming language and RStudio is a friendly graphical developing environment. Both are free to use.
- Berggren, N. and Derakshan, N. (2012). Attentional control deficits in trait anxiety: Why you see them and why you don’t. Biological Psychology, epub ahead of print.
- Bishop, S.J. (2009). Trait anxiety and impoverished prefrontal control of attention. Nature Neuroscience, 12(1):92-8.
- Derryberry, R., and Reed, M.A. (2002). Anxiety-related attentional biases and their regulation by attentional control. Journal of Abnormal Psychology, 111(2):225-36.
- Spielberger, C. D., Gorsuch, R. L., Lushene, R., Vagg, P. R., & Jacobs, G. A. (1983). Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press.