Altruism on Amazon Mechanical Turk

 

Many workers on Amazon Mechanical Turk are willing to help others at a cost to themselves, just like participants in laboratory experiments.

While traditional economic models assume that people are entirely selfish, a central theme in behavioral economics is the existence of ‘social preferences’, or caring for others. Countless laboratory experiments have demonstrated that many people are willing to help others, even at a cost to themselves. This behavior is clearly inconsistent with being motivated only by your own monetary payoff – if you are entirely selfish, you would never pay money to help someone else in the totally anonymous conditions of the lab. In this post I describe an experiment I conducted together with John Horton, and with invaluable technical assistance from Xiaoqi Zhu, that replicates the existence of social preferences on Amazon Mechanical Turk (AMT), showing that many Turkers behave altruistically.

We also demonstrate the principle of priming, another focus of great interest in experimental economics. In priming studies, stimuli unrelated to the decision task (and which do not affect the monetary outcomes) can nonetheless significantly alter subjects’ behavior.

To assess altruistic behavior on AMT, 194 subjects played an incentivized Prisoner’s Dilemma (PD), the canonical game for studying altruistic cooperation. Subjects were informed that they had been randomly assigned to interact with another Turker, and that they would each have a choice between two options, A or B. In addition to a 20 cent “show-up fee”, they were informed of the following payoff structure: if both subjects chose A, they receive each earn a 120 cent bonus; if both chose B, they would each receive an 80 cent bonus; if one chose A while the other chose B, the A player would receive 40 cents while the B player would receive 160 cents. The resulting payoff matrix is as follows (in each cell I first show the row player’s payoff, and then the column player’s payoff):

A

B

A

120,120

40,160

B

160,40

80,80

 

Thus A represents cooperation, and B represents defection. If both people chose A, they both do better than if both choose B. However, regardless of the other’s action, you earn more by choosing B (hence the ‘dilemma’). Rational self-interested players should therefore always select B, and it is altruistic to choose A (helping the other at a cost to you). Given previous evidence from experiments in the laboratory, however, we predicted that AMT subjects would demonstrate a level of cooperation significantly greater than 0 in a one-shot PD.

To explore the effects of priming on AMT subjects, we built on a previous study demonstrating that exposure to religious words and phrases increases altruistic behavior, particularly among those who believe in god (Shariff & Norenzayan 2007). Among the 194 subjects in our experiment, the prime group (N=89) read a Christian religious passage about the importance of charity (Mark 12:21-22) before playing the PD, whereas the no-prime group (N=105) did not. Following the PD, subjects completed a demographic questionnaire reporting age, gender, and education, and indicated whether they had ever had an experience which convinced them of the existence of god.Based on the results of Sheriff & Norenzayan, we hypothesized that the religious prime would increase cooperation, and further hypothesized that the effect would be driven by subjects that believe in god.

Consistent with our first prediction, we observe a level of cooperation significantly greater than 0 in both the no-prime (54% C: sign-rank test, p<0.001) and prime (71% C: sign-rank test, p<0.001) conditions. Consistent with our second prediction, we observe significantly more cooperation in the prime condition compared to the no-prime condition (Chi2 test, p=0.018). Consistent with our third prediction, the prime only increases cooperation among subjects who believe in god (Chi2 test, non-believers: p=0.82, believers: p=0.004). The results are visualized in Figure 1. Using logistic regression with robust standard errors, we also find that these results are robust to controlling for age, gender, country of residence (US vs non-US), religion (Christian vs non-Christian) and education.

Figure 1 Figure 1. Reading a religious passage significantly increases Prisoner’s Dilemma cooperation among those who believe in god, but not among non-believers.


To summarize, we have demonstrated two aspects of Turker behavior:

1. A majority of Turkers chose the altruistic option of cooperating in a Prisoner’s Dilemma. Thus even in the entirely anonymous and profit-motivated online labor market of AMT, many people still choose to help each other. This sort of altruistic cooperation is a fundamental part of the natural world, and is the building block of human societies. For more, see (Nowak 2006).

2. Reading a religious passage about the important of charity makes religious Turkers more altruistic, but has no effect on Turkers who do not believe in god. This shows that Turkers respond in basically the same way as “normal” lab subjects, and is fairly intuitive. Those who believe in god are receptive to calls for generosity phrased in religious language, while non-believers aren’t. Secular primes have been shown to work for both religious and non-religious subjects (Shariff & Norenzayan 2007).

Although AMT workers are certainly not a generally representative sample, this study demonstrates that they show several of the same basic behavioral features observed in behavioral laboratory experiments. Furthermore, AMT allowed this study to be run extremely quickly and inexpensively. The 200 subjects were recruited in less than 2 days, at a total cost of $253. As a behavioral researcher, this is amazingly exciting! I usually spend months and thousands of dollars per study. AMT opens the possibility of exploring countless interesting ideas that otherwise we would have had neither the time nor money to pursue.

For other studies about cooperation, reward and punishment that I’ve conducted at Harvard, see the pdfs on my webpage: www.DavidGertlerRand.com.

21 Responses to “Altruism on Amazon Mechanical Turk”

  1. Shauna

    Hi Dave! Nice study. A question, though – when you give a religious passage about charity, you’re conflating an altruism prime and a religious prime. It would be interesting to run with either a non-charity religious prime or a non-religious charity prime, to try to tease apart the factors. It would also be interesting to see whether the religious nature of the prime obviates the charity content for non-believers (so they too would show a positive effect in the primed condition) or whether they’re just more heartless than believers.

  2. David Rand

    Hi Shauna, thanks for your comment. Your suggestion is a very good one, and something we are currently pursuing. This study was mostly a proof-of-concept, to show that both pro-sociality and priming effects occur on AMT. The lab results of Shariff & Norenzayan 2007 suggest that religious concepts not having to do with charity are enough to make believers more cooperative, while non-religious primes increase cooperation in both believers and non-believers.

  3. Turker

    have you done experiments to see how long this priming effect lasts?
    are there any classes of activities that tend to interfere with this priming effect? (having participant memorize lists of numbers, etc…) thanks

  4. David Rand

    Two very interesting questions!

    Pedro and Ernesto Dal Bo have performed an experiment looking at the effects of priming in repeated games:

    http://www.econ.brown.edu/fac/Pedro_Dal_Bo/rightthing.pdf

    They find that in a repeated multi-player Prisoner’s Dilemma, the effect of the prime is transient – over a series of decisions the boost in cooperation caused by the prime decays. But they also find that when the PD is coupled with a chance to explicitly punish others, the prime has a lasting effect.

    I’m not aware of research which asks what can block or reduce priming effects, but again a great question.

  5. Sc

    Very interesting.

    I wonder if the tiny size of the bonus made a difference here. 40 cents isn’t a huge amount to sacrifice. If there were a million dollars at stake, would the behavior be different? If so, is there a simple linear relationship between bonus size and altruism, or is there a tipping point somewhere? Unfortunately, it would likely be prohibitively expensive to test that.

  6. David Rand

    The issue of stake size has plagued experimental economics for a long time. However, a number of studies have demonstrated the persistence of pro-social preferences with higher stakes – for example $100 games with american college students, and various studies in third world countries with stakes equivalent to several months wages. So although higher stakes may have some degree of effect, it doesn’t matter as much as you would think…

  7. Turil

    I’m wondering if you clarify what you mean by “god” when you asked people if they believe in it. I’ve noticed that in a lot of these studies it’s really unclear, and it’s impossible for me to answer accurately. I know a lot of people who consider themselves spiritual but not religious, or believe in “a consciousness greater than humans” but are loathe to call it “God” or even a “god”, because of the stigma attached to the term these days of sort of reverse discrimination. And also, many folks who believe in God don’t consider themselves religious.

    So I guess my question is what are the specific traits that you are interested in testing out here when it comes to this idea of religion or “god” or “God”?

  8. David Rand

    The specific question we asked was “Have you ever had an experience which convinced you of the existence of god?”, letting them interpret it as they like. These issues that you raise are certainly real, but i think probably cancel each other out as random noise – some people who do believe in some type of god but aren’t religious, and some people who are religious/spiritual but don’t believe in god. The way I interpret our results is to say that a Christian bible passage about charity has a significant impact on cooperation, but not among people who don’t associate themselves with the formal concept of God. The point was not so much about religion/belief in particular, but more about priming in general – we just used religion as a proof-of-concept, that priming effects exist, but that their effect is not universal.

  10. Turil

    Thanks for the follow up David, I’d add one small, but important (to some) change and say that the results suggest that: a Christian bible passage about charity has a significant impact on cooperation, but not among people who don’t associate themselves with their own person concept of God.

    Anyway, thanks for offering the study results, it is interesting.

  12. Judd Antin

    Hi David,

    Great stuff! I wanted to share some links to others who are have done similar experiments using AMT, in case you or others haven’t seen them. The Experimental Turk Blog (http://experimentalturk.wordpress.com/) is entirely dedicated to replicating classic game theory and behavioral econ. experiments on the Turk. Panos Ipeirotis at NYU’s Stern School of business (http://behind-the-enemy-lines.blogspot.com/) has also done some experiments like yours.

    Secondly, IMO altruism is not the right term for the behavior you observe. The reason is that altruism refers to situations where someone is wholly and unselfishly motivated by the rewards of others. Pro-social behavior, on the other hand, is any behavior that benefits others, often (but not always) at one’s own expense. So altruism is one possible motivation for pro-social behavior, but there are lots of others – the expectation of future reciprocity, for example. There’s a great discussion of the confusion around these terms in:

    Simpson, Brent and Robb Willer. 2008. \Altruism and Indirect Reciprocity: The Interaction of Person and Situation in Prosocial Behavior.” Social Psychology Quarterly 71:37-52.

  13. David Rand

    Hi Judd,

    Thanks for sharing the links – both are great blogs for this sort of thing!

    In terms of defining different terms, I agree that altruism is a subset of pro-social. But cooperation in the Prisoner’s Dilemma is altruistic, because it reduces your payoff while increasing the other person’s. No matter what the other player chooses, you earn more by defecting. So cooperating is altruistic, it seems to me…

    Dave

  14. Rowan Nairn

    It seems like an oversimplification to say that the rational choice is to always pick B. Although you know that the experiment is one-shot, completely anonymous and has no side-effects for the participants, they may not be so sure. That uncertainty could increase their perceived utility of cooperation. That may be what Judd is referring to above.

  15. David Rand

    Although its true that we can’t be certain about how the subjects were thinking about the experiment, we did try to make it as clear as possible that it was a one-shot anonymous game with no side-effects. The following text was included in the instructions:

    “You have been randomly assigned to interact with another person who also completes this HIT. You will never know who this other person is, and they will never know who you are. Both of you receive this same set of instructions. You cannot participate in this game more than once. [Explains the interaction setup] Once you and the other person have chosen an option, the interaction is over. Neither you nor the other person receives any bonus other than what comes out of this interaction. “

  16. Thorstein Veblen

    David – The distinction Judd is pointing to is between altruism as a motivation and altruism as a behavior. Most social scientists use the term “altruism” to describe a psychological motivation to help others. Most biologists use the term to refer to a behavior that benefits others at a cost to the self. So while a biologist would say tat cooperation in a one-shot PD is altruistic for the reason you state, most social scientists would say that such a behavior might be altruistic depending on the motivation underlying the action.

    Neither is the “correct” definition, but the distinction between prosocial behavior that is motivated by a sincere desire to benefit others as opposed to some form of self-interest (e.g., pursuit of an improved reputation, status, approval, avoidance of sanctions, “shadow of the future” in repeated interactions) is important to psychologists and social scientists.

  17. Turil

    The other factor that is useful to point out is the environment factor. Cooperation allows for a healthier environment (when my neighbor does well, she’s less likely to need something from me, and is more likely to be generous with me), thus, in a heavily populated area (such as the planet Earth) the most rational choice IS cooperation.

  18. Rochelle

    Hate to burst your bubble, but there are people (like me), who have answered your quizzes, surveys, or whatever you call the flavor of a particular head-game on Amazon’s slave-wage jobs board (Mechanical Turds); but that does not mean you’ve been given any real, honest data.

    Quite the contrary . . .

  19. Josh Eveleth

    Rochelle,

    Thank you for your comments. We are sorry to hear that you’re displeased with us. We do post microtasks in exchange for micropayments, but workers are not required to perform the tasks. Can you provide more specific feedback on your complaint?

  20. Angelo

    I believe it would have been quite different if you changed the “risk” levels and such involved.

    For example, if the payoff became:
    You both chose A: Both get $800
    You both chose B: Both get $600
    You chose A, other choses B: You get nothing
    You chose B, other choses A: You get $1000

    I believe you would get markedly different results.

    That’s because on a deeper level, I believe the unconcious or concious thought running through peoples minds is:
    “I can help them at LITTLE COST to myself” and not
    “I can help them even though I disadvantage myself”.

    Raise the ante even further, to, say:
    20 Years in Prison, 5 Years in Prison, Death Pentalty and Going Free, and I believe a whole lot more will opt for the “safe route”.

  21. curious

    In the type of choices available in a turking environment, I believe that some persons (like myself) will choose the answer they believe will most likely get them paid. I believe this suggests skepticism. I also believe that an answer might change on any given day, dependent on the mood of the subject. e.g., today I’m upbeat and happy, therefore I might be more likely to take risks, or be helpful to another. And the opposite would be true if I were angry, sad, etc.