Comments on: The Case for Online Experimentation http://blog.crowdflower.com/2010/05/the-case-for-online-experimentation/ Mon, 06 Feb 2012 14:03:11 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: David Molnar http://blog.crowdflower.com/2010/05/the-case-for-online-experimentation/#comment-2924 David Molnar Thu, 06 May 2010 04:28:15 +0000 http://blog.crowdflower.com/2010/05/the-case-for-online-experimentation/#comment-2924 We recently used Mechanical Turk as part of a study on how users respond to different security warnings. You can read the paper here: http://dmolnar.com/papers/secdelay-weis2010.pdf To appear at the Workshop on Economics and Information Security 2010. In general, Mechanical Turk is a source of people for these kinds of "experimental economics" questions. I am also starting to look at Mechanical Turk for coding free-form data (e.g. web pages) to feed into an analysis. The hope is that Turkers could replace time consuming tasks such as reading a web page and determining if it is an advertisement for a certain product. This is a much better fit than surveys for the current Crowdflower interface, because it lends itself naturally to the notion of "gold" questions, to "trusted workers," etc. We recently used Mechanical Turk as part of a study on how users respond to different security warnings. You can read the paper here:
http://dmolnar.com/papers/secdelay-weis2010.pdf

To appear at the Workshop on Economics and Information Security 2010. In general, Mechanical Turk is a source of people for these kinds of “experimental economics” questions.

I am also starting to look at Mechanical Turk for coding free-form data (e.g. web pages) to feed into an analysis. The hope is that Turkers could replace time consuming tasks such as reading a web page and determining if it is an advertisement for a certain product. This is a much better fit than surveys for the current Crowdflower interface, because it lends itself naturally to the notion of “gold” questions, to “trusted workers,” etc.

]]> By: Charlotte Wickham http://blog.crowdflower.com/2010/05/the-case-for-online-experimentation/#comment-2906 Charlotte Wickham Sat, 01 May 2010 18:13:16 +0000 http://blog.crowdflower.com/2010/05/the-case-for-online-experimentation/#comment-2906 At a quick skim it's an interesting paper. My comments relate to the figures (because that's generally what I look at first). Figure 1: You probably don't want bars in this plot. I think its generally unwise to use bars for point estimates and especially here where your x-axis doesn't even start at zero. How about dots plus error bars? Figure 2: Can I suggest the reordering of country as: Europe, Other, India, US? At a quick skim it’s an interesting paper. My comments relate to the figures (because that’s generally what I look at first).

Figure 1:
You probably don’t want bars in this plot. I think its generally unwise to use bars for point estimates and especially here where your x-axis doesn’t even start at zero. How about dots plus error bars?

Figure 2: Can I suggest the reordering of country as: Europe, Other, India, US?

]]>