Comments on: Age and Gender Stereotypes http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/ Wed, 21 Jul 2010 00:42:40 +0000 hourly 1 http://wordpress.org/?v=3.0 By: Nicholas http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/#comment-1058 Nicholas Mon, 13 Apr 2009 19:54:31 +0000 http://blog.doloreslabs.com/2009/02/age-and-gender-stereotypes/#comment-1058 Hi, Variable sized buckets is just like varying the bandwidth. The problem with the binning procedure is that since the weights are equal for every point, as you increase the bucket size, the standard error will decrease. If you want to show the variability inherent in the data, the raw data potted with alpha blending would show that, if you want to show the variability in the means, use simultaneous intervals for the smoother. If all you care about is the population trend than, the bin level intervals don't matter. Futzing with the kernel density estimates is not a big deal, it is just that on the edges they give an erroneous picture. One of these days I will send Deepayan a patch with some more sensible options. It is a well known phenomenon that kernel smoothers have really bad problems at the edges of the observed data. Glad you liked the ellipses. Hi,
Variable sized buckets is just like varying the bandwidth. The problem with the binning procedure is that since the weights are equal for every point, as you increase the bucket size, the standard error will decrease. If you want to show the variability inherent in the data, the raw data potted with alpha blending would show that, if you want to show the variability in the means, use simultaneous intervals for the smoother. If all you care about is the population trend than, the bin level intervals don’t matter.

Futzing with the kernel density estimates is not a big deal, it is just that on the edges they give an erroneous picture. One of these days I will send Deepayan a patch with some more sensible options. It is a well known phenomenon that kernel smoothers have really bad problems at the edges of the observed data.

Glad you liked the ellipses.

]]> By: brendano http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/#comment-1019 brendano Thu, 02 Apr 2009 20:53:09 +0000 http://blog.doloreslabs.com/2009/02/age-and-gender-stereotypes/#comment-1019 Nicholas -- we ended hacking around the problem by making a new graph that uses variable-sized buckets, so larger on the ends. Even with the kind of dumb one-sample independent intervals, the phenomenon for the attractiveness plot emerges. Here's the progression of graphs: http://assets.doloreslabs.com/blog/aag_buckets_all.pdf Yes, smoothed density plots are problematic. We're using whatever smoother that "smoothScatter" uses; Lukas (not me) looked into the detail there. All kernel smoothers are kinda weird -- i'd have a hard time believing there are any good reasons to futz with them beyond changing the bandwidth/span parameter. At least, any reasons that are both (1) principled, and (2) whose assumptions apply to real-world data analysis scenarios. I like the ellipses. Alex: Ooh, interesting hypothesis about age of viewers. That's something to look in to... Nicholas — we ended hacking around the problem by making a new graph that uses variable-sized buckets, so larger on the ends. Even with the kind of dumb one-sample independent intervals, the phenomenon for the attractiveness plot emerges. Here’s the progression of graphs:
http://assets.doloreslabs.com/blog/aag_buckets_all.pdf

Yes, smoothed density plots are problematic. We’re using whatever smoother that “smoothScatter” uses; Lukas (not me) looked into the detail there. All kernel smoothers are kinda weird — i’d have a hard time believing there are any good reasons to futz with them beyond changing the bandwidth/span parameter. At least, any reasons that are both (1) principled, and (2) whose assumptions apply to real-world data analysis scenarios.

I like the ellipses.

Alex: Ooh, interesting hypothesis about age of viewers. That’s something to look in to…

]]> By: Alex http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/#comment-1018 Alex Thu, 02 Apr 2009 19:41:50 +0000 http://blog.doloreslabs.com/2009/02/age-and-gender-stereotypes/#comment-1018 Interesting study! It'd be interesting to understand how the age of the viewer impacts this. I'm guessing that your population is skewed in some way (perhaps overrepresented around that 20-25 age group), and my (untested) hypothesis is that people are better at discerning differences in people closer to their own age. Any chance of incorporating viewer age data? Drawing envelopes on your error bars might make the data more "fair"; as it is, your solid lines visually overstate their own accuracy, just because they're so prominent. Interesting study!

It’d be interesting to understand how the age of the viewer impacts this. I’m guessing that your population is skewed in some way (perhaps overrepresented around that 20-25 age group), and my (untested) hypothesis is that people are better at discerning differences in people closer to their own age. Any chance of incorporating viewer age data?

Drawing envelopes on your error bars might make the data more “fair”; as it is, your solid lines visually overstate their own accuracy, just because they’re so prominent.

]]> By: Nicholas http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/#comment-997 Nicholas Wed, 25 Mar 2009 23:04:48 +0000 http://blog.doloreslabs.com/2009/02/age-and-gender-stereotypes/#comment-997 just to echo Hadley's, since age is a continuous variable and so is your response, you should be using simultaneous error bands. Also which smoother are you using? For correlation coefficients ellipses are better, Deepayan has an example at http://lmdvr.r-forge.r-project.org/figures/figures.html look at figure 13.5 Lastly, the I don't love the density plots based on kernel smoothing, the singletons are not treated very well, and you end up with the blurred dots around the edges of the density. Plotting the singletons separately sometimes helps. just to echo Hadley’s, since age is a continuous variable and so is your response, you should be using simultaneous error bands. Also which smoother are you using? For correlation coefficients ellipses are better, Deepayan has an example at
http://lmdvr.r-forge.r-project.org/figures/figures.html
look at figure 13.5
Lastly, the I don’t love the density plots based on kernel smoothing, the singletons are not treated very well, and you end up with the blurred dots around the edges of the density.
Plotting the singletons separately sometimes helps.

]]> By: Hadley http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/#comment-941 Hadley Mon, 23 Feb 2009 14:50:42 +0000 http://blog.doloreslabs.com/2009/02/age-and-gender-stereotypes/#comment-941 Thanks for the explanation - and I'm glad to hear that you're finding plyr helpful :) Thanks for the explanation – and I’m glad to hear that you’re finding plyr helpful :)

]]> By: Brendan O'Connor http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/#comment-937 Brendan O'Connor Fri, 20 Feb 2009 22:14:02 +0000 http://blog.doloreslabs.com/2009/02/age-and-gender-stereotypes/#comment-937 @Hadley: no, not a two-sample t-test. The bars are 95% CI's from a one-sample t-test with a two-sided alternative. What you get by calling R's t.test() on a single vector of numbers. Lots of the extreme buckets have just a few instances, some have only singletons. Giving the counts is probably more useful than all this t-test stuff. When the CI's are so big it's silly to have single-year buckets. This was just a first attempt. We have a better graph since making this one, that uses bigger age buckets at the extremes. With <a href="http://had.co.nz/plyr" rel="nofollow">plyr</a> :) @Bruce: yeah we definitely need to get on that. I figure that binning by race will get people the most riled up as possible... @Hadley: no, not a two-sample t-test. The bars are 95% CI’s from a one-sample t-test with a two-sided alternative. What you get by calling R’s t.test() on a single vector of numbers. Lots of the extreme buckets have just a few instances, some have only singletons. Giving the counts is probably more useful than all this t-test stuff.

When the CI’s are so big it’s silly to have single-year buckets. This was just a first attempt. We have a better graph since making this one, that uses bigger age buckets at the extremes. With plyr :)

@Bruce: yeah we definitely need to get on that. I figure that binning by race will get people the most riled up as possible…

]]> By: Bruce http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/#comment-903 Bruce Fri, 13 Feb 2009 22:39:40 +0000 http://blog.doloreslabs.com/2009/02/age-and-gender-stereotypes/#comment-903 How about showing how perceived intelligence correlates with race? or with gender? How about showing how perceived intelligence correlates with race? or with gender?

]]> By: Hadley http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/#comment-884 Hadley Tue, 10 Feb 2009 13:36:27 +0000 http://blog.doloreslabs.com/2009/02/age-and-gender-stereotypes/#comment-884 A two-sided t-test? Do you mean that the error bars are for the difference of the means, not the means themselves? That would be non-standard, but ok. What alpha are you using - ideally you should be using 1.4 (sqrt(1^2 + 1^2)) because then non-overlapping confidence intervals would correspond to a p-value from the t-test of 0.05 or less. And you must have _really_ small numbers (or distributions that are very heavy tailed) to such wide intervals. Some plots of sample sizes would be informative. You might also want to plot the differences directly - there is a well known visual problem where our visual system compare distances between curves based on the shortest distance between the them, not the vertical distance. A two-sided t-test? Do you mean that the error bars are for the difference of the means, not the means themselves? That would be non-standard, but ok. What alpha are you using – ideally you should be using 1.4 (sqrt(1^2 + 1^2)) because then non-overlapping confidence intervals would correspond to a p-value from the t-test of 0.05 or less.

And you must have _really_ small numbers (or distributions that are very heavy tailed) to such wide intervals. Some plots of sample sizes would be informative.

You might also want to plot the differences directly – there is a well known visual problem where our visual system compare distances between curves based on the shortest distance between the them, not the vertical distance.

]]> By: lukas http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/#comment-881 lukas Tue, 10 Feb 2009 04:09:53 +0000 http://blog.doloreslabs.com/2009/02/age-and-gender-stereotypes/#comment-881 The data is extremely biased towards people aged 20-30. Also, not every image is labeled with every tag, so each graph plots a subset of the data where both variables were labeled by at least one person. To answer your question explicitly, we use a two-sided t-test for the error bars. The data is extremely biased towards people aged 20-30. Also, not every image is labeled with every tag, so each graph plots a subset of the data where both variables were labeled by at least one person.

To answer your question explicitly, we use a two-sided t-test for the error bars.

]]> By: Hadley http://blog.crowdflower.com/2009/02/age-and-gender-stereotypes/#comment-880 Hadley Tue, 10 Feb 2009 04:00:38 +0000 http://blog.doloreslabs.com/2009/02/age-and-gender-stereotypes/#comment-880 How are you calculating the confidence interval? They seem awfully large if you have 100,000 faces in total. How are you calculating the confidence interval? They seem awfully large if you have 100,000 faces in total.

]]>