The CrowdFlower Blog

Dramatic Intro

It is high noon in business listing verification crowdsourcing land. We are throwing down the gauntlet. We are stepping in the ring. We are mixing our metaphors.

Undramatic Intro

Yelp engineers recently described their efforts to correct business listing data using Amazon Turk. They tapped the services of 4,660 contributors; only 79 passed their quality assurance testing (1.7% of contributors were “trusted”), and the data they output was (very roughly) 80% accurate.

This smelled funny to us. Our business listing verification service routinely returns results above 97% accuracy. In fact, some of the most recognizable names in local search and business data pay for that service. (See a full report on 100,000 listings we did for a major search company to see some typical figures). Out of the last couple dozen crowdsourcing tasks we’ve run, the absolute minimum proportion of contributors who were “trusted” was 34%. But more importantly, our platform identifies these trusted contributors within minutes, meaning the best contributors get the job done quickly.

Continue reading »