Login

Login

The CrowdFlower Blog

Crowdsourcing Thought Leadership: Building a successful portfolio of crowdsourcing projects (Part 1)

by

This is part of a series of guest posts by Ram Rampalli, our crowdsourcing partner at eBay.
Part I – Assessment Stage
Part II – Pilot Stage
Part III – Analysis Stage
Part IV – Production Stage

About the author: Ram Rampalli created and leads the crowdsourcing program within the Selling & Catalogs team at eBay Inc. You can follow him on Twitter (@ramrampalli)

Building a successful portfolio of crowdsourcing projects – Part 1

Like many other organizations, eBay has leveraged offshore teams in several low-cost destinations to complete many routine operational processes. While many of these projects have been quite successful, outsourcing continues to pose several key challenges, especially around scalability and cost. As eBay began looking at several alternatives to address these challenges, crowdsourcing emerged as an attractive option.

Crowdsourcing Thought Leadership

(photo courtesy of Sun Fusion Solar)

Continue reading »


From Sea to Shining Sea

by

With Independence Day just behind us, I thought it might make sense to reflect on the people who do CrowdFlower tasks throughout the Union.

Sure enough, we do have people doing CrowdFlower tasks in all 50 states. California, Texas, New York, and Florida are well represented among people working on our tasks, which isn’t especially surprising given that these are the most populous states. A more interesting question would be how the states stack up relative to their populations:

There are some cold weather states, like Vermont and South Dakota, that are punching above their weight class in terms of CrowdFlower judgments submitted, although Maine and North Dakota are lagging behind so that doesn’t really work. Connecticut and New Jersey, two of the wealthier states in terms of average income, are underperforming relative to their population, but Massachusetts and New York are doing just fine. Mississippi and West Virginia, coming from the opposite end of the spectrum in terms of average income, show up strongly, but Louisiana and Montana don’t.

Any thoughts about why some states are more represented? Leave ‘em in the comments.


Designing Incentives for Crowdsourcing Workers

by

In a recent paper, presented at the ACM Conference on Computer Supported Cooperative Work (CSCW), John Horton, Daniel Chen and I used a large-scale experiment to test the effect of different incentive schemes on the quality of crowdsourcing work.

The results surprised us. They suggest that workers perform most accurately when the task design credibly links payoffs to a worker’s ability to think about the answers that their peers are likely to provide.

Horserace!

a horserace experiment! (photo cc-by-sa by iyoupapa)

Continue reading »


eDiscovery, meet Crowd

by

Once upon a time, I had a job that included looking through boxes of documents that were supposedly related to environmental litigation, but were generally (a) unrelated, (b) dusty and (c) mind-numbingly dull. Earlier this year, as I looked back on those dark days, it seemed to me that crowdsourcing would be a great tool for a first pass through documents, helping a legal team focus its efforts away from documents that are obviously not responsive to a given request.

Continue reading »


Announcing the TREC 2011 Crowdsourcing Track, co-sponsored by CrowdFlower

by

We’re proud to announce CrowdFlower’s participation in one of the most prestigious academic conferences worldwide.

The 20th Annual Text REtrieval Conference (TREC 2011), organized by the National Institute of Standards (NIST), will take place Nov 15-18, 2011, at NIST’s campus in Gaithersburg, MD. TREC 2011 will feature a new Crowdsourcing Track that will investigate:

  • How to obtain high-quality relevance judgments from individual crowd workers;
  • How to effectively compute consensus judgments from individual judgments;
  • Interaction between these (i.e., worker accuracy vs. subsequent consensus accuracy achieved).

Continue reading »



Why CrowdFlower?

How it Works What it Means Scalability Technology Innovation and Expertise

Documentation

Requester Interface Gold CrowdFlower API CML Channel API Image Moderation API

Solutions

eCommerce Online Media and Publishing Data Providers Daily Deals & Local Search Brand Management Self-Service

Products

read more...

Customers

read more...

About

Team Press Resources Jobs Contact

Law Talk

Privacy Policy Terms of Service ©2011 CrowdFlower