Login

Login

The CrowdFlower Blog

Crowdsourcing and Retention: From First-Timers to Seasoned Veterans

by

Millions of people have participated in our tasks over the last few years, and tens of thousands of people are active at any given moment. However, crowdsourcing is not a traditional engagement model. Tasks are elective, which means people are free to come and go as they please. It’s a fair question, then, to ask whether they keep coming back.

crowdsourcing

Do people perform tasks only fleetingly, or has crowdsourcing become more of a long-term engagement? Furthermore, just how important is contributor retention in the world of crowdsourcing?

While a majority of people fall into the “one-and-done” camp, many of the most productive contributors tend to have participated in previous jobs. Within any single job, these seasoned veteran contributors also provide far more work than their less experienced counterparts.

Continue reading »


Enterprise Crowdsourcing or: How I learned to stop worrying and trust the crowd

by

Our recent post about confidence bias, where we showed that most contributors vastly overestimate their own ability to complete tasks correctly, raised a lot of questions about how we manage quality at CrowdFlower. You might remember these themes from such classic posts as: AMT is Fast, Cheap and Good or the Wisdom of Small Crowds series [1] [2] [3].

crowdsourcing

via: reddead.wikia.com/

The standard CrowdFlower model is agnostic towards the quality of any individual contributor. Typically, we let anyone attempt a task, using our technology to filter out low-quality contributors and score the responses. Without further ado, what follows is quick review of the steps we take to do that filtering.

Continue reading »


Crowdsourcing a Map, to Eat

by

A parmigiano reggiano aging room

Expanse of Cheese: Caseificio di San Silvestro's Aging Room in Castelvetro (MO)

Last May, I took a trip to Italy for two weeks. A little bit of history: my friend Jessica and I are both Italophiles, and when her mom sent us a link to a video contest where the prize was a round trip flight to Italy, we knew we had to enter. After a week of writing and editing lyrics in a Google Doc — half in Italian, half in English — the resulting music video ended up winning us a trip to the holy land of olive oil, vino, and other delectable edibles.

Apart from being a passionate eater, I’m a passionate supporter of the Slow Food movement, an organization which promotes good, clean, and fair food around the world. Each year, they publish a guidebook to restaurants in Italy that adhere to their principles. In Italy, this usually means each restaurant is handpicked to showcase the traditional food of a particular region; each restaurant supports artisanal methods and products that otherwise might go extinct (were eaters not eating them), and where the food is most likely naturally organic and local anyway.

Continue reading »


Confidence Bias: Evidence from Crowdsourcing

by

crowdsourcing

psychologytoday.com

Evidence in experimental psychology suggests that most people overestimate their own ability to complete objective tasks accurately. This phenomenon, often called confidence bias, refers to “a systematic error of judgment made by individuals when they assess the correctness of their responses to questions related to intellectual or perceptual problems.” 1 But does this hold up in crowdsourcing?

We ran an experiment to test for a persistent difference between people’s perceptions of their own accuracy and their actual objective accuracy. We used a set of standardized questions, focusing on the Verbal and Math sections of a common standardized test. For the 829 individuals who answered more than 10 of these questions, we asked for the correct answer as well as an indication of how confident they were of the answer they supplied.

Continue reading »


Crowdsourcing and SEM (now with even more cat pics)

by

Every modern business wrestles with the elusive lady that is the search engine and the potential she offers to connect with customers. Google and Bing make it easy for anyone to buy keywords and drive customers to a website, but what keywords are our customers searching for? Would a sales manager frustrated with the average 70-80% accuracy of business listings bought from data providers search for “crowdsourcing,” “address checking,” or something else entirely? Since we’re a crowdsourcing company, we had to try crowdsourcing the solution …

In the last two weeks of my summer internship at CrowdFlower, the marketing team challenged me to generate the widest range of search engine seed terms that could be used in SEM keyword tools to generate “hot” search phrases. For those of you who’ve dealt with SEM, you know that thinking of seed phrases to plug into these tools can be a painfully frustrating and surprisingly difficult task. (For those of you who haven’t and don’t believe me, try right now to describe what your company does — or anything for that matter — in 10 significantly different ways.)

Keyword tools are based on your thought process, which takes care of the customers who are thinking in the same way you are, but what about all the people of a different mindset who are trying to find your solution? For example, if I were looking for pet grooming services, depending on my thought process, vocabulary range, and amount of sleep the night before, I could search anything from “pet grooming salon” to “quality feline hair cuts” to “kitty bad hair day.” The challenge was to understand the full breadth of how the crowd approaches a certain problem, essentially the perfect task for the crowd.

crowdsourcing seo crowd sourcing seo

crowdsourcing sem

Looks like PetSmart forgot to buy an ad for "kitty bad hair day"

Continue reading »



Why CrowdFlower?

How it Works What it Means Scalability Technology Innovation and Expertise

Documentation

Requester Interface Gold CrowdFlower API CML Channel API Image Moderation API

Solutions

eCommerce Online Media and Publishing Data Providers Daily Deals & Local Search Brand Management Self-Service

Products

read more...

Customers

read more...

About

Team Press Resources Jobs Contact

Law Talk

Privacy Policy Terms of Service ©2011 CrowdFlower