What is fair payment on MTurk?

My go-to place for convenience sampling is MTurk. It’s fast, it’s easy, it’s flexible, it can be as simple or complicated as you need it to be, the data is high quality and the samples are fairly “representative” of the US population. But what is “fair payment” for “work” on MTurk?

Objectively, if we accept that MTurk work is like other kinds of work, then we should aim to pay an amount that equates to a minimum hourly wage. We can do this by estimating the time that the task takes the average worker to complete, and then choosing a dollar value that would add up to a minimum hourly wage. For example, if a task takes an average of 1 hour to complete we could pay $7.25 or $12 to every worker who completes the task; if a task takes 1 minute to complete, then we can pay 1/60th that amount (e.g., 12 cents or 20 cents).

Subjectively, we can ask people what they think would be fair. I wasn’t aware of any current data that speaks to MTurk worker pay expectations or perceptions of payment fairness so I decided to run a survey and find out!

I think researchers have a tendency to assume that it’s “okay” to pay workers on MTurk less than a minimum wage because there has been some research that shows, despite money being the primary motivation, data quality seems to be just as good regardless of payment. (See Litman et al., 2014)

But I wonder if Litman et al.‘s research is still true today?

I think the visibility of crowdwork and the discourse surrounding it has changed, and people are more aware and more vocal about the exploitation that crowdworkers are subjected to. I wonder if, for these reasons, workers would now be more willing to “withhold effort” if they felt they were not being paid fairly for their work?

Something to think about.