We live in a world full of benchmarks and rankings. Consumers use them to compare the latest gadgets. Parents and policy makers rely on them to assess schools and other public institutions, and sports fans like them for help in sizing up their favorite teams. But what about when rankings are used at the office for appraising staff performance?
It's often assumed that employees who are benchmarked against each other work harder, to either hang onto a high ranking or raise a low ranking. However, Iwan Barankay, a management professor at Wharton, calls that assumption into question in a new study titled, "Rankings and Social Tournaments: Evidence from a Field Experiment."
"Many managers think that giving workers feedback about their performance relative to their peers inspires them to become more competitive -- to work harder to catch up, or excel even more. But in fact, the opposite happens," says Barankay, whose previous research and teaching has focused on personnel and labor economics. "Workers can become complacent and de-motivated. People who rank highly think, 'I am already number one, so why try harder?' And people who are far behind can become depressed about their work and give up."
Barankay's interest in rankings as a motivational tool intensified during the aftermath of the 2008 financial crisis, which "showed us that offering employees financial incentives based on their performance can have unintended consequences," he notes, referring to the sky-high bonuses earned on Wall Street in the run-up to the downturn.
"The practical question I wanted to answer is: What should employers do to make their employees work harder when financial incentives [aren't effective] anymore? It is often thought that people care about their status compared to others -- that people derive some happiness or dissatisfaction from knowing they're better or worse than their reference group," Barankay states. "Of course, rank should matter if money is at stake. But I looked at rank as its own reward. I wanted to find out whether workers truly want to know how they rank against their peers and ... if they knew how they ranked, did it cause them to adjust their effort?"
Unintended Consequences
His study involved 330 employees recruited via Mechanical Turk, Amazon.com's "crowd-sourcing" platform for work conducted and submitted online. Employers post jobs on the website's listings section -- most of which involve piecemeal, routine work, such as organizing photos, writing or editing text, and basic data entry. Prospective employees scroll through the list and select a task they want to complete.
When workers, also called "turkers," click on a job, they are led to a web page that presents a set of tasks. After completing the tasks, a worker can decide whether to continue on to the next job. The jobs typically pay $.03 to $.50 per task, and tasks usually take between a few minutes to an hour to complete. Among the companies that use Mechanical Turk are Google, Yahoo and Zappos.com, the online shoe and clothing purveyor.
"It's a platform that represents the new frontier of work," Barankay states. "The assignments on offer are things you can't program [a computer to do]. They're tasks that require human input, but they're not worthy of [creating] an entire job. It's a way for employers to get some back-office work done and for workers who need flexibility to make some extra cash."
According to Barankay, using Mechanical Turk for a field experiment is attractive for a number of reasons. First, it is a natural environment in which to study human behavior in a way that laboratory settings aren't able to accommodate. Second, the timeframe is short: Experiments can be completed in a couple of hours, although long-term tests can be conducted if needed. Finally, the demographic profiles of the turkers are generally broader than the conglomeration of workers in most companies or in a group of participants in laboratory experiments. "Most important," Barankay adds, "is that the platform gives you data from the real world. Nothing is more compelling than data from actual workplace settings, but getting it is usually very hard."
For his experiment, which lasted two weeks, Barankay advertised jobs on Mechanical Turk that involved analyzing images. Nothing about the jobs appeared to be different from the listings on the site offered by the likes of Google or Yahoo. He paid $.05 for every four tasks completed, irrespective of the quality of the responses. In other words, the turkers made money no matter how well they completed the task.
In the experiment's first stage, Barankay posted two identical jobs, but one offered feedback on the worker's accuracy at the end of the assignment, while the other didn't. Because conventional management wisdom contends that people want to know how they rate, Barankay thought the first job would be more popular. Instead, the job without the feedback attracted more workers -- 254, compared with 76 for the job with feedback.
"This was a surprising outcome, but it speaks to the paradigm of revealed preferences," he notes. "Economists are usually very skeptical about what people say they will do. We focus on what people actually choose to do. Their choices convey information about what they care about. In this case, it seems that people would rather not know how they rank compared to others, even though when we surveyed these workers after the experiment, 74% said they wanted feedback about their rank."
Coming Back for More
In the second stage of the experiment, Barankay randomly divided workers into two groups -- a control group receiving no ranking and a treatment group receiving feedback with a ranking. He then sent an e-mail to all of the workers inviting them to return to do more assignments. The content of all the e-mails was the same, except that individuals in the treatment group found out how they ranked in terms of their answers' accuracy. The aim was to determine whether giving people feedback affected their desire to do more work, as well as the quantity and quality of their work.
Of the workers in the control group, 66% came back for more work, compared with 42% in the treatment group. The members of the treatment group who returned were also 22% less productive than the control group. This seems to dispel the notion that giving people feedback might encourage high-performing workers to work harder to excel, and inspire low-ranked workers to make more of an effort. "This indicates that when people are great and they know it, they tend to slack off. But when they're at the bottom, and are told they're doing terribly, they are de-motivated," says Barankay.
His research also challenges the idea that rankings could provide poor-performing employees with empirical feedback that will dissuade them from staying in their jobs -- at no great loss for the employer. "There has been this sense that people on the bottom will realize they're in the wrong job and just leave, which would also be beneficial to the company," Barankay notes. "There is also the hope that giving feedback about rank helps retain the top performers. But that's not the case. Perhaps this is because top performers move on to new challenges and low performers have no viable options elsewhere.
"Of course, in some instances, providing feedback will be a motivational tool that entices people to work harder. But overall it does not appear that way," he adds. "So the question becomes: Is [ranking employees] worth it?"
Barankay notes in his paper that future work needs to be done to test the effect of rankings in other work environments and "also to explore whether the underlying parameters can be recovered to pinpoint more detailed mechanisms in the data. Only then can we establish if targeted feedback that takes into account the underlying [differences among workers] can be established to generate a positive casual effect on performance." At this stage, however, "the aggregate result is that feedback about rank is detrimental to performance," he writes.
But while his research shows that giving feedback about rank doesn't necessarily lead to increased productivity, it is well documented that tournaments, where rankings are tied to prizes, bonuses and promotions, do inspire higher productivity and performance. When considering these two things together, a lesson emerges, he notes.
"In workplaces where rankings and relative performance is very transparent, even without the intervention of management ... it may be better to attach financial incentives to rankings, as interpersonal comparisons without prizes may lead to lower effort," Barankay suggests. "In those office environments where people may not be able to assess and compare the performance of others, it may not be useful to just post a ranking without attaching prizes."
The critical lesson for employers is to consider how each employee will respond to feedback and then decide whether sharing that information will be beneficial for everyone involved. "A good employer knows its employees very well and should have a good idea how they will respond to the prospect of being ranked," he says. "The key is to devote more time to thinking about whether to give feedback, and how each individual will respond to it. If, as the employer, you think a worker will respond positively to a ranking and feel inspired to work harder, then by all means do it. But it's imperative to think about it on an individual level."
Thanks to Knowledge Wharton
