Winner Take All Incentives And Cheating

Steve Levitt of Freakonomics fame has shown that, when teacher's pay is linked to the the performance of their students on standardized tests, they are prone to cheat -- I mean the teacher's cheat.  Levitt's data from Chicago suggest that about 5% of teachers cheated to get bonuses and other goodies.  A recent New York Times article shows that this problem persists, and tells a rather discouraging story of a principal from Georgia who "erased bubbles on the multiple-choice answer sheets and filled in the right answers." And if you look check out the Freakonomics blog, there is evidence that Australian teachers cheat too.   

The kind of pressures that educators face aren't just financial incentives (although that alone is plenty of pressure as many systems reward only the top performers no matter how well everyone else does), they also risk being fired, demoted, or their schools may lose accreditation, be put on probation, and in some cases, closed for poor performance

The Times article offers an interesting quote that has implications beyond education:

John Fremer, a specialist in data forensics who was hired by an independent panel to dig deeper into the Atlanta schools, and who investigated earlier scandals in Texas and elsewhere, said educator cheating was rising. “Every time you increase the stakes associated with any testing program, you get more cheating,” he said.  

I found this quote to be interesting because a related implication is that, the more pressure that people face for performance, the more likely they are to cheat.  Perhaps the most extreme case are winner take all games.  Just watch how soccer players in the World Cup fake severe injuries to draw fouls again and again, even though they have barely been touched by opponents or not at all.

To this point, BPS research reports a new study by Spanish researchers that shows the dangers of winner take all incentive systems.  The experiment entailed online completing mazes but divided subjects into two conditions.  In the first condition, students were paid based on how many mazes they completed.  So, there were incentives, but not competition or severe pressures to succeed.  In the second condition, subjects were only paid if they completed more mazes then the other five members of their group -- so it was winner take all.  The subjects in the winner take all condition didn't perform any better, but they cheated more (the researchers figured out a clever way to catch them), especially if they were poor performers or women. 

As the researchers conclude:

'It turns out that individuals who are less able to fulfill the assigned task do not only have a higher probability to cheat, they also cheat in more different ways,' the researchers said. 'It appears that poor performers either feel entitled to cheat in a system that does not give them any legitimate opportunities to succeed, or they engage in "face saving" activity to avoid embarrassment for their poor performance.

After years of reading research and working with organizations of all kinds, I have learned to become very wary of winner take all incentives.  Or as often happens in organizations, systems where the top performers get the lion's share of the money and their more ordinary peers get a few crumbs.  These systems not only encourage cheating, unless they are managed with extreme skill, they also undermine cooperation because, "If I help you, it means I am less likely to succeed."

Well, whenever I write about these kinds of problems, people ask me what the alternatives should be, and I confess, that is a tough question.  But I do think that a few guidelines are useful:

1.  When in doubt, anoint a higher percentage of your people as "winners" and a lower percentage as "losers"

2. When in doubt, err on the side of smaller differences in pay between the top performers and everyone else.

3.  Define superstars as people who help others succeed ,not who stomp on others on the way to the top.

Those are my three. Do you have other guidelines to add? This is a tough problem and every organization that I know struggles with such issues. 

P.S. The article summarized at BPS is: Schwieren, C., & Weichselbaumer, D. (2010). Does competition enhance performance or cheating? A laboratory experiment Journal of Economic Psychology, 31 (3), 241-253

P.P.S. Also see this great article by Chip and Dan Heath on "Why Incentives are Effective, Irresistible, and Almost Certain to Backfire."

Comments

[Harris Silverman - Business Coach]

To me there are two other issues here: how you define success, and how you determine rewards.

The reality is that companies tend to define success fairly narrowly, but in fact there are lots of different ways that employees can add value that go beyond what can be measured directly, such as sales numbers. Bob's point about rewarding those who help others succeed strikes me as an attempt to address this issue.

The second problem is that different people value different rewards. Usually the reward is a bonus or some other form of pay, but the extent to which people value money over all other incentives (e.g. status, time off, a good work environment) is generally exaggerated.

Harris Silverman
www.harrissilverman.com

[Phranqlin]

I've worked for companies where managers were forced to give the top 10% of their group a high performance rating and the lowest 20% a bad rating -- regardless of how well or how poorly anyone was actually performing. The performance rating determined bonuses, promotion, and retention, so it was a big deal.

This was very demoralizing and stressful, to say the least. Claiming credit for work that others had done and sabotaging others to make yourself look better were just two of the bad behaviors that this system encouraged.

[Steve McIrvin]

You bring up a very good point, and the depressing thing is that it's been around for as long as . . . Jacob and Esau. The 2nd-born of Isaac's sons, Jacob, got a tiny inheritance compared to his older brother Esau, so he resorted to cheating to obtain his brother's birthright and his father's deathbed blessing. Does human nature ever really change?

[Greg]

A flaw in the "Winner take all" approach is that it does not take into account exceptional results of others.

I had a professor that told a story of his grade his Master's capstone course. He was one of three students. Their grade percentages were within 2% of each other (high 90's). He was awarded a "C" because the grading was based on the bell curve. Mastery of the materiel did not not count for anything.

Objective measurement is necessary. If you have a team of exceptional performers, why cut the bottom 10%? Or if you have a team of sorry performers, why reward any of them?

I think of Super Bowl MVPs...cut the rest of the team because the quarterback is the only MVP?

[Ergoboy]

Bob,

Thanks for putting up the video you did. While I don't have anything academic to add, the video demonstrates something that I suspected in the Winter Olympics this year.

In the Short Track 500m final Apollo Ohno was favored to win. He was against 2 Canadians and one or 2 others in the gold medal race. If you recall the race, Ohno passed one of the Canadians on the inside, and in the process of passing him, barely touched him. The Canadian "fell" (cheated) and skidded to the outside of the track in a dramatic fashion. Ohno was ultimately disqualified, and the Canadians both won medals.

In my opinion, there was a very interesting bit of logic and pre-meditated thought that went into this act of "cheating".

1. Apollo Ohno may be faster than I am, and faster than my Canadian team mate. If he passes me, the probability that I can pass him is very very low, and I lose anyhow. There isn't much sense continuing the race at this point, especially if I was already last.

2. If Ohno passes my Canadian teammate, we "lose" (don't get the gold), and Canada may lose an opportunity at a medal.

3. If Ohno gets DQ'ed, it doesn't matter if he passed us or not. His time doesn't count, and we still stand to be in medal contention. All we have to do is beat one other person and we get a medal. There is a high probability we can do that.

4. If there is an opportunity to "sacrifice" myself to DQ him, I will take it thus saving my teammate and securing their medal contention.

That was my hypothesis of the strategy which lead to the outcome of the race. I thought this sort of strategy was unlikely and far-fetched, since who would stoop so low to do something like that?

The video shows this type of strategy may not be all that rare and may in fact be thought out and rehearsed way in advance.

[Randy Bosch]

Bob, Fascinating and worthwhile study, and comments that are all worth pondering - success without cheating or celebrating failure!

I respectfully disagree with the "failure fast and often in order to succeed" approach so widely in use today. The human psyche is not wired that way, and the approach is not only self-defeating, but one might hypothesize that given our "wiring", cheating is further encouraged. If the culture applauds failures, then the ambitious will more easily be able to cheat and get ahead - even faking failures (e.g., the injured player fakes signify that someone failed)to succeed.

One better approach is to correctly teach experimentation, testing, modeling and the iterative process wherever creative, non-rote, work is involved - and celebrate lessons learned.

Many people are making money teaching "failure is good and necessary" and will object! How to recognize symptoms that might lead to failure, and how to mitigate failure when it occurs is a proper approach.

Dare to Fail? - BALONEY! http://wp.me/pVUDj-4w

[Ann]

It's true. As the stakes go up, cheating goes up. It's happening among students as well. If you are a C student but need an A, you cheat.

Which brings me to another factor we need to consider. Most people cannot produce exceptional results, so when the exceptional is held up as the norm, there's going to be a corresponding burst of cheating. This is exacerbated if the result isn't entirely under the person's control.

Teachers and schools aren't the only ones responsible for student performance. Parents and students have influence that amounts to veto power. A parent can have a lax attitude toward homework; a student can refuse to do the work at all. When this happens, the only sanction school personnel can impose is a failing grade.

So what happens when teachers, administrators and entire schools can be sanctioned or even fired if there are too many failing grades? They're going to cheat, because that's the only way they can control the outcome.

Lowering academic standards is cheating. Teaching to the test is cheating. Measures like altering test papers are what happens when the legal forms of cheating still aren't enough.

Parents are also cheating, some openly by doing their children's work for them, and some subtly, by holding a child back at the start of school. The higher the academic standards for elementary school students, the more likely it is that parents will delay putting their child in school because they know the kid cannot handle the material yet. It's gotten to the point where high school graduation age is rising in some areas, and they tend to be the same areas where kids are expected to start kindergarten, or even preschool, reading.

Just because some kids can read at age four doesn't mean that all kids should be expected to read at age four.

By insisting on excellence as the standard and punishing even average results, never mind poor results, a system of any kind fosters cheating.

[Jake]

I would agree about relationship of pressure and cheating.

I have worked with sales incentives for a number of years with a fair amount of time spent on auditing results. I found that vast majority of cheating was driven by performance pressure. The person was either trying to avoid the pain experienced by under performers or they were at risk at losing their existing status (i.e cheating to prevent slipping from #1 producer to #2 producer).

Rules;
#7: Have multiple performance metrics that can be reviewed more than once a year.

#8: Audit results and have consquences for cheating. Fear of getting caught will help keep some of the potential cheaters honest. However,no matter happens, organizations will encounter people that will cheat. This will give them a process for identifying and removing this people from the organization

[Bruce Lynn]

Unfortunately the focus of this post faulty by only looking at one side of the cost/benefit equation.

If the pay/benefit is fixed (ie. no upside incentive), then the 'cheating' is focused on reducing the input/cost to get the same fixed pay (eg. malingering, feigned productivity, absenteeism, taking advantage of any loopholes to get out of any work or effort). The psychologist's/researcher's game is not relevant here because there is no 'cost' to the participant.

I would argue that motivation to 'cheat' is a constant and the rules and incentives one puts in place just shifts the form of cheating. To argue evidence of correlation between cheating and incentives is like arguing that there is a correlation between breathing and cheating (maybe we should cut off oxygen in the workplace). It's human nature. A similarly flawed logic would be "the more you tighten speed limits, the more people cheat/break them, ergo speed limits are bad."

There two fundamental motivators in life - Fear and Greed. The two balance each other and systems need to work to maintain a healthy balance. Maslovs Hierarchy of needs is a richer model of what one is Greedy for (going up the pyramid) and Fearful of (going down the pyramid).

I would agree that the more you raise the stakes, the more motivation to cheat (eg. fear of death leads to some unconscionable behaviour, a million dollar proposal more inspiring to cheat on one's spouse).

4. Don't be seduced into a utopian fool's errand that 'the right work environment' eliminates cheating.

5. Build performance management systems that align opportunities for upside with concerns about downsides with corresponding rewards and penalties for each.

6. If the business benefits from collaboration and teamwork, do not institute mutually exclusive rewards/penalties. Foster teamwork with rewards for evidence of teamwork (a broader version of Bob's #3). For example give equal rewards for assists as for goals.

[Liam-og Griffin]

Interesting article... well written... one question:

"...but they cheated more (the researchers figured out a clever way to catch them), especially if they were poor performers or women..."

Is it just me or is the 'women' comment left hanging? Is there more detail on that?

[Paul Hebert]

Bob - I can't agree with you enough. I'd also add the following:

Don't assign incentives to "results" - assign them to behaviors. Behaviors I can control and therefore have a direct link from my effort to the reward. Incentives based on outcomes (while they make it easier to do ROI calculations) allow participants to create their own behaviors in pursuit the prize.

Recognition (ie: after the fact awards) work well for those that achieved specific levels of performance that can be held up as norms for performance.

Check the reward against the behaviors - is the reward too rich for the effort required - this is one area that is abused often. As you mentioned - smaller awards don't drive negative behavior to the same degree.

While we're talking incentives let's not forget that companies also need to reinforce culture through punishment. Anyone who cheats should be either reprimanded or (as I would do) fired. That is a behavior that shouldn't be tolerated.

Too often we tolerate one behavior and reward others sending mixed messages.

As others have mentioned - team-based awards are the more effective way to drive behaviors. Today's work is rarely the result of one individual's effort. Don't reward what isn't reality.

[Dwayne Phillips]

How about this, create a situation where you cannot cheat.

For example, in track and field there is the long jump event. You can cheat by stepping over the take off line when you jump. You should be disqualified, but sometimes the judges don't notice a little cheating.

Change the rules: erase the take off line, simply measure the distance from where ever the person leaves the ground to where he lands.

In World Cup soccer, change the rules. If a person is lying on the ground (faking an injury or not), simply keep playing the game without him. The team suffers as they now are playing 10 vs 11.

[Stu]

6. Find situations where the pie is expanding. The impetus of cheating and conflict in a winner-take-all system is that it's a fixed pie. If I win, you lose. It's even worse when the pie is shrinking, like at a failing company - everyone is grasping for their piece of glory. That's why it's better to work on teams where the pie is growing. Everyone works hard and there's enough reward to go around so everyone's well-compensated.

[Rob]

4. As a manager focus on the effort put in by all people.
Over the long term everybody driving hard and not giving up is a key group success factor.
5. Celebrate both successes and failures (failures more so), each failure is a much bigger learning curve and "change for the better" trigger, people should be comfortable that they can have failures at first and as long as they learn from it and get to the result over the long term this is the key factor in their improvement.