How perverse incentives, rather than evil deeds, corrupt society
People often think that society crumbles under the weight of evil conspirators. And although they do exist, the real rot that destroys a society is not deliberate evil, but a system which rewards it.
When a society provides people perverse incentives, people often choose the path of least resistance, undermining the prosperity of society.
Before we understand how incentives create perverse results, we need to understand five principles that guide human behavior.
(1) Actor-observer bias
The actor-observer bias is a psychological phenomenon that allows us to often point the finger at others while excusing ourselves.
The principle states that when we act, we judge ourselves by our intentions, our internal monologue, which demonstrates to us the factors that lead up to an event.
On the other hand, when we judge others, we observe their behavior and have no access to their internal dialogue.
We see, for example, that they are late for work. But we don't see the argument they had with their spouse beforehand and the rebellious child that wouldn't brush their teeth.
(2) Energy conservation principle
All living creatures share a powerful biological bias toward energy conservation. The biological default setting for all living things is to remain at rest unless compelled to act, as remaining at rest in non-urgent circumstances is deeply associated with long-term survival.
This means if you provide people a job to do and give them no incentive to do that job, many people will do as minimal a job as possible.
85% of employees worldwide are "not engaged or are actively disengaged" in their jobs. - Gallup
(3) The hedonic principle
A foundational pillar of behavioral psychology is the hedonic principle, the idea that human behavior is fundamentally driven by the motivation to maximize pleasure and minimize pain, guiding choices towards positive feelings and away from negative ones.
(4) Campbell’s law
Realizing that people need incentives, managers of companies or government organizations set incentive structures up that they can measure.
better grades
more arrests
higher ticket sales.
Oftentimes, these mask or stand in for real results:
Better grades → Student learning
More arrests → Safer streets
Higher ticket sales → Company earnings.
The metric stands in for the real goal. This is where Campbell's Law comes in. It states: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
(5) Unideal decision space
Life rarely presents us with decisions clearly labeled good or evil. Instead, life presents us with a series of decisions that are easy or hard, with shades of good or evil mixed in.
Tying them together: an example
John is a life insurance salesman being trained by his boss Alex to sell life insurance.
Pure intentions (actor-observer bias): As a new employee, he is eager to make his boss happy.
Placing incentives (energy conservation principle): His boss, Alex, has decided to motivate all his employees by telling them that any salesperson who fails to meet the quota by the end of this month will be fired.
After much training, John is ready to sell insurance and his boss has given him a tough quota to meet. Luckily for John, the lead is about to turn into a customer and John is going to be paid a lucrative commission. That's when John asks a critical question, and the answer given by the customer totally precludes their need for buying insurance beyond all reasonable doubt.
Avoiding suffering (the hedonic principle): John knows he has to meet his quota to stay at the job and that his family is depending on him.
Unideal decision space: Now John is faced with a difficult choice. Does he sell insurance to a person who doesn't need it or does he refrain and risk dropping below his quota?
Campbell’s law: The goal of the business is to make money by providing value and often that value is better provided by being honest in the long term. However, since the goal was simplified to number of sales, the perverse incentive ripples throughout the business, encouraging people to harm others rather than provide genuine value.
How corruption changes identity
Human beings tend to let their actions flow from a sense of identity. People who say to themselves, “I'm a good person”, want to protect that identity by doing good things.
Unfortunately, when people like John choose to sell the insurance, they undermine their own identity and they begin to fill in the gap with fake acts of good to make up for the action which contradicted their sense of self-identity.
These good acts are essentially hollow and part of an anxious rationalizing process.
Not only that, a person's sense of being good may collapse, and then having no self-reputation to defend, they let loose and begin to do all sorts of evil.
Michael Dowd as an example
As an NYPD officer, Michael Dowd faced a system that made arresting drug dealers totally impractical. There were just too many and each arrest cost 18 hours of overtime work.
Eventually he realized he could both profit off of the criminals and mitigate crime by “taxing” them.
To the average citizen, he was a good cop. "If you're not doing those things," he argued, referring to the drug trade, "you're safe with me and I will give you the best police service that you ever asked for." He judged himself by his intention. But his behavior was that of a predator. "If you were in the drug business," he stated plainly, "you were mine... I owned you."
Unfortunately, this led to the corruption of his identity. "I became the environment I was living in... no different than the people that were selling crack cocaine."
With nothing left to defend, he found himself protecting drug kingpins as a corrupt cop.
Poor incentives led to poor choices which corrupted his whole identity and through him led thousands to disaster.
This might seem like an uncommon example, but more often than not, these realities can be seen throughout society.
These are the sorts of smaller questions which can lead down a much more precarious path.
Mollen commission example
The Mollen Commission investigated police corruption in the early 1990s.
Police, needing an incentive, settled on arrest counts as an approximation of safety.
Unfortunately, the Mollen Commission found pressure for high arrest numbers led to framing schemes.
Undercover officers lured women to hotel rooms containing pre-placed cash. A woman's mere proximity to the money, absent any explicit agreement, was then used as the sole basis for a prostitution arrest.
This allowed officers to easily meet quotas by targeting vulnerable individuals whose testimony was unlikely to be trusted.
Examples throughout society
No one likes to believe that the people they work with are evil, so it's easy to brush these excesses under the rug, but perverse incentives can be seen to be corrupting society across our institutions.
Short call times → Customer frustration
Support agents hang up on difficult problems or transfer customers endlessly to keep their average call duration, a key metric, low.
Standardized test scores → Ignorance
Teachers narrow the curriculum to "teach to the test," ignoring subjects and critical thinking skills that are not measured.
Conviction rates → Wrongful imprisonment
Prosecutors suppress evidence or pressure innocent defendants into plea deals to maintain a high win record.
Lines of code → Software productivity
Programmers write bloated, inefficient code because they are evaluated on volume rather than elegance or stability.
Healthcare cost savings → Sicker patients
Insurers deny necessary treatments or discharge patients too early to boost short-term profits, leading to severe long-term health complications.
Surgery survival rates → Untreated patients
Surgeons refuse to operate on high-risk patients who desperately need help to protect their personal success statistics.
Engagement → Sensational journalism
Media outlets publish sensationalized or misleading headlines (clickbait) to generate ad revenue, degrading public discourse.
Survival of the fittest
In any scenario in which individuals within a profession can become more successful than their honest counterparts by playing to the metric rather than the metric’s intended impact, that profession is overtaken entirely by those who cheat. Only the cheaters “survive” weeding out the honest.
For example, people wonder why politicians have such a reputation for lying. In a world where politicians are not held to account for lies, lying gives politicians a higher chance of survival and natural selection weeds out the honest.
No matter how “good” people are, if the incentive structure favors cheating, the “good” people will be weeded out.
The solution
It's easy to blame professionals in different fields for being evil. But in reality, in almost all cases, a poor incentive structure is at fault.
Hanlon’s razor
"Never attribute to malice that which is adequately explained by stupidity."
In reality, it's not the malicious intent of people, but the lack of time and effort spent on building better incentive structures that leads to disastrous results.