Perverse incentives

They are more perverse than you thought

Wikipedia defines a perverse incentive as "an incentive that has an unintended and undesirable result that is contrary to the intentions of its designers." The reward someone gets is bigger when they start doing the wrong thing. As a short aside the AI safety community has studied this problem in the context of how to get an extremly intelligent AI to behave in a desirable way, which they refer to as The Alighnment Problem. It's unsolved and the general consensus is that it is extremely hard.

But let's leave that topic for another time and consider normal humans, and human organizations. It's plainly obvious that a crooked person might follow the perverse incentive to get more money. And even the typically-honest person may waver just a tiny bit.

However, one thing that is rarely acknowledged or understood is that perverse incentives can be a problem even when everyone involved is perfectly good and honest. Stop for a minute and consider how that might happen.

Ok, whether you did or not, here is the answer: The incentive will raise the incompetent to the top and push the competent to the bottom. This might become clearer with an example.Private prisons have a perverse incentive in that they need criminals to house. No criminals no business. Consider two private prisons. Each is run by a truly well meaning ceo, and similarly the staff are certified good people. They all do their very best to help rehabilitate their inmates and turn them into upstanding members of society. However they have different opinions of the best way to accomplish that.

In the first prison, they take a practical approach. Reasoning that the cause of crime is that people don't have a choice they try to provide one. They teach their prisoners vocational skills and try to set them up with day-one jobs and housing so that the transition to normal society will be as smooth as possible.

In the second prison, they take a spiritual approach. Reasoning that the cause of crime is a spiritual taint, they focus on purifying the heart. Prisoners are taught religion and philosophy, they read poems and psychology. They are taught how to emphatize and help other people.

It doesn't matter which, but one of these two appraoches works out better, and the other works out worse. One prison gets their recidivism rate so low that it starts cutting into their inmate population. Crime is at an all time low in their area.

Sometime later the "better" prison goes bankrupt. It is sold to the "worse" prison. Now that it is owned by the other prison it is made to adopt the practices of it. Gradually crime starts increasing again and business is booming.

This whows up the system itself can end up psychopathic without a single participant intending it, or even realizing that it is happening. In the example the difference in outcome came from using different approaches, another possibility is quality of implementation.

A vocational teacher that speaks with an incomprehensible drawl, that is boring, often home sick could be good for business. A psychologist that uses outdated theories, that misunderstood most of what he learned, that is unpersonable and fails to inspire change - that could be good for business. The thorough hiring manager leads his business to ruin, and it is then taken over by the sloppier one.

However all is not bleak of course. There wont be an endless spiral of ever worsening competitors. Things can only get so bad before it becomes obvious that something is wrong.