Survivorship Bias: Why classical musicians might not want to think like rock bands

This is going to be the first in a series of posts exploring narrow ideas of “Success” in discussions from some Classical Music Crisis folks.

Survivorship bias also flash-freezes your brain into a state of ignorance from which you believe success is more common than it truly is and therefore you leap to the conclusion that it also must be easier to obtain. You develop a completely inaccurate assessment of reality thanks to a prejudice that grants the tiny number of survivors the privilege of representing the much larger group to which they originally belonged.

This comes from a piece by David McRaney on Survivorship Bias and could typify the “Classical Music Crisis” stance that Pop Music models for success in Classical Music is the way to go. One of the latest pieces in this vein is by Ivan Trevino, drummer for alt-Classical Cello Rock group Break of Reality, and his recent piece in the Strad Magazine. The article, “Classical musicians should think more like a rock band,”* is filled with anecdotes that John Chittum has challenged relatively thoroughly here so I won’t rehash most of what he says. I’m much more interested in the type of reasoning and biases that create this way of thinking. McRaney’s piece does a wonderful job of outlining the bias that drives how we view success and the models for it we’ve uncritically accepted.

Survivorship Bias and WWII Bombers

Abraham Wald’s diagram of bullet damage before (left) and after (right) the flight. Shaded areas indicate the locations of bullet damage to returning aircraft.

McRaney tells the story of how mathematicians and statisticians came to be heavily used during World War II and gives a vivid example of how one of them, Abraham Wald, prevented the US military from committing the Survivorship Bias in the problem of armoring bombers which, during the early years of the war had a roughly 50% chance of returning home after a run.

As McRaney explains it:

How, the Army Air Force asked, could they improve the odds of a bomber making it home? Military engineers explained to the statistician that they already knew the allied bombers needed more armor, but the ground crews couldn’t just cover the planes like tanks, not if they wanted them to take off. The operational commanders asked for help figuring out the best places to add what little protection they could.

[…]

The military looked at the bombers that had returned from enemy territory. They recorded where those planes had taken the most damage. Over and over again, they saw the bullet holes tended to accumulate along the wings, around the tail gunner, and down the center of the body. Wings. Body. Tail gunner. Considering this information, where would you put the extra armor? Naturally, the commanders wanted to put the thicker protection where they could clearly see the most damage, where the holes clustered.

That’s exactly what the Survivorship Bias would lead us to conclude. If the surviving bombers had damage in areas that are visible, then it would make sense to add more armor plating in those spots to increase their chances of surviving. To make the analogy with Rock Bands: If surviving (i.e. successful) Rock Bands show strengths in certain areas, then all Rock Bands should mimic (i.e. “add more armor”) to those areas to increase their chances of surviving.

Not so, says Abraham Wald. As McRaney explains:

Do you understand why it was a foolish idea? The mistake, which Wald saw instantly, was that the holes showed where the planes were strongest. The holes showed where a bomber could be shot and still survive the flight home, Wald explained. After all, here they were, holes and all. It was the planes that weren’t there that needed extra protection, and they had needed it in places that these planes had not. The holes in the surviving planes actually revealed the locations that needed the least additional armor. Look at where the survivors are unharmed, he said, and that’s where these bombers are most vulnerable; that’s where the planes that didn’t make it back were hit.

Taking survivorship bias into account, Wald went ahead and worked out how much damage each individual part of an airplane could take before it was destroyed – engine, ailerons, pilot, stabilizers, etc. – and then through a tangle of complicated equations he showed the commanders how likely it was that the average plane would get shot in those places in any given bombing run depending on the amount of resistance it faced. Those calculations are still in use today.

The military had the best data available at the time, and the stakes could not have been higher, yet the top commanders still failed to see the flaws in their logic. Those planes would have been armored in vain had it not been for the intervention of a man trained to spot human error.

A question should be forming in the front of your brain at this point. If the top brass of the United States armed forces could make such a simple and dumb mistake while focused on avoiding simple and dumb mistakes, thanks to survivorship bias, does that mean survivorship bias is likely bungling many of your own day-to-day assumptions? The answer is, of course, yes. All the time.

Most Rock Bands aren’t “Superstars”

The Economics of Superstars has been studied nearly as long as Cultural Economics has been a field.  Back in 1981, Sherwin Rosen opens his seminal piece with three examples of the economic disparities of Superstars in the fields of full-time comedians, bestselling elementary textbooks on economics, and classical music.

The market for classical music has never been larger than it is now, yet the number of full-time soloists on any given  instrument is also on the order of only a few hundred (and much smaller for instruments other than voice, violin, and piano). Performers of first rank comprise a limited handful out of these small totals and have very large incomes. There are also known to be substantial differences in income between them and those in the second rank, even though most consumers would have difficulty detecting more than minor differences in a “blind” hearing.

We love our idols, i.e. Superstars. They feature prominently in media (both traditional and now digital) and we have more info about them than we could ever hope for even when much of it might be speculation. The thing is, there can only be so many superstars in any field at any time and in addition to getting disproportionate amount of revenue in their fields they often get a disproportionate amount of media attention. We don’t usually do feature articles on the Classical Superstars that never happened.

Neither do we often have features on the local Rock Bands that never break out into a regional, national, or international level. In the context of the Survivorship Bias, these are the bombers that never made it back home. By focusing on the success stories and what conditions made those happen, we leave out the stories of the much more numerous musicians who don’t have those levels of success, and as a result we’re informed by a biased sample of what success entails which leads to a situation where, as McRaney states, “the difference between failure and success may also become invisible.”

“You succumb to survivorship bias because you are innately terrible with statistics.”

I’ve discussed Sampling Bias, Publication Bias, and other reasoning errors in “Classical Music Crisis” discussions at this blog, and the Survivorship Bias certainly falls within the first. McRaney gives tons of wonderful examples from various fields (even Science) where this reasoning error happens.

He also illustrates how this could happen in the arts:

Similarly, photographer Mike Johnston explains on his blog that the artwork that leaps from memory when someone mentions a decade like the 1920s or a movement like Baroque is usually made up of things that do not suck. Your sense of a past era tends to be informed by paintings and literature and drama that are not crap, even though at any given moment pop culture is filled with more crap than masterpieces. Why? It isn’t because people were better artists back in the day. It is because the good stuff survives, and the bad stuff is forgotten. So over time, you end up with skewed ideas of past eras. You think the artists of antiquity were amazing in the same way you associate the music of past decades with the songs that survived long enough to get into your ears.

In the end, “If you spend your life only learning from survivors, buying books about successful people and poring over the history of companies that shook the planet, your knowledge of the world will be strongly biased and enormously incomplete.”

And obviously, if we use anecdotal evidence from a few successful Classical Music groups which have adopted a Pop Music model, I think we should wonder how many “failures” there have been amongst those who have also done the same. This is related to the selection bias that seems to be rampant in discussions of the future of classical music that I’ve discussed relative to what we even consider to be classical music in the first place and how we can make claims about anything as long as we define the terms in our favor.

_______________________

*The article seems to be an adaptation of a post at Ivan Trevino’s website.

*EDITED for content, clarity, and minimization of polarizing langauge*