Appreciating the little things - Rush Hour + Joshua Bell + The Washington Post = not an experiment

This video and the the description that accompanies it on YouTube tell the story of how Joshua Bell (a famous violinist who played to a sold out audience the night before) played Bach on a $3.5 million violin in a Subway at Rush hour and almost nobody noticed.

It is presented as an experiment, which you might be able to justify if you incorrectly defined an experiment as doing something and checking to see what happened without worrying about why it happened, with astounding results. However what is truly remarkable is that it is neither an experiment nor an event with astounding results. Here's why:

It's not an experiment

In an experiment you try to test an idea. You do this by carefully adjusting one variable in a situation at a time while holding everything else constant. In an experiment you are in control and you measure the effect that changes you make to one thing (say the price of the violin in this case) have on the measure that you are interested in (people's tendency to stop and listen). Here's why the situation they describe is, at best, an awful experiment or, most accurately, an anecdotal observation

A comparison is made between the people in the Subway and the people in the concert hall the previous night. It is noted that those in the Concert Hall sat and listened intently to the performance while those in the subway did not. The reasons for this difference is suggested to be that when we don't have time to appreciate beauty we don't appreciate it. While this is possible it is by no means proven and this 'experiment' is incapable of addressing this idea. Why? It has failed to control for, and therefore has failed to rule out, the 4 following potential explanations among many others:

1) The audience
The people in the Subway and the people in the concert hall are, in all likelihood, not the same people. As such there are any number of potential reasons for them not stopping to listen to the music. While the fact that it is rush hour may be one of these other plausible explanations include a different taste in music, a dislike of subway stations (reducing the total beauty content of the situation), and even a higher proportion of deaf people (deaf people are unlikely to be well represented at a concert). These are just a few reasons.

2) The beauty signals
Those attending the concert the night before would have been given all kinds of signals about how beautiful they should think the music was. For one thing they probably knew what was going to be played, who was going to play it, and how much they had to pay for a ticket. The combination of all these branding elements sends a stronger signal to pay attention to what is happening than a guy standing in casual dress in a subway station does. As such it is entirely possible that it wasn't the time pressure that stopped people perceiving the beauty but the fact that the performance itself may have less beauty in it than we would otherwise expect.

3) The measure doesn't match the conclusion
This one is actually the most important of the bunch. While the authors talk about our inability to notice beauty this is not what they actually measured. They didn't stop anyone and ask if they noticed the music and, if so, what they thought of it. They measured the length of time people stopped for and the amount of money they placed in the violinist's case.

4) The change in sound Quality
A quiet concert hall sounds far different to a busy subway station at rush hour. If people were less likely to notice the beautiful music in the subway station it could be because it was less audible over the surrounding noise or it could even be that the acoustics of the station itself made the music sound less beautiful. This failure to control the quality of what your testing with is a fatal flaw.

It's not actually surprising

While the reasons I have outlined above are probably enough to convince anyone that the experiment performed by the Washington Post was destined to succeed in finding a difference between the number of people listening to a violinist play a piece in a concert hall and the number of people listening to the same violinist playing in a subway there are a number of other things to keep in mind. These findings are well established through strong experimental testing but are somewhat less sexy than a violinist in a subway when addressed in isolation.

1) The best predictor of whether someone will stop for someone else is whether they are in a hurry or not
There is a famous psychological experiment that pitted priests in training against the general public. Who would be the most likely to stop and help someone in need? As it turns out, whichever person was not in a hurry. Stopping then appears to be a function of hurry.

2) Conformity
Like it or not we try not to be special unique snowflakes. Instead we try to fit in with what everyone else is doing. If everyone else at the concert is listening to the musician so do we. If everyone else is ignoring him and trying to get to work, so do we. We even engage in behaviours that run completely opposite to our own beliefs.

3) Schemas
We tend to act the way we think we should act in any given situation. In a subway we think we should be in transit so we travel from point A to point B. When in a concert hall we think we should listen attentively to the music and appreciate its beauty even if we don't really know what that means. When we see someone dressed casually playing in a subway with no indicators of importance we take this to mean that they are a busker. When we see someone dressed well in a concert hall we take this to mean they are a professional. As such behaviour doesn't really give much of a clue as to what we think of the music or whether we notice. It is probably a better indicator of how we think we should behave.

4) Most people are useless at discriminating between high and low quality unless they have specialist knowledge.
Note in the description of the video they seem to suggest that the violin should be beautiful and produce beautful music because it costs $3.5 million. If we recorded music from that violin and a cheaper violin under equal controlled conditions and presented it to the author of the article and a few hundred other people without telling them which was which do you think they would be able to tell the difference? In most cases the answer would be no. This does not suggest that we can't identify beautiful music in a Subway station because it's in a subway station. Instead it suggests we can't tell the difference between beautiful music and non-beautiful music unless we are told that the music will be beautiful or not beautiful. If it's being played in a concert hall we are likely to think it must be beautiful because the setting tells us so. If it is being played in a subway we might think the person playing it is quite talented but, because they're in a subway, obviously they're not that talented and the music then can't be that beautiful.

Taking these factors into account the results are not surprising in the least and also do not suggest that rushing it what causes us to miss beauty. It may mean we are less likely to stop but that is not the same as saying that we do not notice it.

Maybe their results just mean that Bach being played on a 3.5 million dollar violin isn't actually that beautiful to most people

That is an equally valid interpretation of the same data. This time instead of blaming the public for being too busy I'm blaming the classical music crowd for being out of touch. Instead of saying "Aha! When people are in a hurry they don't notice beauty" I have said "Aha! When people have nothing else to do and have paid to listen to music that they like enough to pay money for they are more likely to listen to that music".