George Rebane
Years ago when our kids were in grade school, our dinners involved discussions of matters that most people considered too deep for children. Our kids knew nothing of this and participated wholeheartedly – kinda like the bumblebees who don’t know that they’re not configured for flight but fly anyway. As a nascent systems techie, practicing aerospace engineer, and perennial grad student, I was very involved with the science of uncertainty, risk, and things barely known. One of the things that intrigued me and Jo Ann as parents of young children was how we try to communicate our own assessments of uncertainty, and how these assertions are understood by others.
The thing that really set off an investigation of this was my reading of how the Bay of Pigs disaster was approved. It turns out that President Kennedy charged the Joint Chiefs to assess the CIA’s plan for the expatriates’ invasion of Cuba. The general in charge of the study reported to JFK that there was a “fair chance” of success for the invasion. With this input, the president gave the go ahead, and the rest is history. Monday morning quarterbacking the disaster, the general was interviewed and opined that he intended ‘fair chance’ to communicate about a 30% chance of success. JFK hearing of this later said that he never would have approved the invasion had he known that success came with such a low probability.
In the 1970s Kahneman and Tversky did a lot of research on the matter, publishing their landmark results in ‘Judgement Under Uncertainty: Heuristics and Biases’ (1974). Having read this as part of my day job, we decided to run an experiment to see how well we communicated uncertainty within our family circle given the wide range of family member ages involved. We generated a list of uncertain statements based on how many black marbles there were in a bowl of a hundred other colored marbles from which a marble was drawn randomly. These statements were more or less similar to the ones reported in Behn and Vaupel (1982) who cited a study by Donald H. Woods published in the Harvard Business Review that involved a large group of student subjects to “test the communicative content of some common probability phrases.” The results were intriguing.
To give you an example of some of the phrases to which the students were asked to assign a probability, consider the following –
- There is a much better than even chance that …
- There is a possibility that …
- There is a high likelihood that …
- There is little chance that …
- There is a high probability that …
- They will probably …
- The possibility is low that …
The results were reported in a columnar format with headings ‘Lowest’, ‘Lower Quartile’, ‘Median’, ‘Upper Quartile’, ‘Highest’ for each of the presented probability phrases which the students were asked to assess according to their own lights.
When the Rebane family did our assessments of the number of black marbles out of a 100 that each of the phrases represented for us individually, the results were striking. All five of us flew in very tight formation, especially given the wide range of ages from about seven to thirty-six. Jo Ann and I had a hard time believing the results until it occurred to us that we all learned to communicate uncertainties with each other within our tight family circle. With this in mind, it was not unusual that our linguistic biases had formed a tight and very effective quantitative group; we learned from each other.
Now that result does not imply that other families' assessments of the same phrases will match that of the Rebanes. But it does suggest that each family or closely working cohort will, over time, develop a similar understanding and expression of everyday uncertainties. And it also strongly suggests that people from different cohorts attempting to communicate uncertainty should augment their probabilistic phrase with a quantitative statement such 20%, or 45-55, or 99 out of a hundred. At a minimum this would go a long way to preventing the redux of Bay of Pigs disasters.
OK, so what? Well, in these pages we discuss and debate a lot of issues and notions which have a bit or more of uncertainty connected with them. A lot of our debates also generate more heat than light because we don’t successfully communicate our uncertain beliefs. I’d like to see what kind of a spread we have, or how tightly we agree, given the Harvard probability phrases. If you want to participate and see how you stack up with RR readers, then download, fill out (your numbers in place of the blue Xs), and email back to me a copy of this spreadsheet – Download ProbPhrases – shown below. I’ll do the tabulation and post the results which, if a sufficient number of you respond, should be very interesting. And I don’t need to go into a long dissertation on how such communications (of carefully wordsmithed phrases) in the media impact people’s views about almost everything that is reported. BTW, you can copy the spreadsheet and do this experiment with your family, friends, and/or colleagues. I guarantee that the results will be very revealing, and useful for future communications.
Behn & Vaupel (p77) report that “ambiguous probability phrases also plague legal communications. For example, there is much confusion over the commonly used statement that, for a conviction, “a reasonable man” must believe that the evidence indicates a person to be guilty ‘beyond a reasonable doubt.’ A survey of 347 judges found that some thought this meant that there had to be a 70-percent chance that the defendant was guilty (one judge even thought this meant only a 50-percent chance) and nearly a third of the judges thought it meant 100 percent. A group of 69 jurors gave similarly divergent responses.”
And all of us continue using such phrases while believing that we are communicating effectively.
Comments