In preparing my talk I reviewed some of what has been written on Decision Based Design. What I learned made me realize that I have the obligation to begin my remarks today with a precautionary note. My home discipline of cognitive engineering and the design of human-machine systems is, like yours, heavily concerned with both design and decision making. But where your own problem seems to concern supporting the engineering design process using the models and methods of the decision sciences, the central problem in my field is how to support decision making through effective design. As you can see, one of these problems is framed backwards with respect to the other, but I hesitate here to say which one.

For us in cognitive engineering, it is the decision making that is the tough, nebulous problem, specifically, how to support the decision making of pilots, physicians, power plant operators, manufacturing system controllers, and the like. On the other hand, engineers and technologists concerned with human-machine systems seem to have absolutely no problem designing all kinds of new automation that keeps us rushing to keep pace. For us, then, the goal is to figure out how to inform and support these designers, so the end users of their designs are better decision makers. Mainly through trial and error as much as anything, we have learned that this problem is much trickier than we originally thought.

I am going to take some time today to explain why the problem of supporting decision making through design has proven to be so difficult, in the hope that there might be a lesson or two here for people endeavoring to support design through decision making. At one level, of course, we are both engaged in the same endeavor: The desire to improve some aspect or product of human cognition by the use of analytical methods. Engineers are analytical people, that's what we get paid for. And we seem to be most successful, and perhaps happiest, when the products of our analyses are translated into reality in the most pure and unadulterated form possible. Today, this usually means in the form of technology that realizes our ideas as computer algorithms, circuit designs, bridges, and the like.

But it turns out that when the goal is to impact people's actual work practices, how they make judgments and decisions in their day to day jobs, the engineering mindset that is well suited to producing stand-alone artifacts is woefully insufficient. In the worst case we stand the chance of violating our Hippocratic oath and actually doing more harm than good. In the best case we will simply be ignored.

Now I'd like to cut to the chase and give you the moral of my talk today up front, and take the rest of my time to try to convince you of its truth or utility. I will introduce this moral in the form of an apparent dilemma, a dilemma that has arisen out of the past 30 or so years of cognitive engineering research. I strongly believe that the path toward effectively supporting the judgment and decision making of experienced professionals depends on how we resolve this dilemma. That is, it depends on our own creativity in seeing how the following two claims are only apparently, but not actually, in conflict.

Claim 1: The judgment and decision making of experienced practitioners can be studied, described, and understood with analytical methods and models.

Claim 2: The judgment and decision making of experienced practitioners cannot be usefully enhanced by trying to make practitioners behave more analytically or rationally.

In order to motivate these claims, and pave the way toward the resolution of this apparent dilemma, I need to give an extremely brief historical overview of cognitive engineering.

The field got it start as Human Factors around the time of World War II, where the problem was to effectively couple soldiers, sailors, and aviators with increasingly complex equipment, vehicles and weapons systems. The psychological issues relevant at the time consisted of what we now think of as "low level" abilities: vision, perception, motor skills, and the like. While so-called "knobs and dials" human factors problems still exist today, in large part many of these human factors problems and questions have been made moot by technological advances. Advances in technology allowed designers to automate many of the low level control functions previously performed by people. As a result, the role of the human operator in human-machine systems over the past 25 years or so has shifted from manual control to a more supervisory mode of control. As you are probably aware, the pilot of a modern airliner is only rarely engaged in manually flying the aircraft. His or her major function is to monitor and manage the collection of automated systems that perform the actual control and navigation functions.

What was the impact of this shift in the humans role on human factors? Most importantly, it quickly became clear that we needed much better knowledge of higher level abilities such as judgment and decision making, as these were the primary activities performed by human operators, and the majority of accidents and incidents could be traced to failures or errors in performing these cognitive functions. Fortunately, at this same time a variety of relatively new scientific developments provided researchers with the tools to investigate human judgment and decision making in a scientific fashion. Here I am referring to advances in cybernetics, economics, and the decision sciences, which provided analytical models of decision problems for which we could calculate optimal solutions.

Thus, in the late 60s, 70s, and the early 80s a tremendous number and variety of studies were performed investigating strengths and weaknesses in human judgment and decision making. However, all these studies had one thing in common. Namely, they all involved presenting humans with some form of judgment or decision problem for which the researcher had calculated the normatively correct solution. These normative solutions were calculated on the basis of frameworks such as utility theory, bayesian reasoning, probability theory, and sometimes even basic logic.

Here is a classic example:

Below are four cards, each with a letter on one side and a number on the other. What cards would you turn over to test the validity of the rule: if there is a vowel on one side of the card, there is an even number on the other side?

What can be learned from experiments such as these? This particular problem gave rise to the idea that people display a confirmation bias, or a tendency to wrongly seek out possibly confirming rather than disconfirming evidence when seeking to test the truth of a hypothesis.

Research in this tradition, (that is, comparing human behavior to the prescriptions of normative models) has resulted in a list of more than a dozen biases to which people are generally susceptible, and has painted an overall picture of human judgment and decision making as being terribly flawed. Reading this literature sets one to wonder how the human race has survived as long as it has.

But before we move on let's do a second example:

Below are four personal checks. You are a cashier at a store which has the policy: "If the amount of the check is greater than $100, then the manager must approve the check by signing it on the back." At the end of your shift you are going through the checks you have taken to ensure that you have complied with the store policy. Which of the following checks would you turn over to make sure you have complied?

What can we learn from this case? Note how easy it was for you to see that checks "a" and "b" needed to be turned over, while "c" and "d" clearly did not. And this problem has identical logical form to the first problem, posed more abstractly, above! Why is this problem easier? The answer is that we have the benefit of experience with situations such as the one above, and rely on that experiential knowledge to short circuit the demand to think analytically about the problem. We have many such experiences: "if you don't clean your room you won't get desert for dinner!"

Now, I'd like you to consider one last problem:

A child has been kidnapped from his bedroom on the second floor of a suburban home, sometime after 9pm. A ladder is found leaning against the house adjacent to the child's bedroom window. It is known that it snowed between midnight and 1am and no footprints were found in the snow at the time when the kidnapping was discovered in the morning. What is your best estimate of when the kidnapping must have occurred?

Aside from how simple this problem seems, what is the big difference between this problem and the previous two?

Note that in this case there is no available normative model we can use to derive the solution but we would all agree that the correct answer is between 9pm and midnight. How did we make this inference in the absence of a normative model? We used our experientially grounded knowledge of causation, human behavior, the properties of snow, and possibly much more to come to our conclusion. There simply is no normative model that could perform this task. And the lesson from the history of artificial intelligence is that it will be quite some time before we will achieve such a model.

What is the moral of the story so far? Normative approaches such as Bayesian and utility theoretic models trade a lack of experiential knowledge for sophistication in calculation. Human cognition trades a lack of a talent in sophisticated calculation for experiential knowledge. What sets chessmasters apart from players of lesser ability is their huge store of knowledge about game positions and what to do about them, not any superior ability to calculate decision trees along the lines of IBM's Deep Blue computer. We tend to call this form of expertise "intuition" but a chess master's ability to recognize thousands of board positions is no more mysterious than your ability to recognize thousands of words or human faces.

In the last 10 years or so the research focus in cognitive engineering has shifted away from comparing human cognition with the prescriptions of normative models and toward understanding the experiential bases of professional cognition. In none of the application domains in which I work is any engineer confident enough of the purely analytical methods that they would seriously propose taking the pilot, physician, or even the lathe operator completely out of the decision making loop. Thus, the problem is how to couple practitioner intuition with the resources and products of the decision sciences in the most effective way possible.

To many technically trained minds I realize this idea may sound like giving up the battle before it is won. Didn't I say previously that research viewing human judgment and decision behavior through a normative lens resulted in a very pessimistic picture of human abilities? Yes, but we must keep a few things in mind about this research. First, research that compares human intuitions to normative models samples only that very tiny subset of actual decision problems for which we can calculate a universally agreed-upon prescriptive solution. And even then there has often been tremendous disagreement about which normative yardstick should be used to grade human behavior in particular problems. For example, the simple "turn over the correct cards" problem we performed earlier has given rise to argument after argument in the literature concerning what the true "right" answer to this problem really is. The selection of a normative framework for solving a particular decision problem is often a very subjective matter indeed.

Second, the studies that compare human behavior to normative prescriptions are in an important sense never fair. They don't really contrast a person's use of an intuitive method with their use of an analytical method for solving a decision problem. Rather, they compare a person's use of an intuitive method for making a judgment or decision with some world expert's ability to formulate and execute an analytical method for making that judgment or decision. By the term "world expert" of course I mean the researcher who is conducting the study about the failings of human judgment and decision making and hopes to publish a paper on the matter. Not only is this in some sense unfair, more importantly it is not at all the kind of comparison that can usefully guide how we should support decision makers in their everyday work environments. We are not going to be there to analytically formulate and solve decision problems when a patient goes into cardiac arrest or when production lines must be re-balanced to produce a new product. As a result, if we are going to seriously recommend that practitioners use our normative methods, we need to show that these methods, when executed by the practitioners themselves, produce better decisions than if we allowed practitioners to operate upon their expertise and professional judgment.

What do we find when we directly compare a person's use of intuition and that same person's use of an analytical method to perform judgment and decision making? An interesting experiment addressing this problem was done by Ken Hammond and his colleagues at the University of Colorado. Hammond collected a group of professional highway engineers and had them make judgments of the capacity, safety, and aesthetics of various stretches of highway. Before the experiment, Hammond had a separate group of expert engineers recommend and define analytical methods that could be used to make these capacity, safety, and aesthetic judgments. In the experiment, each engineer had to make each judgment both intuitively and analytically. In the intuitive case, each engineer was shown a filmstrip of each highway and simply asked to record his judgments. In the analytical case, each engineer was given the highway data required to implement the analytical method that had been developed by experts, along with a calculator and instructions on the method itself. Perhaps you can anticipate the findings: In making some of these judgments, the engineers' intuitions actually outperformed their analyses. The effect was largest of course in the aesthetic and safety cases. At a very general level this study also shows that the possibility of demonstrating the superiority of intuition over analysis arises only when we compare a given person's intuitive performance with that same person's analytical performance.

Finally, Hammond's experiment (and others like them) have also revealed an interesting finding concerning the types of errors characteristic of analytical and intuitive reasoning. Research has shown that when one uses an intuitive process, errors tend to be very frequent but reasonably small and normatively distributed around the correct solution. In contrast, errors produced by analytical reasoning are more rare but are much more likely to be quite large. Anyone who has had the pleasure of teaching an undergraduate engineering course using calculators has observed this result. Students trained on calculators, with no or little experientially based intuitions about numbers are all too likely to answer test questions with negative probabilities or to write that an 8 meter beam cut into three equal lengths results in three point oh beams each of length 2.66666666666667 meters.

So now let's return to our dilemma:

Claim 1: The judgment and decision making of experienced practitioners can be studied, described, and understood with analytical methods and models.

Claim 2: The judgment and decision making of experienced practitioners cannot be usefully enhanced by trying to make practitioners behave more analytically or rationally.

And to help us find a way out of this apparent dilemma I am going to use the same trick we used in the card-turning problem. What I am going to do is rewrite the same claims but this time using different cover stories. As in the logical reasoning problem, my hope that the rephrasing of the problem might help us tap into some experience that could provide some intuitions on how to proceed.

Consider the following two variants of our original dilemma:

Claim 1a: The behavior of molecules undergoing chemical reactions can be studied, described, and understood with analytical methods and models.

Claim 2a: The behavior of molecules undergoing chemical reactions cannot be usefully enhanced by trying to make molecules behave more analytically or rationally.

Claim 1b: The behavior of bridges under loads can be studied, described, and understood with analytical methods and models.

Claim 2b: The behavior of bridges under loads cannot be usefully enhanced by trying to make bridges behave more analytically or rationally.

Clearly, alternative version (a) presents no dilemma at all: the way to enhance chemical reactions is to do good chemical engineering. Similarly, version (b) is no dilemma either: the way to enhance the behavior of bridges is through good civil engineering. By analogy the original dilemma is not real but only apparent, and the solution to the problem lies in doing good cognitive engineering.

Decision making, to my knowledge, is the only area of science in which, when model and data disagree, we blame the data and not the model. When we find that people do not behave according ot the prescriptions of our theories, we blame the people and not the theories. This same mindset nearly got Galileo killed.

In closing, I just want to thank the organizers for the invitation to speak to you this morning and you for your attention. After very briefly familiarizing myself with Decision Based Design I realized that since it was outside my area of expertise I wasn't likely to have many good intuitions on the problem, and if I tried to get analytical about it I would run the risk of saying something really stupid. So I decided to give you a brief overview of how people in my field view the problem of supporting decision making in actual work contexts, and I'm happy to leave it to you to draw any connections to your problems, if there are any. Thank you very much.