Design of effective support tools for human interaction with a complex system requires models of both the human and the system. Models of the human are necessary in order to understand and evaluate the behavioral demands imposed by the system. Models of the system are necessary in order to comprehend the complexities of the work domain. The potential problem with two separate and independent models is that there is no guarantee that these models will "interface". One modeling framework's language and structure may be virtually untranslatable in the terms of the other model, and moreover, in union, not very illuminating in understanding mutual implications for design. In the research depicted in this paper, we propose to wed two models; one, a process model of human cognition, and the other, a model based on a particular descriptive framework of system structure, the Abstraction Hierarchy (Rasmussen 1986).

Research in cognitive engineering and human-machine systems has focused on providing tools, particularly automated tools, to support human activities in real world work environments. In taking a systems approach to this problem, we must consider both the human and the task environment, and in cases where humans interact with the world through technology, the representations of the world and possibilities for action provided by the interface as well (e.g. Woods & Roth 1988).

This research addresses a portion of this problem, namely, integrating information about system users and the task environment into an aiding tool. In particular, this research describes a way to integrate models of both the user and the environment in a fault diagnosis situation, with the goal of improving human performance.

Design of effective support tools for human-system interaction requires separate models of both the system and the human. Models of the system are important to understand and evaluate critical system features and overall systems structure, and the constraints these elements place upon an operator's interaction with that system. Models of the human are important to understand and evaluate the demands imposed by the system upon the cognitive processes of the operator. In both cases, the modeling task is useful, not just because the final models can serve as grounds for empirical testing about the theories on which the models were based, but also because the modeling process itself encourages formalization of prior theories about human-machine interaction and may lead to better understanding of the structure of the human-machine system in general (Rouse 1980). Particularly when problem domains are large and complex, formalization of the domain in the terms of a specific modeling methodology can serve to guide the analysis in a direction most likely to produce successful support systems.

1.1. Problem Description

In this project, the problem domain in question is fault diagnosis in complex human-machine systems. For an intelligent agent to diagnose faults in such a system, the following task conceptualization is employed: Given some indication that something is wrong at a very abstract level (e.g. a high level goal such as demand is not being satisfied), the fault diagnosis task embodies not only the identification of a faulty component at a low level of abstraction (e.g. a "physical" component of a system), but also the concurrent generation of a sequence of operators to describe the process by which that faulty component is identified. Note that this conception of the task does not address problems of fault detection or recovery.

The research program outlined below investigates the potential union of two independent and previously unrelated modeling frameworks in developing both normative and descriptive models of fault diagnosis in a reasonably complex system. The model of the system is based on the Abstraction Hierarchy (AH), a theoretical framework of knowledge representation which has been proposed as a psychologically relevant description for complex systems (Rasmussen, 1985). The model of the human is based on Soar, a unified theory of cognition implemented in a production system architecture which has been suggested as a useful tool for modeling general problem-solving behavior of intelligent agents (Newell 1990).

The potential problem with two separate and independent models is that there is no guarantee that these models will "interface". One modeling framework's language and structure may be virtually untranslatable in the terms of the other model, and moreover, in union, not very illuminating in understanding mutual implications for design. However, since there is evidence that the AH has psychological relevance as a domain representation, it is not unexpected that a compatible process model of cognition can be found. Soar provides a potentially clean mapping to the AH, in that 1) the vocabularies used to describe the two theories overlap to a large degree despite their independence, and 2) both theories are explicitly goal-oriented, the AH through its means-end links and Soar by the very definition of its architecture. Given the similarities in their respective languages and structures, further investigation into their compatibility seemed appropriate.

The remainder of this paper is organized as follows. First, a brief overview of the modeling strategy will be outlined. Second, a historical overview of the three phases of our research program will be provided, illustrating the progress achieved thus far and the role of this paper in the larger program. Third, the research context, a thermal-hydraulic process simulation called DURESS (Vicente 1991) will be briefly summarized. Fourth, the system model will be discussed, detailing its instantiation in the domain of DURESS. Fifth, the user model will be described and synthesized with the system representation to form a holistic modeling framework and the benefits of such a representation scheme will be illustrated. Finally, we discuss future improvements to the model and the progress of the overall research program.

2.1. Overview

This project employs a separate system model and user model. The user model includes two components: a search mechanism and a knowledge base. The approach described in this paper is similar to an overlay technique. In the overlay architecture a model of student knowledge is compared to a model of expert knowledge (Wenger 1987). Our approach differs from the standard overlay technique largely because we replace the model of expert knowledge with a veridical system model.

2.2. Research Program

The abstraction hierarchy, a widely cited representation framework in the cognitive engineering community, has been used to create a formal model of a thermal hydraulic microworld (Bisantz & Vicente 1994). This system representation was accompanied by a relatively unsophisticated reasoning mechanism, which provided a normative trajectory through the AH in a fault diagnosis task. Since this system was intended as a tool to demonstrate the applicability of the AH system representation in such a task, there was little or no intent to implement cognitive plausibility in the problem-solving mechanism, beyond a too-generic appeal to search heuristics and problem spaces. A more sophisticated modeling tool was necessary in order to provide an effective model of the human.

The first phase of the research program was intended to address this. Using the powerful theoretical basis of the Soar architecture, we attempted to bring psychological plausibility into the overall human-machine system representation by instantiating a normative problem solver which accurately diagnosed faults in a complex system. An important benefit of this phase was that we now had a representation of a true problem-solving agent as opposed to a trivial computer algorithm. There were still a number of shortcomings in the final results, however; most importantly, the implementation of the problem solver was still solely normative, and there was no outlet for describing the operator's actual process of performing the fault diagnosis task. In order to verify the utility and evaluate the appropriateness of the cognitive model, it is important to be able to provide descriptive models of problem solving as well, which will then allow testing of the model against empirical data.

This paper outlines the second and current phase of the research program. The intention is to effectively implement a descriptive model of actual human problem-solving processes in the fault diagnosis task. The control architecture and knowledge representation allow us to more critically evaluate the degree of cognitive plausibility in our modeling framework. In addition, the explicit separation of system knowledge and the problem solver's knowledge, which was not addressed previously, also lends a greater degree of plausibility to the model. The representation itself, and the reasoning mechanism that accompanies it, can now account for errors and inefficiencies in the problem-solving process. Specific details regarding potential improvements to the model will be described below.

While the development of this process model represents a good start in obtaining a descriptive model of fault diagnosis in complex systems, it will be beneficial to return to an implementation of the cognitive model in an architecture that has some theoretical cognitive plausibility, such as Soar. The theoretical basis behind a more general problem-solving architecture can lend credence to the generalizability of our model across fault diagnosis tasks and domain representations. In addition, future phases of the research will concentrate on modeling existing verbal protocols of the fault diagnosis task.

2.3. Problem Domain: DURESS

The present research was conducted within the context of DURESS (DUal REservoir System Simulation), a dynamic thermal-hydraulic process simulation (Vicente 1991). The system consists of two redundant feedwater streams, each consisting of a pump and three valves, which can be configured to supply water to two reservoirs. The system goals are to keep each of the reservoirs at a prescribed temperature, and to maintain enough water in each reservoir to satisfy each of the current externally determined demand flow rates.

2.4. System Model

2.4.1. Abstraction Hierarchy. The abstraction hierarchy (AH), a representation framework that has been proposed for describing complex work environments (Rasmussen 1985), is a multilevel representation format, with each level describing the complete system in terms of a different set of attributes or "language". Higher levels of abstraction represent the system in terms of purpose and functions, whereas lower levels represent the system in terms of physical implementation. In effect, each level of the AH is a different model of the same system. Levels are connected by links which reflect means-ends relations between objects. The AH is intended to represent the set of goal-relevant constraints governing the operation of the controlled system, and as a result, it does not contain representations of any specific events or operator tasks. For a more detailed description of the AH, the reader is referred to Rasmussen (1985) and Vicente & Rasmussen (1990).

2.4.2. Instantiation in DURESS domain. The AH was used as a basis for developing a formal representation of DURESS. DURESS was described in terms of objects which comprise the system at each level of abstraction, along with the means-end links connecting those objects across levels. Two types of means-end links are included in the formalization, reflecting either the means by which a function or goal can be accomplished (a link to the level below), or the goals or functions an object can affect (a link to the level above). This allows traversal of the links in either a top-down (from ends to means) or bottom-up (from means to ends) direction, respectively.

There are two useful additions to the AH included in the knowledge representation. The first is a part-whole decomposition dimension which is orthogonal to the means-end dimension (Rasmussen, 1985). This allows reasoning through different levels of system decomposition in addition to different levels of abstraction. For example, a feedwater stream can be decomposed into a pump and three valves. As with the means-end hierarchy, both top-down (from whole to part) and bottom-up (from part to whole) links are included. Second, topological connections between system components are also included which reflect connections between system objects at the same location in the means-end/part-whole space. The interpretation of the topological connections depends on the level of abstraction. For example, at lower levels of abstraction, topological links may reflect spatial relations or physical connections between objects; at higher levels, they may be categorized by the direction of causal propagation.

While these two additional dimensions are by definition independent from a means-ends representation of a given system, the term "abstraction hierarchy" is often used to refer to the entire means-ends/part-whole/topological space. We will adopt this convention below. The following paragraphs describe the instantiation of this three-dimensional framework for system representation in the context of DURESS.

2.4.3. Implementation. For the part-whole dimension, three levels of resolution were selected: component, subsystem, and system. Objects at the component level of decomposition are the pumps, valves, heaters, and reservoirs. At the next level, these components are aggregated into meaningful subsystems; thus, the objects are now feedwater streams, reservoir subsystems, and heater subsystems. Finally, at the system level, the entire system is described as a single whole.

Similarly, the means-ends dimension, orthogonal to the part-whole dimension, was resolved into five distinct levels of description, described below: functional purpose, abstract function, generalized function, physical function, and physical form.

Together, these two dimensions form a three-by-five grid of possible system representations, and each cell in this grid is a distinct descriptive model of the exact same system. However, for a given physical system, it is not necessarily appropriate to consider system descriptions at all fifteen cells in this grid. For example, results from several experiments and field studies have shown that in practice, there is a coupling between levels of abstraction and level of decomposition (Vicente 1991). At higher levels of abstraction, operators tend to think of the system at a coarse level of decomposition, whereas at lower levels of abstraction more fine-grained levels of decomposition are more natural. For instance, it is more appropriate to describe overall goals at the level of the entire system, while the location and appearance of the system objects are more naturally described at the level of individual components.

Analysis of DURESS has revealed that six of these fifteen cells provide an adequate system representation (Bisantz & Vicente 1993). These are described below, organized by their level in the means-ends hierarchy. The topological links within each cell are also described.

Functional purpose. Objects at this level of abstraction correspond to overall system goals, and therefore are appropriately described at the system level of the part-whole decomposition.

Abstract function. This level can be described in terms of the conservation of mass and energy for each reservoir subsystem. In addition to shifting downward in abstraction from the functional purpose level, this corresponds to a decomposition from the system to the subsystem level. Topological links at this level indicate the flow of mass and energy through the subsystems.

Generalized function. Flows and storage of heat and water are described at this level of abstraction. For both the subsystem and component cells, the topological links indicate the direction of flow of water and heat.

Physical function. The states of system components are described at this level of abstraction. Because only individual components have measurable states in this system, the descriptions are at the component level of decomposition. Topological links at this level indicate physical connections between components.

Physical form. At this level, the appearance, condition, and location of each component is described. Topological links, not included in this system model (see Bisantz & Vicente 1994 for discussion), reflect spatial relationships between components.

2.4.4. Sample diagnosis task. To get a feel for how fault diagnosis might occur using an AH representation of DURESS, a somewhat high level description of a sample (albeit normative) fault diagnosis episode may be helpful. Here, "nodes" correspond to objects in a particular cell of the AH system representation, and "links" along three dimensions connect them to related nodes in other cells.

Figure 1: Partial User & System Model (lines in User Model show sample search path)

Initially, a cognitive agent (CA) may get an indication, at the level of Functional Purpose, that Water Demand 1 is not being satisfied. The goal of the fault diagnosis task, as defined here, is to identify a component at the Physical Form level which is faulty. The CA, however, does not know which component is faulty, and, instead of trying to randomly evaluate the many components at the Physical Form level directly, will try to reason through the AH by following links through faulty (or, perhaps, non-faulty) nodes in order to localize the faulty component.

The CA knows that Water Demand 1 is linked to Mass Inventory 1, Mass Source 1, and Mass Sink 1, through means-ends links to the Abstract Function level. To see if the fault has propagated up to it, the CA decides to check Mass Inventory 1. The CA finds that Mass Inventory 1 is not faulty. The CA then checks Mass Source 1 and finds that it is faulty. Since the CA has found a fault, the next step is to try to find something connected to Mass Source 1 which is also faulty. This process of following means-ends, part-whole, or topological links to identify candidate faulty objects continues until the CA follows a means-ends link to the Physical Form level, arriving at a node corresponding to the physical object Reservoir 1 Object, and has completed the diagnosis task.

Given this system representation, the fault diagnosis process is to find nodes connected to the node given as faulty in the input state of the problem (e.g. a faulty node at the Functional Purpose level), find nodes connected to those nodes, and so on, until a faulty node at the Physical Form level is found. Note that there can be multiple types of links (means-ends, part-whole, topological) from a specific node, as well as multiple links of one specific type from that node. The CA must be able to choose, from this possibly large set of links, one to eventually follow. In addition, it is likely that the CA does not possess a knowledge base of the exhaustive set of all possible links in any reasonably complex system; our representation scheme accommodates this, as described in the Discussion section.

2.4.5. Implications for User Model. A system model sufficiently representative of the veridical system can serve to guide the development of a corresponding user model. However, there is no guarantee that even a perfect system model will relate in any way to the user's representation of the system. To establish this relation, there must be reason to believe that the system model is cognitively plausible as well. If this is the case, the two-pronged modeling strategy has the potential to preserve information equivalence across models.

The possible psychological relevance of the AH as a system representation is twofold: the structure of its knowledge representation is psychologically plausible, and it can facilitate effective fault diagnosis. The first claim is partly based on empirical research from a number of diverse domains showing that problem-solving protocols can be mapped onto an AH representation (see Rasmussen 1986 and Vicente & Rasmussen 1992 for reviews). The AH can also facilitate cognition by allowing resource-bounded agents, as people are, to deal with systems that would be unmanageable if they had to observe the whole system in full detail all at once. Using an AH representation of the domain, operators can cope with complex systems by shifting their representation when necessary from a low level of physical details to a higher-level abstraction of system goals. For example, an operator may be more easily able to identify and describe a system fault at a higher, less detailed level of abstraction, and then use the means-ends relationships provided by the AH to constrain the search down through the hierarchy to the faulty components.

This search constraining property is dependent on the goal-oriented nature of the AH and is the second reason for the psychological relevance of the AH. The AH is explicitly goal-oriented since the various levels in the hierarchy are linked by a means-end relation. Thus, search can be constrained by initiating the problem-solving process at a high level of abstraction, deciding which part of the system is relevant to current goals, and then concentrate on the sub-tree of the hierarchy which is connected to the subsystem of interest. This "zooming-in" pattern is illustrated by Rasmussen (1985, 1986) in an electronic troubleshooting domain. The important point to note is that this is an efficient form of search (Korf 1987) since it allows one to ignore parts of the system that are not pertinent to the function of current interest. Thus, an AH representation allows people to engage in goal-directed problem solving in a computationally economic manner.

2.5. User Model

After an appropriate system representation was identified, the next step was to develop a user model, including both a knowledge base and an accompanying searching mechanism, which was descriptive rather than normative. One benefit of an AH-based system representation is that the problem space can be considered completely decomposable. This is not due to the characteristics of any problem-solving agent that might reason on the system, but due to the structure of the problem environment itself. The AH provides any problem-solving agent with a complete and thorough decomposition of the system in question, trivializing the amount of problem decomposition that the agent itself is required to do in order to find a valid solution.

The means-ends structure of the system model suggests that an appropriate problem-solving mechanism in this problem domain should be based on means-ends analysis. In this manner, we expanded upon relatively simple representational constructs for modeling purposes, although there are some significant differences between our solution and a typical means-ends analysis algorithm. These will be discussed in more detail below. The search mechanism is implemented in Lisp, in an architecture whose algorithmic decomposition resembles that of Soar. We include goals, operators, states, and selection-spaces in our system, as well as an elementary look-ahead feature. The following sections assume familiarity with these generic constructs (see Newell 1990 for reviews).

2.5.1. Knowledge Base. As described above, the user model includes a knowledge base which reflects the user's knowledge about the system. The AH system representation of DURESS is implemented as a semantic network. Each node in the AH is considered as a frame, and the slots in that frame refer to the various properties of the node. For example, there is a slot called has-parts-link whose value is a list of all the nodes that are connected via a part-whole decomposition. An operator difference table contains the list of possible operators which a cognitive agent can use while traversing the AH system representation during a fault diagnosis task. For example, there is an operator called follow-topological-link which takes an operator to the next node connected by this type of link. The operators are represented in a general format with variables which take values such as specific nodes, types of links, etc. This allows a small number of represented operators and flexibility in reordering the operators.

Two key conceptual issues motivated the design of the knowledge base: maintaining the independence of knowledge inherent to the system and an agent's knowledge of the system, and ensuring the explicit separation of the system representation and the accompanying searching mechanism.

It is important to note that there are two distinct system representations implemented in our model: the system model described above, which reflects the AH structure of the system itself, and one included in the user model, which reflects a cognitive agent's knowledge of that system structure. Cognitive models are often created with a single representation encompassing both user knowledge and system knowledge. However, we cannot assume that a CA carrying out the fault diagnosis task has perfect knowledge of the DURESS system.

Specifically, we assume that the CA does not initially know the states of all the nodes (faulty or not) in DURESS, with the exception of the faulty node given as an input to the fault diagnosis task. However, we assume that the state of a node will be available to the CA when the CA can check the node to see if it is faulty. This checking capability is analogous to physically perceiving the fault in a component, whether directly, or through the mediation of a computer display.

Also, we assume that the CA may not even have knowledge of the existence of all the nodes and links between them in the AH representation of DURESS. Regardless of the knowledge of links, nodes, and states of nodes that the CA has available, these links, nodes, and states all exist in the actual system, and are sometimes available to the CA. For this reason, it is essential to separate the representation of the system itself from the representation of the agent's knowledge of the system.

In fact, the CA's knowledge about the system is not necessarily a subset of the knowledge in the system, but can contain information about components of the system which is not represented in the system itself. For example, once the CA has "visited" a node, that is, checked a particular node to determine if it is faulty or not, that node gets marked as visited in the CA's knowledge base, but not in the system knowledge.

To move towards the development of descriptive models, it is imperative to separate the knowledge representation from the searching mechanism, for at least the following three reasons. First, changes in the knowledge base alone can reflect varying search strategies, giving more power to model actual subjects while utilizing the same generic searching heuristics. Second, the ordering of operators in the search mechanism can be changed to represent individual choices, because many operators can potentially apply at a single choice point. Third, links and/or nodes can be deleted purely from the knowledge base to represent missing knowledge on the part of the cognitive agent without affecting the integrity of the user model.

2.5.2. Searching Mechanism. The second component of the user model was a problem solving mechanism, which was implemented via a means-ends analysis approach with important additions. For instance, an operator difference table is often ordered so that more important differences get resolved first, but in our system, there is sometimes a larger set of operators that can potentially resolve a single difference. In other words, there is not an injective mapping from a particular difference to a unique operator. Thus, the operator selection rules must account for this.

Often in the means-ends analysis approach, any operator will be able to fire under the assumption that its preconditions will eventually be satisfied by applying other sub-operators. In our algorithm, this is not the case. It is possible that sub-operators can fire which will lead to nodes that do not resolve a difference at a higher level, and thus the set of preconditions for the original operator cannot be satisfied. The mechanism would then move on to check the next operator in the feasible set. This brings our system closer to a production system style of architecture, in that the entire feasible set of possible operators can be explored if necessary. In a more limited means-ends analysis approach, the limitation of one-to-one operators can be prohibitive, largely with respect to the requirement that the CA must have knowledge of a huge number of operators.

In one sense, this mechanism can be considered a "planning" mechanism because it generates a sequence of operators that can transform a given start state (i.e. fault identified at the functional purpose level) to a given goal state (i.e. fault identified at the physical form level); this sequence would correspond to the intuitive definition of "plan". However, in this problem domain, the goal state may not be completely specified. In particular, the CA does not have a priori knowledge of the faulty node at the physical form level. If a sequence of operators that led to a particular faulty node were stored as a specific plan, that would indicate that every time a CA knew about a fault at the most abstract level, she would immediately associate it with a specific fault at the physical form level, in accordance with this stored plan. However, this is not an effective method of fault diagnosis, as many different faults at the physical form level could potentially propagate up the AH to a single node at the functional purpose level.

2.6. Combined Models: Output

The combined program, synchronizing the system model and the two-part user model, produces a complete trajectory through the abstraction hierarchy, from the faulty node at any relatively abstract level to the precise fault existing at the physical form level. The particular nodes which were checked for faults are collected and stored, along with the precise sequence of operators which outline the solution path. Not all nodes along this path may be faulty, so the path is not necessarily optimal, but a valid solution will be obtained. Simply put, since the fault exists in the system itself, and propagates throughout every cell in the AH representation, if the CA is able to find the fault through any trajectory, it has succeeded. Of course, a normative trajectory can also be generated through manipulation of the algorithm, but this is not nearly as interesting as the value of this framework for descriptive modeling, to be described below.

The advantages of this representation scheme chiefly concern its facilitation of modeling cognition. First, this representational scheme allows us to build a model of the CA that fails at the fault diagnosis task the way a human might. For example, suppose a human operator does not have enough experience to know the physical layout of the plant. She may know that there are pumps and heaters, but she may not know that a specific pump sits right next to a specific heater. This would correspond to an absence of knowledge of the follow-topological-link operator; in a case where a topological link must be followed in order to find the fault, this person would fail.

Another benefit is that in a situation similar to the above, the representation scheme allows our user model to recover from a lack of knowledge in the way a human might. For example, even if a person does not have the knowledge to directly find a faulty component from, say, a particular node one level above the faulty component because she is not aware of their means-end relationship, the fault can still be potentially identified. In this case, a person would have to circumvent this missing knowledge by reasoning with any other available knowledge. In order to do this, she could initiate a search among non-faulty nodes until she reached one that was connected to another faulty node; this may be the faulty component in question or may potentially lead to it. This corresponds in our model to a link between nodes which is missing from the CA's knowledge base, and is circumvented by the application of operators which move to non-faulty nodes until a faulty node can be reached.

This representation scheme also facilitates the learning of links between faulty nodes that are not currently represented in the CA's knowledge base, a feature similar to Soar's chunking mechanism. For instance, in the above example, say that the CA reaches a faulty node for which she has no knowledge of a specific link connecting it to another faulty node. A search through non-faulty nodes would then be instantiated. If this search leads to a faulty node, then a link between the initial faulty node and the newly found faulty node can be learned, added to the CA's knowledge base, and used in later fault finding episodes. The CA would know that the two nodes are somehow connected, but would not know the specific semantic classification (means-end, part-whole, topological) of that link. The link would be stored as "unclassified", but that would by no means prohibit its use in the future.

Another attractive feature of this representation scheme is that it can potentially account for alternative, perhaps inefficient or faulty, diagnosis strategies. One way this can be achieved is through varying the ordering of operators. For instance, we know that the most efficient fault diagnosis tasks follow links which connect to faulty nodes. If we manipulate the ordering in the operator difference table so that topological links to a non-faulty node (for instance) apply before those that go to faulty nodes, we would expect a relatively inefficient trajectory, although the solution may still be achieved. Similarly, since we know that the most efficient paths follow means-ends links first, inefficient strategies might be simulated by manipulating the order of operators by link type.

While there are many benefits to this coupled representation scheme, there are also several shortcomings which need to be addressed in future revisions. Currently there is no mechanism for taking into account multiple faults in this model; that is, if more than one fault is present in the system at any point in time, it is difficult to find the set of all possible faults. The assumption is that the CA is attempting to diagnose a single fault which has propagated throughout the AH representation of the system. Clearly, this problem could be solved through a simple depth-first search across the set of all links, but the more interesting question would be to tailor our goal-based architecture to develop a process model for multiple fault diagnosis.

In addition, there are some conceptual problems with the assumptions of the current model which become apparent as we shift toward descriptive models of human operators. First, the algorithm described above assumes a consistent ordering of the operator difference table throughout the course of a task; however, the assumption that humans carry around a constant ordering of operators within or across fault diagnosis episodes is suspect at best. Similarly, there is an implicit ordering of the importance of individual nodes in the CA's knowledge representation that remains constant. For example, given a particular node, if many nodes are connected to it via one specific type of link, these nodes will be checked for faults in an entirely arbitrary order. In addition, it is not necessarily the case that this ordering of nodes will remain constant across problem-solving episodes, or, for that matter, within problem-solving episodes.

However, an implicit assumption related to this, which we do feel is justified, is that a CA will always select what she feels is the most important node. Although this may seem intuitive, it is an important assumption included in our model -- that much of the variance of the fault diagnosis process for a human operator can be accounted for by the ordering of the operators in the operator difference table, and by the ordering of the nodes, as discussed above.

Additionally, it is important to note that we have abstracted away one of the most important features of the DURESS simulation, namely, its dynamic nature. DURESS is implemented as a computer interface to a hypothetical thermal-hydraulic process and an operator is able to dynamically track the state changes throughout the fault diagnosis process. Often new information can be gleaned by witnessing trends in the perceptual data which may alter the diagnosis strategy, and perhaps this would even make solutions immediately apparent to a vigilant operator.

Finally, in many cases, the human has access to the system environment through a mediating computer interface. For a complete description of the human-environment system, it is necessary to build an independent interface model which can relate perceptually available cues about system function to the operator inferences which motivate problem solving strategies. An interface model may permit an abstract evaluation of an interface. For example, the way the user model interacts with the interface model may reveal inefficient search paths induced by particular interface designs.

One potential application of this modeling scheme involves the design of a training or tutoring system for the fault diagnosis task. A model which can simulate a CA diagnosing faults can assist a tutoring system by teaching declarative knowledge of the system as well as illustrating normative techniques for fault diagnosis in the system. For example, a student might demonstrate her knowledge of the system by carrying out a fault diagnosis episode. If the student takes an odd path during the course of problem solving, the tutoring system might suggest a piece of knowledge about the system, such as a node that is linked to a node that the student has already visited, which might influence the student to take the correct or better path, and, in that process, learn the new piece of knowledge.

In the long run, we hope to use this modeling framework to develop a model which will be able to exercise real-time control over a simulation of an actual complex system, utilizing the theoretical constructs as instantiated in our models to determine the proper sequence of control actions. To implement this will require a very robust cognitive model and an equally representative system model. The differences in strategies of the model and of a human operator should at this point be directly comparable through analysis of actions taken on the system interface, as well as indirectly comparable through analysis of verbal protocols and model output.

To conclude, the work described above is an intermediate phase in a research program whose eventual goal is to couple process models of fault diagnosis based on Soar with system representations based on the AH. This research tactic has a two-pronged approach: first, the psychological plausibility of the AH is examined through the lens of the Soar paradigm; second, the utility of Soar models in describing real-world fault diagnosis tasks is analyzed through the lens of the AH paradigm. While neither theory is rigorously tested in itself by this methodology, because negative results cannot reject one or the other, the results can provide converging support for a combined theory of human fault diagnosis, as well as illustrate some of the implications for design.

The authors wish to thank Alex Kirlik and anonymous reviewers for their comments on an earlier draft of this paper.

Bisantz, A.M., and Vicente, K.J. 1994. Making the Abstraction Hierarchy Concrete. International Journal of Human-Computer Studies 40:83-117.

Korf, R.E. 1987. Planning as Search: A Quantitative Approach. Artificial Intelligence 33:65-88.

Newell, A. 1990. Unified Theories of Cognition. Cambridge, MA: Harvard University Press.

Rasmussen, J. 1985. The Role of Hierarchical Knowledge Representation in Decision Making and System Management. IEEE Transactions on Systems, Man, and Cybernetics, SMC-15:234-243.

Rasmussen, J. 1986. Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering. New York, NY: North-Holland.

Rouse, W.B. 1980. System Engineering Models of Human-Machine Interaction. New York, NY: North-Holland.

Vicente, K.J. 1991. Supporting Knowledge-Based Behavior Through Ecological Interface Design. Ph.D. diss., Department of Mechanical and Industrial Engineering, University of Illinois at Urbana-Champaign.

Vicente, K.J., and Rasmussen, J. 1992. Ecological Interface Design: Theoretical Foundations. IEEE Transactions on Systems, Man, and Cybernetics, 22.

Vicente, K.J., and Rasmussen, J. (1990) The Ecology of Human-Machine Systems II: Mediating "Direct Perception" in Complex Work Domains. Ecological Psychology 2:207-250.

Wenger, E. (1987) Artifical Intelligence and Tutoring Systems. Los Altos, CA: Morgan Kaufmann.

Woods, D. and Roth, E. (1988) Cognitive Systems Engineering. In M. Helander (ed.) Handbook of Human-Computer Interaction. New York, NY: North Holland.