No earlier issue with same topic
Previous article
SIGCHI Bulletin
Vol.29 No.4, October 1997
Next article
No later issue with same topic

Object Models in User Interface Design: A CHI 97 Workshop

Mark van Harmelen, John Artim, Keith Butler, Austin Henderson, Dave Roberts, Mary Beth Rosson, Jean-Claude Tarby, Stephanie Wilson


This workshop was held during April 23-24 as part of CHI 97, to discuss the use of object models in user interface design, and particularly to:

The position papers [] presented at the workshop revealed a spread of practice and theory. There were four broad areas where user interface design activity involved object modeling:

Over the two days of discussion, there were two striking points of agreement and commonality among the participants:

Firstly, there was quite remarkable agreement on task (and process) analysis as being a major source of both objects and operations on objects for subsequent use in user interface design and system design:

The strong connection between processes, tasks and objects is not surprising: (i) User interface design is the place in software design where user tasks and processes are most directly impacted. (ii) The use of objects to describe systems is appealing to user interface designers; objects have been used in user interface design since the development of the first commercially available GUI, the Xerox Star. In combination, these factors make a fusion of object modeling with process and task analysis highly desirable. Furthermore, experiential evidence revealed that there is an eminently exploitable fit between these techniques.

Secondly, we generally incorporated the following process in our design activities:

scenario generation (in the broadest sense, including process and task analysis) -> type-like description (i.e. object modeling) -> further reasoning about scenarios

We spent some time examining the suitability of object modeling for user interface design, spurred on by one of us [Henderson].

The remainder of this summary describes issues and conclusions from the workshop, and a framework that we formulated during the workshop and developed after the workshop, both to act as a common reference for our discussions, and to help us understand and characterize the field.

Design Worlds

In an early phase of workshop activity, we constructed a framework (figure 1) within which to view user interface design activities. In this, user interface designers support users who have particular goals and task-based behavior and operate within some context. We call this framework a design world.

Figure 1: A Design World

In this setting, user interface designers not only design user interfaces, but also design system purpose, functionality, and impact on the users' world. We regard all of these aspects as properly being within the scope of user interface design.

A users' world is a system that is of interest to a population of users and designers for some period of time. A users' world can be described in different terms and at various levels of abstraction.

Referents are entities in the users' world that have meaning to users. Referents may be users themselves, physical objects in the users' world, or events and tasks. Not only may referents already exist in the users' world, but they may also be added as the result of the introduction of computer systems.

Designers create descriptions of the world and of putative systems. Descriptions populate a description space which contains the domain knowledge and design artifacts that enable the process of user interface design. A design method is a way of performing user interface design. A design method includes design techniques like task analysis or modeling. Application of a design technique produces a description expressed in a notation. Valuable techniques and descriptions that can be combined or show promise for combination are discussed below.

Object Models

While there are many kinds of descriptions, some kinds of descriptions are object models which are composed of objects and links between them. Objects may be abstractions and representations of referents, or they may be abstractions which assist in the design of an interactive system. We use the `object' in the conventional sense -- an encapsulated entity that has defined behavior exhibited by a public interface of operations to the world. Operations may be invoked by users or by (other) operations. Each object's behavior is described by a type, and many objects may conform to the same type. We construct models out of objects, and describe the models and objects in type diagrams. In type diagrams, boxes represent types and annotated lines represent associations between the types. This is the conventional way of performing object modeling, although most workers in the field erroneously talk in terms of classes instead of types. More correctly, types define behavior, and a class is an implementation of a type. We should be interested in designing using the purer concept of a type, rather its implementation. The reader who is unfamiliar with object modeling is referred to [Rumbaugh, 1991].

In this summary we use a simple variant of Rumbaugh's Object Modeling Technique (OMT). Subsequently we intend to move to a newer notation, Unified Modeling Language [Booch and Rumbaugh, 1995], for our ongoing work. Our variant of OMT merely substitutes OMT's cardinality markings with more readable textural information. We emphasize the naming of roles, rather than associations. A guide to the notation is shown in figure 2.

Figure 2: Guide to the Notation

Task Models

Other kinds of description are task models. Like object models, task models are representations of referents; in this case, of referents which are tasks in the users' world. A familiar concept within the user interface design community, task models describe the activities in which users engage to achieve some goal. We describe tasks in terms that are meaningful to users' work, rather than in terms of keystroke-level interactions with a specific system. A task model details the users' goals and the strategies adopted to achieve those goals, in terms of the actions that users perform, the objects involved in those actions and the sequencing of activities. Task models can describe (i) the tasks that users perform with existing systems, and (ii) the tasks that are envisioned as being performed with one or more proposed systems.

Task models are typically abstractions of user behavior across many users performing many instances of a task. Designers create the model from information elicited using various data collection and analysis techniques (e.g. interviews, observations, protocol analyses). Traditionally, users have been regarded as rather `passive' information resources during this process, but there is now much interest in users as active and equal participants who can collaborate with designers to describe their current tasks and to envision the form of their future tasks.


A related concept, also common in user interface design, is that of a scenario [Carroll, 1995]. Scenarios are narrative descriptions telling a story of how users engage in some activity in their world; scenarios are expressed in the language of the user and depict what the user does and experiences from the user's perspective. In contrast with task models, scenarios are typically concrete and specific, giving details of one instance of a user task, and are a rich source of contextual information.

Work Models and Process Models

In an attempt to increase the chances that a planned system will have the desired impact, explicit work analysis and modeling has become very popular in the user interface design community. Besides helping ensure that a system will function in its work context, work models can also be used to (i) analyze a current situation to determine what improvements are needed and how they can be gained, and (ii) help predict whether and where a given computer system will support an improved situation.

A variety of modeling techniques are commonly applied. The formality of these techniques can vary from scenarios [Jacobson, et al, 1993], which are very informal and can't support much analysis, to IDEF3 [Meyer, et al, 1992], which is formal enough to support discrete event simulation. Modeling techniques also vary by the type of view that they use to represent activities and goals, e.g.:

During the workshop we concentrated on process modeling. In this summary we note that process modeling is one kind of work modeling.

Models of Design Worlds

In order to understand the use of object models in user interface design, we found it useful to model user interface design worlds (see figure 1 for an example design world).

Figure 3: Model of a Design World

Figure 3 presents a type diagram that represents a simple model of a design world. This model is not complete, and will be refined later in this summary. One important feature is that the model of the design world incorporates an object model as a description of a computer system. This system may or may not have been implemented. Designers produce the object model using information provided by users. Objects in the model represent referents in the users' world. In this design world, the object model is, in user interface design terms, a conceptual model of an application. There may be an implementation of the conceptual model that produces implementation referents in the users' world, and these implementation referents can themselves be modeled. As user interface designers we are, in principle, not interested in the mechanistic aspects of the implementation, but we acknowledge that, within our task of designing support for system users, implementation details do have an effect on user interface design. Thus in practice, we may well be interested in some implementation detail.

Figure 3 is general; the cardinalities in the type diagram allow for different numbers and kinds of descriptions. Some of these may inform, transform to or generate other descriptions:

Informing, transformation and generation are ordered on a continuum that varies from informal to formal use of descriptions.

We view descriptions as being central to the business of user interface design. Later in this summary we discuss the space of descriptions in the context of object modeling and user interface design.

With its incomplete dashed inheritance association from Description, figure 3 hints at descriptions which are not object models, and figure 4 presents a design world which has been elaborated to include a task model. A task model is a representation of how referents are used by users. Each task in the model may itself be a referent that is of import to the user. As mentioned above, the task model may contain both tasks that exist in the users' world and tasks envisaged as being carried out with the new system. As designers, we use the task models to inform both the design of the user interface and the underlying implementation of the new system. A model of the users' tasks will be embodied in the system.

Figure 4: Model of a Second Design World

Process models can also be added to figs 3 and 4; as discussed below, developing a comprehensive model of design worlds and descriptions is one of our ongoing activities.

In summary, task- and process-based descriptions (task and process models) capture behavioral aspects of users' worlds, while object-based or object-oriented descriptions (object models) both capture the structure of the world and allow attachment of object-based semantics to the task- and process-based descriptions. In combination, these descriptions provide powerful ways to record developing designs, to inform design activities, to convey design information to colleagues, to serve as a basis for reasoning about the developing design, to provide user interface design specifications for a later implementation phase, and to provide reference material for later system maintenance and extension activities.

However, caution is necessary; in themselves object, task and process models are not sufficient to convey the richness of the design material in a typical user interface design. To address the need for a project team to collaborate across work, task and computing support design, we defined a more encompassing (but not necessarily minimal, complete or sufficient) set of descriptions of the users' world and of the system being developed:

Issues Concerning the Description Space

During the workshop, part of our time was concerned with issues connected with the description space. These issues are summarized below, categorized according to descriptions in general, and object modeling in particular:

At the beginning of each sub-section, there is a small iconic representation of the description space that relates to that of figure 1.

Particular issues that we identified and that still remain open to discussion are reproduced below. Material that we agreed on and that we feel is non-contentious is simply presented without an associated issue.

A Characterization of Object Models

We were concerned with the kinds of object model we needed for the design of interactive systems, and the kinds of objects within these models.

Generally we distinguished between several kinds of object models, all subtypes of Object Model (see figure 5).

Figure 5: Model of the Useful Object Models

A domain model specifies user domain semantics. A domain model is composed of types which model referents and represent the users' world at a sufficient level of detail to perform user interface design. Users may appear as part of the domain model if they are required to be in the model. The domain model may include implementation referents, be they currently in the users' world, or be they implementation referents which the designer predicts will be introduced into the users' world as a result of the introduction of the planned system. This is a key to being able to model systems that extend the users' world. For example, prior to the introduction of e-mail, it would have been possible to design task models for users of e-mail systems, model e-mail systems with objects for user interface design purposes, and design consequent e-mail system functionality.

A core model specifies that part of the user domain semantics that will be implemented. A core model is a subset of a domain model that contains just those referents that are to appear in the proposed system.

Both domain and core models are composed of objects representing referents. Some referents, presumably for reasons of lack of import to the system and its use, might be excluded from the models. Both models, as discussed above, may include predicted implementation referents that do not exist in the world as yet.

An interaction model augments the core model: An interaction model provides a specification of the interactive facilities to be used by the users when invoking core functionality in the system. Thus an interaction model provides a description of the semantics of (i) the presentation of the core model and (ii) the interaction with the core model: It captures the view structure of the application, and non-presentational aspects of interaction styles and mechanisms. We acknowledged that interaction model objects might be used as a basis for automatic generation of user interfaces during an implementation phase, but did not explore this topic. Types in this model include those that represent transactions, views, selection mechanisms, and windows.

An interactive system model is an aggregation that is composed of both a core model and an interaction model. An interactive system model is useful: When using an implementation a user needs, inter alia, to understand both the semantics of the system and the means of interacting with the system. Thus in the design of a system, the core model must be augmented with an interaction model, and both the models must be considered together. This corresponds to our widespread agreement that while we are interested in designing functionality independent of user interface, we feel that this divide that can not be made in practice.

An implementation model is a reified interactive system model that is to a lesser or greater extent biased towards an implementation. In practice implementation models will be successively reified to a level of detail from whence it is deemed desirable to generate implementation code.

We made a very strong distinction between the interactive system model and the implementation model, being particularly concerned about implementation details "polluting" or biasing any aspect of the interactive system model. However, the extent to which user interface implementation facilities affect the interaction model and even the core model was not discussed. We agreed that as user interface designers we are, in principle, not interested in mechanistic aspects of the implementation, but we acknowledge that implementation details do have an effect on the user interface design of interactive systems.

Object Models in Relation to Other Descriptions

The relationship of object models to other descriptions is not fully explored here, we merely report on our major concerns during the workshop, where we were interested in the relation of three kinds of descriptions to object models:

One great advantage of object modeling is that models have state, which can be described in a variety of different ways. In particular, any process or task execution, or user interaction can be described in terms of state changes over the interactive system model, be this description one which is expressed formally, in mathematical terms, or informally, in natural language.

Task Models and Object Models

Task models and object models are described above; when these are used in conjunction tasks may be described using descriptions of referents, namely, objects in an object model. Tasks may have semantics attached to them in the form of pre- and post-conditions over the object models.

Furthermore there is evidence of task descriptions being a useful precursor to object extraction for the domain model, and this gave rise to a tantalizing issue:

Issue: There was an overriding desire to see the nature of a close binding between task analysis and object extraction and definition. We felt that one way of approaching this would be a search for an understanding of the commonalities and differences in the ways that tasks and objects are related in various participants' methods and tools.

Process Models and Object Models

Virtually all types of work modeling, including process modeling, need to capture responsibilities, activities and ordering of activities that are needed to accomplish specific work goals. Unfortunately, there are mismatches between work models and software models that make it very difficult and unreliable to transform work models into software requirements. The mismatches are: (i) The level of granularity needed in software models is much finer than in work models -- in the latter, humans typically have the advantage of applying higher-level procedural knowledge from familiar work domains. (ii) Software models must somehow integrate both data and processes, but within work models data is generally neither modeled nor represented in sufficient detail for software modeling purposes.

Issue: The difficulties and unreliability of translating between user-centered work models and technology-centered software models undermines the value of both, and general solutions are needed to bridge between the two kinds of models.

User Interface Design Descriptions and Object Models

Certain issues of interaction are not addressed by the interactive system model. For example, presentational layout, or low level interaction. Ways in which the low level interaction can be tied to the object model were presented at the workshop [van Harmelen].

Issue: We are interested in a minimal and, for design purposes, useful set of concrete user interface descriptions that can be used in conjunction with object models of an interactive system in order to specify the user interface.

Characterizing the Description Space

Issues: Here we are interested in the structure of the description space:

Several participants [Artim, van Harmelen, Tarby] brought object models of their method's description spaces to the workshop, these are available via the web site.

Issue: In the long term we are interested in how to structure descriptions to build systems that allow for ontological drift; change in the users' world.

Issue: For the construction of descriptions we are interested in method: Firstly, in design activities; how do we construct individual descriptions? Secondly, for the construction of user interface design methods: How can we effectively use information in one description space to inform the design process in another description space? What are the requirements of traceability and impact analysis, what is the responsiveness to change?

In summary, we are vitally interested in a broad encompassing view of method and a framework that accommodates the diverse methods that were discussed in many of the position papers.

An Encompassing Meta-Model

As an ongoing post-workshop activity, some of us started refining our models of the design world; initially to try to describe the different design methods presented at the workshop. A draft version of this model appears in figure 6. In particular, we need to refine the model in respect of work, process, task models and scenarios.

Figure 6: Draft meta-mode

Part of the model is a meta-model; it specifies a space of different combinations of descriptions and relationships between descriptions, as might be produced as a result of using different design methods. This meta-model is composed of Description, its subtypes, and the associations between these types. We currently refer to this as the description meta-model. There may be other meta-models that describe different aspects of design methods.

While the design methods presented at the workshop can be accommodated within the draft description meta-model, this conformance has been achieved at the cost of some simplification. In the description meta-model, there comes a point where individual design technique differences between methods mean that, in order to achieve conformance, we sometimes used abstraction to hide a level of detail in different design methods. Thus, for example, we do not present the fact that, in one method, use cases in a use case model may have scenarios illustrating a use case. Instead we leave this under-specified, and intend to later re-approach the topic of granularity in the description meta-model.

We also freely acknowledge that the meta-model that we are building is deficient in one major way: We do not know how to construct design methods, let alone meta-models, which deal with the ontological drift in the world, consequently our implemented systems remain static and brittle in a world of change. We stress that this is also a general problem for system design and construction -- one of us [Henderson] is working on approaches to deal with ontological drift.

However, we are strongly pragmatic as to the benefits of meta-modeling:

Concerns about Object Modelling

At the workshop, Henderson questioned the metaphysical foundations of the object-orientation as a basis for modeling the world, and we spent a considerable amount of time discussing this topic, often in the context how to deal with ontological drift. Henderson's post-workshop view is:

"In both mathematics and philosophy there is a rising concern with the very positivist view of the world that is so deeply imbedded in the theory and practice of computer science and computer systems design. While there is no question that many systems are being built using these techniques, I think we should not let a workshop like this pass without at least noting that there is serious thinking in the philosophy of computer science that may well lead to a sea-change in this whole area. It may take some time before that thinking reaches the day-to-day working edge of the practice of systems design, but I think we can see evidence of these changes on the research horizon already.
In 1926, David Hilbert laid out a mathematics for formal systems, based on symbols and types, operations determined by rules, rules application based exclusively on types. Computer science grew out of mathematics, and so its foundational assumptions, its meta-physics, grew out of the meta-mathematics of the time, namely, formal systems. Hilbert's program, because it fit so nicely with the emergent practice of computing as carried out by the mathematicians who started the field, has been adopted wholesale as the theoretical underpinning for computer science ever since. Most conceptual frameworks, for designing computer systems are formal: programming languages, items in dialog boxes, even hardware. And of course, and most clearly, object technology. And this gets passed on to many of the applications that we build: drawing, e-mail, calendar, MUD, workflow, and document management programs.
Object Model User Designer Task Model Implementation Process Model Domain Model Prototype Referent Type Type Referent Users' World User's Model Description Designer's Model Implementation
Refere Implementation Model Interactive
System Model
Core Model Interaction Model

Figure 5: Draft meta-model

Work ? Model
Under this view, the world has sharp edges. Take for example, calendars: they are a structure of events, each event of a specific type, each type giving specifically the fields needed, and the operations that are permitted, the start and end times being exact, the people involved being precisely identified. All very much the Hilbert metaphysics.
My concern is that, as a means of describing (modeling) the world, the Hilbert program -- and even more strongly its embodiment in object orientation -- is not really up to describing human practice. People deal with much richer understandings of the world than the crispness that formal systems permits, indeed encourages, to be expressed.
Again, take calendars. Yes, we may see many crisp events to record. But we also see: meetings which are set for "sometime Thursday afternoon", meetings which someone in this group should cover but we're not sure who, a need to meet with someone for about an hour later in the week, a need to insure that next week there be at least a day of writing time in chunks no smaller than 3 hours, a meeting during which short phone calls can be scheduled provided there are not too many, a meeting that has been moved three times and has taken on a changed purpose -- a fact to be remembered during the meeting. This is a world of contingencies, uncertainties, multiple possibilities, inarticulateness and inarticulability, and change. (For an excellent analysis that challenges our `standard' view of objects and their adequacy in `modeling' the world, see Brain Cantwell Smith's wonderful new book, "On the Origin of Objects".)
This is not a world in which Hilbert does very well. And hence it is not a world that our Hilbert-based computer systems can handle very well. As a result, when modeling the world using tools based in the Hilbert metaphysics (and specifically, using object technology), a key part of the real work of systems analysts and designers is to figure out how to take the richness of the world and crisp it up so it can be embodied in our current computational technology. Indeed, system analysis has a large component of world re-synthesis! The job is to invent a new Hilbertian view of the world (one that therefore can support `computerization' as we now know it) that is close enough to the lived-in world for people living in that world to fit it more or less easily into their activity.
And people making use of the resulting systems have to be constantly matching them into the world. That is often hard work. It is also something that we have become so accustomed to that we tend to assume that this is the way computers and their use must be. We don't even question whether we can do any better.
There is a small but growing feeling that indeed things could be better. That we might be able to get beyond Hilbert. Interestingly, mathematicians now tend to reject Hilbert's view of their practice of mathematics as being far too narrow: `That's not the way anyone I know works.' And even in some sub-fields of computing (e.g., robot vision and navigation, speech understanding and handwriting recognition, graphics design and media authoring, control theory), richer views of the world are successfully underwriting practical systems based on non-Hilbertian, non object-oriented, computer science. Consequently an important issue is:
Issue: Are the objects of object technology adequate for describing (modeling) the world?"


We concluded the workshop with unanimous agreement on two views. We labeled these views "the `minority' view" and "the `majority' view", on the basis of number of people actively promoting them.

The `minority' view is that object modeling is not, on its own, an adequate basis for performing computer system design, even when coupled with task analysis and other traditional means of user interface design. Reasons for this view include (a) an inability of our object-based approaches to deal with ontological drift, and (b) the inability of most current object modeling techniques to deal appropriately with many user interface design requirements, from issues as broad as lack of full support for task and process-based design, to issues as detailed as the inability of object modeling notations to deal with time and timing constraints in user interface design.

Despite these limitations, the `majority' view strongly supported object modeling as a viable technique in the design of interactive systems.

The `majority' view is that object modeling is a useful and helpful technique in the construction of interactive systems, including the design of their user interfaces. This view is based on our experiential evidence that:

Besides articulation of and agreement on these two views, the workshop succeeded in several ways:

Besides these successes, we, of course, had tremendous fun. We may run a follow-up workshop at CHI 98, adopting this summary as our starting point.

Useful References

Position papers submitted to the workshop are available at

Booch, G., Object-oriented Design with Applications, Benjamin/Cummings Publishing, 1991.

Booch, G., and Rumbaugh, J., Unified Method for Object-Oriented Development, Documentation Set 0.8, Rational Software Corporation, 1995.

Butler, K., "Designing Deeper", Proceedings DIS'95, ACM Press, 1995, pp131-142.

J. M. Carroll (Ed.), Scenario-based design: Envisioning Work and Technology in System Development, John Wiley and Sons, 1995.

Diaper, D., Task Analysis for Human-Computer Interaction, Wiley, 1989.

Fischer, G., Redmiles, D., Williams, L., Puhr, G. I., Aoki, A., and Nakakoji, K., "Beyond Object-Oriented Technology: Where Current Approaches Fall Short", Human-Computer Interaction, 10(1), 1995a, pp. 79-119.

Jacobson, I.; Christerson, M.; Jonsson, P.; & Overgaard, G., Object-Oriented Software Engineering: A Use Case Driven Approach, Addison-Wesley, 1993.

Kieras, D., Towards a Practical GOMS Model Methodology for User Interface Design, in Helander (Ed.), The Handbook of Human Computer Interaction. North Holland, 1990, pp137-157.

Kirwan, B. & Ainsworth, L. K., A Guide To Task Analysis, Taylor and Francis, 1992.

Mayer, R., Cullinane, T., deWitte, P., Knappenberger, W., Perakath, B., and Wells, M., The IDEF3 Process Description Capture Method Report, Armstrong Laboratory, 1992.

McCormick, E. J., "Job Analysis: Methods and Applications", AMACOM, 1979.

Roberts, D., Berry, D., Isensee, S., and Mullaly, J., "Developing Software using OVID", IEEE Software, July 1997.

Rosson, M. B., & Carroll, J. M., "Narrowing the gap between specification and implementation in object-oriented development", in [Carroll, 1995], pp247-278.

Rosson, M. B., & Carroll, J. M, "Integrating task and software development in object-oriented applications", in Rosson, M. B., and Nielsen, J., (Eds.), Proceedings of Human Factors in Computing Systems, CHI 95, ACM Press, 1995, pp377-384.

Rumbaugh, J., Blaha, M., Premeriani, W., Eddy, F., & Lorensen, W., Object-oriented Modelling and Design, Prentice-Hall International, 1991.

Smith, B.C., On the Origin of Objects, MIT Press, 1996.

van Harmelen, M., A User Interface Design Methodology, Matsushita Tokyo Research Laboratory Technical Report, August 1992.

van Harmelen, M., "Object Oriented Modelling and Specification for User Interface Design", in Paternó, F., (Ed.) Design, Specification and Verification of Interactive Systems, Eurographics Workshop, June 1994, Springer-Verlag, 1995.

van Harmelen, M., "Melding Object-Modelling and User Interface Design", Object Expert, Nov./Dec. 1996.

Wilson, S., and Johnson, P., Empowering Users in a Task-Based Approach to Design, Proceedings DIS'95 (Symposium on Designing Interactive Systems), ACM Press, 1995.


Mark van Harmelen
Cutting Edge Systems Ltd.
36 Brightwell Walk, Manchester M4 1LZ, UK

+44 161 832-2236

No earlier issue with same topic
Previous article
SIGCHI Bulletin
Vol.29 No.4, October 1997
Next article
No later issue with same topic