Issue
Article
Vol.29 No.4, October 1997
Article
Issue
Issue |
Article |
Vol.29 No.4, October 1997 |
Article |
Issue |
The CHI 97 Workshop on Awareness in Collaborative Systems gathered together 15 people interested in the issues of providing awareness information in collaborative systems. Our agenda was aggressive and broad-reaching. We attempted to address all of the outstanding issues in this area. We were successful to a degree. We met, we aired many issues, but we resolved very few.
The goals of the workshop were to develop a research strategy for evaluating the effect of awareness mechanisms on users and organizations, and to begin to develop guidelines for designing appropriate awareness solutions.
We organized the workshop into sessions of small focus groups followed by large group summaries and discussions. During each small group session there were three parallel groups meeting. We chose the focus group topics based on our knowledge of the state of research and the position papers. The discussion topics were: development of guidelines for designing awareness solutions, outlining the dimensions of awareness information, discussion of evaluation methods of awareness mechanisms/solutions, proposal of potential solutions, discussion of theoretical models, and development of a research strategy. The specifics of each discussion group are included in the next session.
In this section we present each of the small group discussions, outlining the directions given to the group and summarizing the discussion.
Directions: For various types of practical situations, propose suggested guidelines for awareness solutions. How should solutions vary according to group size, goals, context, and ethical and psychological considerations? What are the advantages and disadvantages of various solutions?
This group suggested using the Quintilian questions (aka the five "W" words): Who, What, When, Where, and How as proposed by Gutwin and Greenberg [1996] and McDaniel [1996]. The answers to these questions in each individual design situation should provide a comprehensive set of requirements for design.
Directions: Develop an overview of awareness issues. How do awareness mechanisms vary and how do contexts vary where awareness mechanisms are used? What are the meaningful cues involved? How do feedback mechanisms and user interfaces vary? How might these relate to the types of tasks that can be accomplished?
This group looked at a number of factors which effect the design of awareness mechanisms. These include people and their roles, the environment, projects, groupings, boundaries, tasks, awareness resources, self-presentation, and audience. The diagram in Figure 1 summarizes these factors.
Figure 1: The Seven Dimensions Of Awareness Information
Each of the seven dimensions is on a continuum. Abstraction refers to the level of abstraction in the display of information. For example, live video is real information, selected frames from video is more abstract, and an in/out indicator is totally abstract.
The temporal dimension refers to the time factor in awareness information. Is information available for the past as well as for the present situation? Is there any prediction of future activities?
A snapshot or a single data point versus a summary over time, space or other attributes is captured in the aggregation dimension. This dimension also covers interpreted versus raw information.
The locus of control of information ranges from the perceiver to the receiver. Either party could have full control over all information or control of some.
Personalization is also about control. Will the system be individually personalized or only on a group wide basis?
Is information provided explicitly as with an office In/Out Board or implicitly via cameras or other sensors. In other words, how much effort or consciousness does the person on the gathering end have to have about making the information about themselves available?
Finally, the perspective of the presenter versus the perspective of the viewer. The presenter may want minimal information made available but the viewer may need more information.
Directions: Discuss ways in which awareness mechanisms can be tested and evaluated. What types of experiments and other studies have been done or are possible? What can be measured? How can we determine if awareness mechanisms are useful, distracting, and so forth?
Evaluation of awareness information systems is a difficult problem that has not been adequately addressed in the literature. This group identified the following five major issues associated with evaluating awareness mechanisms. These were: (1) identifying what is being evaluated, (2) what techniques are valid, (3) the time aspect of evaluation, (4) development of test instruments and databases, and (5) can the evaluation be done in isolation.
Techniques for evaluating awareness mechanisms must be based on what is being evaluated. Providing awareness information involves at least three components: translation of information, presentation of the information, and the information itself. In order to evaluate the mechanism all three of these pieces must be evaluated.
Are traditional techniques appropriate? Are new techniques required?
Longitudinal studies are often necessary, but expensive. How do groups change over time? People are very adaptable
A body of test instruments which can be used in various situations and with a variety of groups is needed to facilitate cross study comparisons.
Awareness is a social construct and so, tools to support it cannot be studied in isolation. Studies must look at groups, and also look at how the tools fit with other tools individuals and groups are using.
The final outcome of this group was a delineation of what we can measure and how we can measure it. We can measure the process a group uses and the outcome of the group.
We can measure these variables in the laboratory and in the field. Consideration for these lab and field studies are the granularity of the data collected, short-term versus longitudinal studies, and frequency of data collection. In addition, the scale, resources, and scope of these studies can be varied.
Directions: Identify specific problem areas which awareness mechanisms can address and propose solutions. What are new awareness mechanisms or implementations that have not yet been tried? Brainstorm novel solutions, identify areas of awareness which haven't been adequately addressed, and explore some radical alternatives.
This group proposed one somewhat radical solution: the CHI98 Electronic Conference Companion. The conference companion would provide:
These would be provided in a portable electronic program, with position sensing, identification, interest markers, annotations (both private and shared), voting facilities, and audio capture.
We identified the following issues:
Of course, there are many other possible solutions for awareness information.
Directions: Consider the prospects for theoretical models of awareness. Are current models adequate? Have areas been neglected? Do alternative metaphors or groupings of the issues provide additional insight? What questions need to be answered by good models for the future? What research remains to be done in order to have satisfactory models?
This group proposed the chapter titles for a book on Modeling Awareness for Collaborative Work. These chapters would be:
In addition, they developed the diagram in Figure 2 as an overview of modeling awareness. This diagram indicates that awareness can be modeled on two levels: macro and micro. The micro level consists of the task, attention, interaction, and embodiment submodels. Some aspects of these models are further elaborated upon elsewhere in this bulletin [Vertegaal, Velichkovsky, and Van der Veer].
Figure 2: Modeling Awareness Information
Figure 3: Iterative Awareness Research and Development Cycle
Directions: Develop a research strategy for studying awareness issues in the future. Define the problems that need to be solved and the types of approaches that would give us additional information. What are the empirical questions that remain unsolved? How can these be addressed and how can the answers be applied? Can we create a collaborative research strategy to study awareness?
In this group we outlined a comprehensive research strategy for awareness in collaborative systems. We adopted a framework developed by Carl Gutwin and Saul Greenberg as illustrated in Figure 3 (taken from Greenberg and Johnson 1997) We then outlined short and long term strategies. Greenberg's framework prescribes a cycle of research and development. We refer to these phases as information, translation, display or presentation, and evaluation
The information phase indicates the need to determine what awareness information people want and/or need. Once the information is identified, the translation phase is entered. In this phase the information is translated to a groupware setting, if possible. Then the question of displaying the information is addressed. Finally, in the evaluation phase, the outcome of the three previous phases is evaluated. If it does not work, is it because the information, the translation, and/or the display is wrong? Or is the evaluation method not valid? The results of the evaluation lead back to another cycle beginning with the information phase.
This cycle can be applied in both short and long term strategies. The short term research agenda is intended to identify holes in the current body of research and to provide the basis for the long term strategy.
The short term research strategy begins with a synthesis of what we know/have learned from current point systems and using the above framework to organize the knowledge. In addition, gathering and summarizing the experiences from current point systems, and extracting the commonalties is necessary.
The long term strategy will be driven by the results of the short term activities. In particular, answers to the following questions should be used for the long term strategy.
What's missing from the current body of research?
Are there new ways to present awareness information?
Are there additional translation capabilities that can be brought to awareness information?
Can awareness be evaluated in isolation?
Independent research projects based on these strategies will lead to a cohesive body of information to guide design, development, and future research.
This workshop brought together a number of researchers in this area for the first time. It is but a beginning to the collaborative efforts necessary to make progress. We plan to have gatherings at future workshops and conferences and possibly a workshop devoted entirely to awareness.
In addition, we have formed an awareness mailing list and a comprehensive web site for those interested in the issues of providing awareness information in collaborative systems. The mailing list can be joined by sending a request to mcdaniel@acm.org. The web site is located at http://www.crew.umich.edu/~brinck/cscw/awareness.html. For additions or corrections to the web site, please contact Tom Brinck at brinck@umich.edu.
We would like to acknowledge the efforts of the Workshop Co-chairs, Rhona Charron and Mark Leonard, without whom this workshop could not have happened.
Greenberg, Saul. [1996] Workspace Awareness for Groupware. CHI 96 Conference Companion. ACM.
McDaniel, Susan E. [1996] Providing Awareness Information to Support Transitions in Remote Computer-Mediated Collaboration CHI 96 Conference Companion. ACM.
Vertegaal, R., Velichkovsky, B. and Van der Veer, G.C. "Catching the Eye: Management of Joint Attention in Cooperative Work". SIGCHI Bulletin (29) 4, ACM 1997.
Bérard, François and Coutaz, Joëlle. (1997) Awareness As an Extended Notion of Observability.
http://outlet.imag.fr/fberard/CHI97Awarness/
Dringus, Laurie A. (1997) Position Paper for Awareness in Collaborative Systems.
http://scis.nova.edu/nova/hci/position.doc
Dourish, Paul [1997] Extending Awareness Beyond Synchronous Collaboration http://www.best.com/~jpd/chi97-awareness.html
England, Dave. (1997) Awareness in Virtual Organisations.
http://www.csc.liv.ac.uk/~england/awareness.html
Greenberg, Saul and Johnson, Brad. (1997) Studying Awareness in Contact Facilitation.
http://www.cpsc.ucalgary.ca/projects/grouplab/papers/97-StudyingAwareness.CHIWorkshop/saul.html
Gutwin, Carl and Greenberg, Saul. (1997) Workspace Awareness.
http://www.cpsc.ucalgary.ca/projects/grouplab/papers/97-StudyingAwareness.CHIWorkshop/gutwin.html
Koenemann, Jurgen. (1997) Awareness Issues in a Virtual School Environment. http://simon.cs.vt.edu/~koeneman/AwareWkshpCHI97.html
Lee, Alison. (1997) Awareness Research Based on NYNEX Portholes.
http://www.cs.utoronto.ca/~alee/chi97_awareness.html
Mynatt, Elizabeth. (1997) Two Cases for Awareness: As Thread for Long-Term Collaboration and As Fodder for Forming Tacit Knowledge.
http://www.parc.xerox.com/cdi/members/mynatt/position.html
Proctor, Rob and McKinlay, Andy. (1997) Awareness as a Factor in Relationships at Work.
http://www.dcs.ed.ac.uk/home/rnp/position.html
Rodden, Tom, Palfreyman, Kevin, and Mariani, John. (1997) AWARENESS: The Making Public of Work.
http://www.comp.lancs.ac.uk/computing/users/tam/chi97.html
Scholtz, Jean. (1997) Position paper for CHI97 workshop: Awareness in Collaborative Systems.
http://www.crew.umich.edu/~brinck/cscw/awareness/scholtz-submission.html
Smythe, Phil. (1997) Awareness in Collaborative Systems.
http://www.crew.umich.edu/~brinck/cscw/awareness/smythe-submission.html
Vertegaal, Roel. (1997) Towards Conveying Joint Attention States.
http://reddwarf.wmw.utwente.nl/pub/www/persons/vertegaal/publications/chi97/workshop.html
Watts, Leon. Concepts of awareness in human activity.
http://www.york.ac.uk/~law4/ncscw/WattsAware.html
François Bérard and Joëlle Coutaz.
Laurie A. Dringus
Paul Dourish
Dave England
Saul Greenberg and Brad Johnson
Carl Gutwin and Saul Greenberg
Jurgen Koenemann
Alison Lee
Elizabeth Mynatt
Rob Proctor and Andy McKinlay
Tom Rodden, Kevin Palfreyman, and John Mariani
Jean Scholtz
Phil Smythe
Roel Vertegaal
Leon Watts
Susan E. McDaniel is a Ph.D. candidate in computer science and engineering at the University of Michigan, where she is actively researching people awareness issues in remote computer-mediated collaborative situations. She has a master's degree in computer science from Colorado State University and has worked at US West Advanced Technology Laboratory in Boulder, CO.
Tom Brinck has a master's degree in computer science from Stanford University and a master's in cognitive psychology from the University of Michigan, where he's currently a Ph.D. candidate in psychology doing research in communication with drawings. Tom has done education research at Apple Computer and CSCW research at Bellcore, where he was involved in the development of the Rendezvous groupware toolkit.
Susan E. McDaniel or Tom Brinck
701 Tappan Street
CREW
University of Michigan
Ann Arbor, MI 48109-1234, USA
email: mcdaniel@acm.org, brinck@umich.edu
Issue |
Article |
Vol.29 No.4, October 1997 |
Article |
Issue |