Issue
Article
Vol.28 No.4, October 1996
Article
Issue
Issue |
Article |
Vol.28 No.4, October 1996 |
Article |
Issue |
The CHI 96 Workshop on Psychological Issues of Virtual Environment Interfaces provided a common ground for diverse research into the psychology of virtual environments (VEs). There is a small but growing research community investigating those issues, but there is no research forum devoted to them. Publications on the psychology of VE interfaces are scattered across various conferences and journals.
The goals of the workshop were to develop a cogent description of the problem area, identify its importance and primary areas of difficulty, its most salient features, and to work toward identifying a general research agenda on VE interfaces for the HCI community at large.
In this report, we try to satisfy three goals: 1) To present the structure of the workshop, as we thought it supported a productive experience, 2) to report a summary of the workshop, including the topical content and some of the results, and 3) a bibliography of some important books and papers relevant to the workshop topic, contributed by all of the workshop participants.
The workshop consisted of interactive and informal group discussions, instead of individual presentations. This promoted discussions that included all workshop participants and facilitated the useful exchange of ideas and opinions.
All participants had contributions to make on almost every topic. However, with any group larger than ten (there were nineteen participants), it would have been impractical to expect enough time for everyone to participate fully in every discussion. Although a traditional panel gives several people sufficient time to present their thoughts, it limits the prospects for open discussion. Seeking a middle way, we chose an innovative panel discussion format to structure the workshop discussions. Each participant served on the panel of at least two panel discussions.
There were six panel discussions. The topics are described below, in the Discussions section. For each panel, approximately seven participants were designated as panelists. The designations were nominal---their purpose was to provide an organized infrastructure that would support productive discussions. Discussion was not limited to panel members. Each discussion was open to every participant.
The designated panelists started each discussion, but did not make initial presentations as in a traditional panel. Their responsibility to the whole group was to pull the discussion back into productive channels if it drifted, although the workshop organizers bore the final responsibility for moderating the discussion.
Altogether, eight discussions were scheduled. The first six were the topic-oriented panel discussions already mentioned. The other two discussions occupied the afternoon of the workshop's second day. One was reserved for emergent and overlooked topics. The other was devoted to summarizing, bringing together conclusions, and answering the goal questions.
Instead of spending scarce time at the workshop making individual presentations, the organizers asked the participants to carry out a pre-workshop task. The task introduced all of us to each other's work before arriving at the workshop. We opened the first day of the workshop ready to begin productive discussion.
Participants were initially selected on the basis of position papers submitted to the organizers. For the pre-workshop task, each participant read all of the position papers and submitted one or two brief questions about each one.
The organizers edited the questions into a package and distributed it by anonymous ftp to all the participants, three weeks before the workshop. The goals were to make everyone thoroughly familiar with the papers and to give time to reflect on the questions that others had about each paper.
The workshop theme was "What are the components -- of human, task, and system -- that are relevant to usability and how are their roles understood?" The discussions were about ways that virtual environment design could benefit from research on a variety of subjects.
The "Spatial Orientation and Wayfinding" discussion covered spatial orientation and wayfinding in virtual worlds, spatial learning and memory, frames of reference, and perspective.
"Affordances and Tasks" concerned task analysis and affordances for travelling from one virtual location to another, directed or browsing, and for acting in the virtual environment.
A panel discussion that produced considerable light and heat was one devoted to "Social Perception and Expression" and social interaction in multi-user VEs.
The "Multi-Modal Perceptual and Motor Abilities" panel discussion concerned the importance of designing the virtual world to leverage on multi-modal perceptual, psychomotor, and cognitive abilities---not just striving for photo-realism.
"Usefulness and Usability, User Acceptance" ranged over the topics of understanding the dimensions of usefulness and usability for VE interfaces, issues emerging from applied research on virtual environments, and barriers to user acceptance.
The "Evaluation" discussion concerned methods for evaluating VE interfaces, identifying and understanding human-VE interface issues, and experiences with usability testing of whole systems.
In each discussion we considered what we would propose as a future research agenda for the CHI community (and others) in this area.
A great deal of productive discussion took place during the workshop. The following two sections collect some of our consensus about two important basic issues in VE interface design at the present time. The first is about striving for photo-realism in visual scene content. The second concerns the more general topics of usefulness and usability.
An issue that came out in the "Multi-Modal Perceptual and Motor Abilities" panel discussion concerned the emphasis on realistic graphics as a goal in VE research. Visual realism has several disadvantages in VEs, but the problem is...realism is cool.
It is easy to evaluate success when the goal is achieving realism. No training is needed nor an expert intermediary to explain it to you. Look at a radiosity image and you can see it looks realistic.
There are fundamental disadvantages to visual realism in VEs, besides being slow to compute and requiring extensive graphics hardware. Visual clutter gets in the way of performing tasks. What's useful for one task or user will be clutter for another task or user.
Visual realism impedes the flexibility needed to create experiences that depart from realism. It restricts a user's ability to annotate the virtual world. Both properties are important for multi-user VEs to support social perception and expression for work and play.
Visual realism is not necessary or sufficient depending on the user and task. But no one knows what to strip out. Or knows what it is about real life that makes it so usable. (You may notice on occasion that reality is not as usable as you'd like. Why is that?)
An example of a useful component of visual realism is the simple textures that provide optical flow in VEs. They contribute a psychologically useful element that is derived from natural visual scenes, without having to be high-fidelity.
Suppose we are designing a VE for exploring a scientific visualization, possibly high-dimensional. What is realism for abstract data? Is a molecular model real or an abstraction?
As the ecological psychologist James J. Gibson said, in natural environments surface is support. A flat horizontal surface with extent affords navigation. The fact that people navigate in two dimensions may contradict your expectations. People navigate in a 2D environment across the ground or some 2D substitute for the ground.
It is not a trivial transition for people to navigate in unconstrained 3D motion. How do you design the constraints, or the affordances, that implement a navigation metaphor? The constraints that are useful depend on the system, user, and task. A VE interface designer must also consider what the 3D interface hardware lets one create.
Experience suggests that 2D structures in 3D environments are usable, but that 3D structures in 3D environments are not, possibly with the exception of highly expert users, given our current understanding. One of our goals is to increase our understanding of the basic problems faced by non-expert users.
A challenge in VE interface design is that we provide new and interesting ways to get lost---and don't understand well how to fix that.
We need to boil down common issues from different disciplines to produce new joint knowledge. For example, from architecture: the shapes of spaces can carry information about their location and purpose. And careful use of lighting can enhance orientation.
We can try to design and use consistent languages for VE design, with vocabularies and grammars that let users see part of an environmental structure or process and use that part to predict the unseen remainder. This would help to enable re-use, new uses, and extensions in VEs by structuring the unknown with an understandable logic.
The known HCI methods apply to VE interface design and need to be applied vigorously because with VE applications, in a task analysis, the domain experts don't uncover everything relevant and we -- the technology experts -- don't know enough about what we can offer.
The workshop focused on human aspects of VE interaction---the psychological basis of difficulties and advantages of using VEs. The broadly interdisciplinary character of the fundamental issues makes research in this area difficult. The National Research Council's Committee on Virtual Reality Research and Development identified a wide range of problem areas as requiring extensive research for further virtual environment development.[1]
The CHI'94 Workshop on the Challenges of 3D Interaction report[2] listed a set of future goals that the participants "believe[d] the research community needs to" work on. Several of those goals overlap with the topics discussed in the 1996 Workshop on Psychological Issues of Virtual Environment Interfaces.
Ellen Isaacs and Vicki O'Day, the CHI 96 workshops chairs, were a great help during the workshop organization process. The audio-visual support was first-rate, thanks to A-V Chair Curt Stevens.
Casey Boyd
Dept. of Computer Science and Institute of Cognitive Science, University of Colorado, Boulder, USA
Rudy Darken
Naval Research Laboratory, Washington, D.C., USA
Richard Branham
Department of Design, The University of Kansas, Lawrence, USA
Matthew Chalmers
UBILAB, Union Bank of Switzerland, Zurich, Switzerland
William R. Cockayne
NPSNET Research Group, Department of Computer Science, Naval Postgraduate School, Monterey, USA
Matthew Conway
Dept. of Computer Science, University of Virginia, Charlottesville, USA
John R. Gersh
Applied Physics Laboratory, The Johns Hopkins University, USA
Stephen A. Gilbert
Dept. of Brain & Cognitive Sciences, MIT, USA
Scott B. Lewis
Intel Corporation, Hillsboro, OR, USA
Andrew Liu
Nissan Cambridge Basic Research, Cambridge, MA, USA
Catherine Ramzy
CNS and HPP Groups, Beckman Institute, University of Illinois, Urbana-Champaign, USA
Dave Roberts
IBM UK Limited, Warwick, England
Diane Schiano
Interval Research, Palo Alto, CA, USA
Mark Schlager
SRI International, Menlo Park, CA, USA
Dylan Schmorrow
Naval Air Warfare Center Aircraft Division, Human System Integration, Warminster, PA, USA
Chris Shaw
Department of Computer Science, University of Regina, Saskatchewan, Canada
Raoul Smith
College of Computer Science, Northeastern University, Boston, MA, USA
Michelle Vincow
Beckman Institute, University of Illinois, Urbana-Champaign, USA
Hans Weber
Department of Computer Science, University of North Carolina, Chapel Hill, USA
This lists some of the sources cited in participants' position papers.
Alexander, C., A City is Not A Tree, Design, 206, pp. 46-55, 1966.
Allen, G.L., et al., Developmental Issues in Cognitive Mapping: The Selection and Utilization of Environmental Landmarks. Child Development, 1979. 50: p. 1062-1070.
Aretz, A. J. (1991). The design of electronic map displays. Human Factors, 33(1), 85-101.
Begault, Durand R. 3-D Sound for Virtual Reality and Multimedia. Academic Press, 1994.
Bertin, J., Graphics and Graphic Information Processing, Walter de Gruyter, Berlin, 1981.
Blades, M. and C. Spencer, How Do People Use Maps To Navigate Through the World? Cartographica, 1987. 24(3): p. 64-75.
Blades, M. and C. Spencer, The Development of 3- to 6-Year-Olds' Map Using Ability: The Relative Importance of Landmarks and Map Alignment. The Journal of Genetic Psychology, 1988. 151(2): p. 181-194.
Boyd, C. (1995). Human and Machine Dimensions of 3D Interfaces for Virtual Environments, Doctoral Consortium, CHI'95, Denver, Colorado.
Boyd, C., M. Jones, and J. Thielen (1993). Visualizing the Performance of Parallel Programs: Interface design using task-centered walkthroughs. Technical Report CU-CS-683-93, University of Colorado, Boulder.
Card, Stuart K., Jock D. Mackinlay, and George G. Robertson. "A Morphological Analysis of the Design Space of Input Devices," Transactions on Information Systems, 9(2), pp. 99-122, ACM, April 1991.
Chalmers, M. and P. Chitson, Bead: Explorations in Information Visualisation, in Proc. SIGIR'92, published as a special issue of SIGIR Forum, ACM Press, pp. 330-337, June 1992.
Chalmers, M., Using A Landscape Metaphor to Represent a Corpus of Documents, Proc. European Conference on Spatial Information Theory, Elba, September 1993. Published as Spatial Information Theory, Springer Verlag LNCS 716, A. Frank and I. Campari (eds.), 1993, pp. 377-390.
Colston, H. L., & Schiano, D. J. (1995). Looking and lingering as conversational cues in video-mediated communication. Short paper in Proceedings of the Annual Meeting of the ACM Computer-Human Interaction Special Interest Group (CHI'95), Denver, CO.
Conway, Matthew, Laura Vogtle, and Randy Pausch. "One-Dimensional Motion Tailoring for the Disabled," Presence, 3(3), p. 244, 1994.
Darken, R. (1995). Wayfinding in Large-Scale Virtual Worlds. SIGCHI, ACM.
Darken, R. and J.L. Sibert. A Toolset for Navigation in Virtual Environments. in User Interface Software and Technology. 1993. Atlanta, GA: ACM Press.
Darken, R. and J.L. Sibert, Navigating in Large Virtual Worlds. International Journal of Human-Computer Interaction, 1996, in press.
Darken, R. and J.L. Sibert, Wayfinding Strategies and Behaviors in Large Virtual Worlds. ACM SIGCHI 96, 1996, in press.
Dixon, K. W., G. M. Krueger, V. A. Rojas and D. C. Hubbard (1989). The effect of instantaneous field of view size on the acquisition of low level flight and 30 degrees manual dive bombing tasks. Proceedings of the SPIE - The International Society for Optical Engineering 1116: p. 110-21.
Forsey, David and Richard Bartels. Hierarchical B-Spline Refinement. Computer Graphics (Proceedings of SIGGRAPH `88), 22(4):205-212, August 1-5 1988.
Gersh, J. R., "Exploring the design of rule-based supervisory control systems: From the Gulf of Evaluation to the Persian Gulf," Proc. ACM CHI-91 Workshop on Advances in Computer-Human Interaction in Complex Systems, New Orleans, La, pp. 43-48, April 1991.
Gersh, J. R., "Information presentation and action taking in complex systems - mapping the territory," presented at the ACM CHI-92 Workshop on Advances in Computer-Human Interaction in Complex Systems, Monterey, Calif., May 1992.
Gersh, J. R., and B. W. Hamill, "Cognitive engineering of rule-based supervisory control systems: Effects of concurrent automation," Proc. 1992 IEEE International Conference on Systems, Man, and Cybernetics, Chicago, Ill., pp. 1208-1213, October 1992.
Gersh, J. R. and B. W. Hamill, "Effects of controls and displays on the use of rule-based supervisory control systems," Proc. 1993 IEEE International Conference on Systems, Man, and Cybernetics, Le Touquet, France, v. 3, pp. 376-381, October 1993.
Gibson, J. J. The Ecological Approach to Visual Perception. Houghton Mifflin, Boston, 1979.
Gomez, Louis M., Dennis E. Egan, Evangeline A. Wheeler, Dhiraj K. Sharma, and Aleta M. Gruchacz. "How Interface Design Determines Who Has Difficulty Learning to Use a Text Editor," in ACM CHI'83 Proceedings, p. 176, 1983.
Gordon, I. E. (1989). Theories of Visual Perception. New York, John Wiley and Sons.
Guiard, Yves. Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. The Journal of Motor Behavior, 19(4):486-517, 1987.
Hamill, B. W. and J. R. Gersh, "Decision-making performance in rule-based supervisory control: Empirical Development of a cognitive process model," Proc. 1992 Symposium on Command and Control Research, Joint Directors of Laboratories, Monterey, Calif., pp. 100-106, June 1992.
Hertzog, Christopher and Bert Rypma. "Age differences in Components of Mental Rotation Task Performance," Bulletin of the Psychonomic Society, p. 209, 1991.
Hodges, L. F., , J. Bolter, E. Mynatt, W. Ribarsky and R. van Teylingen (1993). Virtual Environments Research at the Georgia Tech GVU Center. Presence 2(3): 234-243.
Ingram, R. and S. Benford, Legibility Enhancement for Information Visualisation, Proc. Visualization 95, IEEE Press, pp. 209-216.
Landau, B., Early Map Use as an Unlearned Ability. Cognition, 1986. 22(3): p. 201-223.
Levy, E., Zacks, J., Tversky, B. and Schiano, D. J. (1996, accepted). Gratuitous graphics? Putting preferences in perspective. Paper in Proceedings of the Annual Meeting of the ACM Computer-Human Interaction Special Interest Group (CHI'96, Vancouver, B.C.) IRC Internal Technical Report 96-XXX.
Lin, X., D. Soergel, and G. Marchionini, A Self-Organizing Semantic Map for Information Retrieval, in Proc. SIGIR'91, published as a special issue of SIGIR Forum, ACM Press, pp. 262-269, October 1991.
Liu, Andrew and Lawrence Stark. "The Effect of Monocular Blur on Simulated Telerobotic Manipulation". IEEE Trans. Rob. Autom., 7(3): 372-376, 1991.
Liu, Andrew and Stacey Chang. "Force Feedback in a Stationary Driving Simulator". Proc. of the 1995 IEEE Intl. Conf. on Systems, Man, and Cybernetics, vol. 2, pp. 1711-1716, Vancouver, BC, Canada, October 23-25, 1995.
Liu, Andrew, Gregory Tharp, Lloyd French, Stephen Lai, and Lawrence Stark. "Some of What One Needs to Know About Helmet-Mounted Display Performance". IEEE Trans. Rob. Autom., 9(5):638-648, 1993.
Liu, Andrew, Gregory Tharp, Michitaka Hirose, and Lawrence Stark. "Visual Factors Affecting Human Operator Performance with a Helmet Mounted Display". In SAE Transactions - Journal of Aerospace, vol. 100, pp. 976-84, Society of Automotive Engineers, 1992.
Lynch, K., The Image of the City, MIT Press, Cambridge, 1960.
Mackinlay, Jock D., Stuart K. Card, and George G. Robertson. "Rapid Controlled Movement Through a Virtual 3D Workspace," Computer Graphics, 24(4), pp. 171-176, ACM, 1990.
May, M., P. Peruch, and A. Savoyant, Navigating in a Virtual Environment With Map-Acquired Knowledge: Encoding and Alignment Effects. Ecological Psychology, 1995. 7(1): p. 21.
McCormick, E. P., and Wickens, C. D. (1995). Virtual reality features of frame of reference and display dimensionality with stereopsis: Their effects on scientific visualization (Technical report No. ARL-95-6/PNL-95-1). Aviation Research Laboratory Institute of Aviation, University of Illinois at Urbana-Champaign, Savoy, Illinois.
Muehrcke, P.C., Map Use: Reading, Analysis, and Interpretation. 1978, Madison, WI: J. P. Publications.
Mullet, K. and Schiano, D.J. (1995) "3D or not 3D: `More is better' or `less is more'"? Panel in Proceedings of the Annual Meeting of the ACM Computer-Human Interaction Special Interest Group (CHI'95), Denver, CO.
Passini, R. (1992). Wayfinding in Architecture. New York, Van Nostrand Reinhold.
Passini, R. and P. Arthur (1992). Wayfinding: People, Signs, and Architecture. New York, McGraw-Hill Book Company.
Passini, R., G. Proulx and C. Rainville (1990 Jan). The spatio-cognitive abilities of the visually impaired population. Environment and Behavior 22(1): 91-118.
Pausch, Randy, M. Anne Shackelford, and Dennis Proffitt. "A User Study Comparing Head-Mounted and Stationary Displays," in IEEE Symposium on Research Frontiers in Virtual Reality, October, 1993.
Petrie, Helen, Sarah Morley, and Gerhard Weber. "Tactile-Based Direct Manipulation in GUIs for Blind Users" in Proceedings of CHI'95, ACM/SIGCHI, May, 1995.
Piantanida, T. P., D. Boman, J. Larimer, J. Gille and C. Reed (1992). Studies of the field-of-view/resolution tradeoff in virtual-reality systems. Proceedings of the SPIE - The International Society for Optical Engineering 1666: p. 448-56.
Presson, C.C., N. DeLange, and M.D. Hazelrigg, Orientation Specificity in Spatial Memory: What Makes a Path Different From a Map of the Path? Journal of Experimental Psychology: Learning, Memory, and Cognition, 1989. 15(5): p. 887-897.
Rieman, J. A Field Study of Exploratory Learning Strategies. Ph.D. Dissertation, University of Colorado, 1994.
Sadalla, E. K.,Burroughs, W. J., & Staplin, L. J. (1980). Reference points in spatial cognition. Journal of Experimental Psychology: Human Learning and Memory, 6, 516-528.
Schiano, D.J. and McBeath, M.K. (1996, submitted). Virtual Reality versus Real Virtuality: The Role of Realism in VR Environments. Symposium on Virtual Environments proposed for the meeting of the American Psychological Society, San Francisco.
Schlager, M.S. (1994). An approach for designing virtual environment training systems. In CHI'94 Conference Companion, New York: ACM Press, 317-318.
Schlager, M.S., Boman, D., and Piantanida, T. (1993). Virtual environment systems for maintenance training (Volume 3). SRI Technical Report No. 8216, Menlo Park, CA: SRI International.
Schlager, M.S., Boman, D., Piantanida, T. and Stephenson, R. (1992). Forecasting the impact of virtual environment technology on maintenance training. In Sixth Annual Workshop on Space Operations, Applications, and Research , NASA Pub. 3187, Houston, TX, August 4-6, 691-699.
Schlager, M.S., Mumaw, R.J., and Hoecker, D.G. (1993). Analytic tools for designing virtual environment training systems: Nuclear power plant maintenance applications. In Proceedings of the 1993 Conference on Intelligent Computer-Aided Training and Virtual Environment Technology, NASA JSC, Houston, TX, May 5-7.
Shaw, Chris and Mark Green. THRED: A Two-Handed Design System. Multimedia Systems Journal, 3(6):to appear, November 1995.
Shaw, Chris and Mark Green. Two-Handed Polygonal Surface Design. In UIST 1994 Proceedings, pages 205-212, Marina del Rey, California, November 2-4 1994. ACM SIGGRAPH/SIGCHI.
Shepard, Roger N. and Jacqueline Metzler. "Mental Rotation of Three-Dimensional Objects," Science, 171, p. 701, 1971.
Stenning, K. and J. Oberlander, Reasoning with Words, Pictures and Calculi: Computation Versus Justification, in Situation Theory and its Applications, vol. 2, Stanford University, 1991.
Subbiah, Ilavenil, Leandro Veltri, Andrew Liu, and Alex Pentland. "Developing Mental Maps and Learning Locations of Objects Along a Route While Driving in a Simulated Environment." In preparation. Presented in part at the Eastern Psychological Assoc. Annual Meeting, Boston, MA, April 1995, and OPAM, Nov. 95, Los Angeles, CA.
Swartz, M., D. Wallace and S. Tkacz (1992). The influence of frame rate and resolution reduction on human performance. Proceedings of the Human Factors Society 36th Annual Meeting. Innovations for Interactions, Santa Monica, CA, USA, Human Factors Soc.
Tachi, Susumu and Kenichi Yasuda. "Evaluation Experiments of a Tele-existence Manipulation System," Presence, 3(1), p. 35, 1994.
Tatum, S. A., J. C. Byrum and P. W. Rourke (1994). Design Validation Using Computer Models in Lieu of Full-Scale Mockups. Marine Technology 31(3): 225-230.
Warren, W. H., Jr., D. R. Mestre, A. W. Blackwell and M. W. Morris (1991). Perception of Circular Heading from Optical Flow. Journal of Experimental Psychology: Human Perception and Performance 17(1): 28-43.
Welch, Robert B., Andrew Liu, Theodore T. Blackmon, Barbara A. Mellers, and Lawrence Stark. "The Effect of Pictorial Realism, Delay of Visual Feedback, and Observer Interactivity on the Subjective Sense of Presence in a Virtual Environment". Presence: Teleoperators and Virtual Environments, 1996, (in press).
Wickens, C. D. (1992). Engineering Psychology and Human Performance. New York, NY: HarperCollins.
Wickens, C. D., & Baker, P. (1995). Cognitive issues in virtual reality. In W. Barfield & T. F. III (Eds.), Virtual Environments and Advanced Interface Design New York: Oxford University Press.
Willerman, Lee. The Psychology of Individual and Group Differences, W. H. Freeman, 1979.
Witmer, B. G., J. H. Bailey, B. W. Knerr and K. Abel. Training Dismounted Soldiers in Virtual Environments: Route Learning and Transfer.
Woods, D. D. (1984). Visual momentum: A concept to improve the cognitive coupling of person and computer. International Journal of Man Machine Studies, 21, 229-244.
Zhai, Shumin and Paul Milgram. "Input Techniques for HCI in 3D Environments," in CHI'94 Conference Companion, April 24-28, 1994.
Casey Boyd is completing his Ph.D. thesis at the University of Colorado, Boulder. His research interests include task analysis, usability, design, and evaluation of virtual environments.
E-mail:
cboyd@cs.colorado.edu
Rudy Darken recently completed hisdissertation in Computer Science at The George Washington University. His research interests include spatial orientation and wayfinding in large-scale virtual worlds.
E-mail:
darken@enews.nrl.navy.mil
Issue |
Article |
Vol.28 No.4, October 1996 |
Article |
Issue |