Issue
Article
Vol.30 No.2, April 1998
Article
Issue
Issue |
Article |
Vol.30 No.2, April 1998 |
Article |
Issue |
Only recently have we begun to appreciate, and experiment with, the possibility of escaping the static and rigid constraints of writing on paper.
Imagine you are looking at a small area on a computer screen. Words appear and disappear on the screen one by one. As they appear, meaning is expressed as forms change dynamically over time. (Wong [4]).
Dynamic documents allow visual manipulation of document content over time. This adds a new expressive dimmension which can be used to enhance the message contained in the document.
In the work described here, we use dynamic presentation to hilite, and thus enhance the semantic structure of documents. To do this, we use the linguistic techniques of meaning analysis and content abstraction to produced semantic hilites of documents in terms of topics and shifts in discourse structure [2, 3]. These semantic objects are then given visual treatment within a dynamic on-line presentation of the document.
There are several appealing aspects to dynamically presenting abstractions of document content. The user need not be actively involved: as documents arrive at the desktop, they can be analysed and the resulting content abstractions can be displayed autonomously. Should the user have the time or inclination to focus on a particular document, interactive controls will be at their disposal; alternatively, each new arrival can be presented under its own schedule, followed by another, and so on. The presentation cycle can be customised to make use of arbitrary combinations of granularity of expressiveness. Notions like semantic highlights and demarcation of context are easily mapped onto visual metaphors, and thus naturally support the expression of content by means of variations of form. Cognitively, short phrases with high semantic load are amenable to punctuated display following a natural rhythm of visual perception.
We start by briefly describing the linguistic analysis used as the basis of the dynamic documents. Following this an overview of dynamic documents is presented. Finally several prototype document viewers are described.
One of the problems of information management, when presented with a growing surplus of text documents, is getting some appreciation -- rapidly, compactly, and yet with a usable degree of depth and representativeness -- of the information contained in a document. Informally, this is usually referred to as the "aboutness" of a document. This is represented as a set of highly salient, and by that token most representative, phrases in the document. Viewing topicality in its stricter, linguistic, sense, we define topic stamps to be the most prominent of these phrases, introduced into, and then elaborated upon, the document body. On the basis of this definition, we have developed a computational, algorithmic, procedure for generating a set of abstractions for the core meaning in the document [1], ultimately resulting in a capsule overview of the document based upon suitable presentation of the most representative, and most contentful, expressions in the text. These abstractions comprise layered and inter-related phrasal units at different levels of granularity, following the containment hierarchy presented in figure 1. For more details about the linguistic analysis techniques we use please see [3] in this volume.
Figure 1: Object hierarchy produced by linguistic analysis
Dynamic content delivery is based on ideas of temporal typography developed by Wong [4]. This work is based upon synergy of psychological studies of reading, graphic design, and temporal presentation of text. Graphic design history is rich with examples of experimenting with visual treatment of written language. Designers have begun to explore temporal presentation of text in television and film media. Studies of reading, which to a large extent form the basis of Wong's work, have explored dynamic presentation of content, related to the interactions between meaning and intent of a text-based message. However, Wong's studies of the dynamic relationship between meaning and delivery formats assume that the annotations for meaning in her experiments have been done by hand. In contrast, here we focus on leveraging automatic document content analysis technology, capable of delivering meaning analyses and content abstractions precisely of the kind which can be effectively coupled with dynamic content delivery.
With the migration of news delivery over the World Wide Web and the growth of information `push' vendors, we are beginning to see some methods for presentation of news stories which use notions of dynamic delivery of content. Most of these are variations on the same theme: news delivery using the ticker metaphor. Thus both ABC's news site (http://www.abc.com) and Pointcast (http://www.pointcast.com) employ a traditional horizontal ticker, CNN Interactive (http://www.cnn.com) arrange their ticker vertically, CBS (http://www.uttm.com) combine a ticker with photos from a major story.
The important insight here is that tickers are dynamic objects, which can be programmed to continuously update themselves from a news feed and to cycle in a pre-defined regime, therefore not requiring user intervention. Furthermore, they can be dispatched to an area of the workspace (monitor screen) where constant, yet inobtrusive, news delivery can take place in the periphery of the user's main activity: thus a choice exists between pro-active engagement with the news source, and passive (and almost subliminal) monitoring of news data.
None of the examples above, however, combines a ticker with an automatic summarisation engine. To a large extent this is because sentences -- especially inconsecutive ones, in the absence of visual markers for discontinuity -- do not lend themselves easily into the word by word, left to right, presentation mode. This is clearly a situation where phrasal units of a sub-sentence granularity can be used much more effectively. In addition, psychological experiments on active reading (see [4]) show that when text is presented dynamically in the manner of a ticker, subjects' reading speeds are significantly slower than for text presented statically. On the other hand, dynamic presentations of text which show words or short phrases in the same location but serially, one after the other, have reading speeds comparable to those for normal static texts.
To date, no applications have been developed utilising temporal typography for dynamic delivery of content abstractions. Wong has looked at how dynamic type in general can be used for four different communicative goals: expressive messages, dialogue, active reading and real time conversation (see [4]). Most relevant to this discussion are her experiments on active reading. In one of these she used a basic rapid serial visual presentation (RSVP) method (words or phrases presented sequentially one after another, on the same line and at the same position) to deliver a sequence of news headlines. In a second set of experiments called Highway News, she used three dimensions, combined with a zooming motion, to present a sequence of text highlights. "News headlines are placed one after another in the z-dimension. Headlines are presented serially according to active input from the reader. The reader presses a mouse button to fly through the rows of headlines -- as if flying over a highway of text." [4]. These experiments show the strong feasibility of high impact, low engagement, delivery of semantically prominent text fragments being utilised as a powerful technique for visualising certain types of inherently linear information.
None of the work cited above relies on automatically generated meaning abstractions as its input; yet, it is clear that the topically-rich capsule overviews generated by the document analysis technology we have developed are just the kind of semantic highlights which Wong's experiments in active reading assume. Conversely, up till now there has been no thought as to how the nature of topic-based capsule overviews in particular would fit the notion of dynamic type. This is the subject of the following section.
At the time of writing, three different viewers explore the ideas discussed above. The difference is largely due to the variety of operational environments in which the viewers have been applied. A variation on a news ticker is designed to be deployed in situations where screen real estate may be at premium, or where several different channels of information may be delivered simultaneously to the same `in-box'; typically such situations assume that users would only want to get a very general idea of document content. For situations where more screen real estate might be available, and/or it may be known ahead of time that more detail concerning document content might be required, a different viewer develops ideas from rapid serial visual presentation (RSVP). Yet another interface caters to the need to be able to get immediate access to the full text of a document, without losing the benefits of rapid skimming through content highlights while fully maintaining information about the larger context.
All of the viewers assume an environment where incoming documents get analysed to capsule overview level; the results of the analysis are embedded into the original text by means of special purpose tags. These augmented text files are then used by the dynamic viewers, the augmentations being translated into specific on-screen visualizations.
TopicsTicker is a minimalist, hands-free, peripheral-vision-directed ticker tape, with certain aspects of its display tuned for serial delivery of a document's topic stamps: the string in the left panel is the document title, and the right panel is where the display cycles through the document's topic stamps. When running over a set of documents, switching from one document to the next is cued by a colour change and a vertical scroll.
Figure 2: TopicsTicker
RSVP presents the user with a sequence of topically salient phrases as per the capsule overview analysis. In its basic mode with no user interaction, RSVP cycles through salient relational contexts in a document, maintaining the order in which they appear in the text. Each context phrase is displayed as the prominent object on the screen; at the same time the context is overlaid onto topic expansions (displayed as translucent text). This facilitates further interpretations of the context strings by the user: expansions relate phrasal contractions in the string displayed to their full canonical forms in the text, make clear antecedents of dangling anaphors, and so forth. Note, for instance, the background display of the full form of the antecedent for the anaphoric "he" in the foreground.
Figure 3: RSVP showing a topic stamp in the background and its relational context in the foreground
Cycling though the complete set of salient contexts, in their original sequence, offers a good indication of aboutness at a given level of depth and detail. Granularity of display is adjustable via a parameter: thus RSVP could be reduced to a topics ticker by only cycling through the document's topic stamps, or it could be used to display sequences of sentences. Relational contexts offer just the right balance between terseness (phrases are more easily perceived and assimilated than sentences) and informativeness (phrases larger than `bare' topic stamps convey richer data). The amount of time a phrase is displayed is dynamically calculated; the intent is to optimise the full document display regime so that individually, each phrase can be processed by the user, while globally, the entire set of salient contexts can be cycled through rapidly.
There are provisions for maintaining overall context, by continuously displaying the title of the current document, as well as for allowing context switching, by user selection of a document from a pop-up menu.
RSVP is designed as an entirely autonomous viewer: after all the content highlights in a document have been displayed, the next document will be loaded and the cycle repeated (just like in TopicsTicker, a suitable visual cue signals document change). This makes it very appropriate for situations where readers might not have much time, bandwidth, or opportunity to interact with the display, but they would want to be peripherally aware of new documents that come into the system. On the other hand, if a particular context catches the users attention, a visual `zoom' mechanism makes use of the multiple levels of analysis of the document (as defined via the containment hierarchy, see figure 1). This will reveal, on demand, progressively larger and more detailed document fragments: sentences, paragraphs and segments. For instance, further specifics concerning what "he says" (see the earlier illustration) is immediately available by a single click in the display area.
Figure 4: After the user has clicked on the topic stamp and its relational context, RSVP reveals more details about the document by moving the relational context to the background and showing the topical sentence in the foreground
At any given point of time and depth of detail, the display uses a combination of visual cues to highlight the information-bearing unit which is in focus, and associate this with the larger context in which it appears in the original source. In particular, properties of type, both static and dynamic, come to convey various aspects of the document analysis: primary focus of attention is denoted by using heavy black typeface; background context, by using translucent text; semantic relatedness, by overlaying the focus and context onto the same display area; different level of attention to detail, by visually and perceptibly zooming in when more detail is requested, and by zooming out when the user retreats back into the default `fly-through, from bird's eye view' mode. Note that while such visual devices are very effective for delivering document highlights, they rely crucially on being able to carry out the layered linguistic analysis described earlier.
The RSVP viewer is particularly well-suited for deployment in a screen saver mode, in a background window on a desktop machine, or on a large screen projection in communal areas. In any of these situations, a topic or context might catch a reader's peripheral attention, and then they can decide to take a further look. RSVP thus naturally extends, and fits into, the emerging "push" model of information delivery.
The ViewTool viewer freely borrows some of the ideas of RSVP. However, the emphasis here is to present a fuller overview of the salient topic stamps in a document, contextualising this to a document `thumbnail', indicative of the distribution of these highly salient objects in the text. At the same time, a separate `details' area constantly displays additional information pertinent to the current information-seeking context deployed by the user. The details area is used both for dynamic display of richer contexts, as in the RSVP viewer, and for providing access to the full text, or topically coherent segments from it, on demand. Thus, the aim of this viewer is to develop a more elaborate notion of context, while maintaining permanent focus on the salient highlights (topic stamps) in the document. ViewTool further seeks to offer more interactivity to the user, in ways which make the targeted exploration of portions of the document natural and transparent.
ViewTool places the capsule overview of a document within the context of the document itself. This is maintained by synchronized display of discourse segments, topic stamps, and relational contexts in three panels. The whole document is displayed in the left panel; this is deliberately unreadable, and is intended to function as a document thumbnail serving as a contextual referent for the topics presented in the central panel. With the use of an appropriate colour coding scheme, it also serves as an indicator of the distribution of topically prominent phrases in the document. The central panel lists the highly salient topic stamps. Contextualisation for these is achieved by aligning the topic stamps for a given discourse segment with the textual span of that segment in the thumbnail. This offers an immediate overview of, for instance, what is being discussed in the beginning of the document, or in the end, or which topics keep recurring throughout, and so forth.
Figure 5: Viewtool where a user is scanning the topic stamps in the middle panel. Currently they are holding the mouse over a set of topic stamps from the discourse segment selected in the left-most panel. The right panel displays in an RSVP-like manner, the topic stamps and their relational contexts
The central panel is sensitive to the user's focus of attention: as the mouse rolls over a topic stamp, the discourse segment from which this topic has been extracted is highlighted in the left panel. The highlighting also indicates the segmentation of the source documents into topically different, and distinct, text sections. This design makes it easy to do rapid selection of areas of interest in the document, as it is mediated by the topic stamps per segment display. Again, the granularity of analysis and the layered contextual information in the capsule overview make it easy to offer immediate and more detailed information about any given set of topic stamps: simultaneously with highlighting the appropriate discourse segment in the left panel, relational contexts for the same set of topic stamps are displayed cyclically, in RSVP-like fashion, in the right panel. This ensures that topic stamps are always related with contextual cue phrases. Thus an additional level of detail is made available to the user, with very little `prompting' on their part; if it is still the case that the full text of the segment would be required, clicking on its `proxy' topic stamps (in the middle panel) would display this in the right panel. The larger area available there, as well as an automatic readjustment of the size of type, ensures that the text is readable.
Figure 6: Viewtool after the user has clicked on the set of topics hilited in the middle panel, the right most panel changes to show the discourse segment that contains those topic stamps
As a natural extension of the same metaphor, clicking on the document proxy in the left panel brings up the full document text in the right panel. The full text always uses colour markup to indicate, in yet another way, topically salient phrases and their relational contexts.
By always maintaining the larger document context for any grain of information of relevance and interest in the documents, ViewTool is an ideal skimming tool, because it provides additional information that may be important in deciding whether looking more closely at the document would be required. For example, users can get a sense of the size of the document, whether it contains any pictures, and other visual cue features. They can see the density of topics and the relevant ordering of the topics in relation to the different sections of the document. The tool offers the ability to see arbitrarily detailed contextual information relevant to a topic, while leveraging that same containment hierarchy of layered information units to prevent overload.
The viewers are fully implemented in Java, and are deployed in a variety of ways within a suite of intranet tools for collaboration and communication within Learning Communities. In general, an on-line newspaper has been configured as the primary news source within a learning community. By means of a variety of Web spiders and document filters, external news stories are collected and `published' in the newspaper. RSVP is used for primary delivery of the external news, on a dedicated page, projected on large display in a shared common area. TopicsTicker offers a brief overview of the latest news on the front page of the newspaper. ViewTool is available as an alternative browser, for more pro-active and focused access to the document -- particularly in situations where the newspaper is being viewed on personal workstations. ViewTool is also used for browsing of personal information feeds, sent to a document analysis server engine via a simple e-mail protocol. In general, any configuration of viewers can be deployed for personalised `windows' into continuous news feeds, combining a variety of screen delivery modes.
The notions of temporal typography and dynamic delivery of content mediated via content highlights offer an appealing approach to on-line document presentation and one that can offer new ways to think about how to support rapid browsing of on-line documents.
This project was the joint work of an interdisciplinary team. Yin Yin Wong is an interface design consultant, who designed the initial prototypes of the RSVP and ViewTool Dynamic Viewers. Chris Kennedy is a linguist who worked on formalizing some aspects of the analysis technology. Sascha Brawer is a computational linguist who re-engineered the earlier analysis prototypes into a real-time architecture for content analysis. Jason Swartz programmed the Java versions of the dynamic viewers. He designed the Topics Ticker. We would like to thank all members of the team for their contributions to this work. We would also like to thank Stephanie Houde for her support, design critiques and comments.
[1] Boguraev, B.K.B. and Kennedy, C. Salience-based content characterisation of text documents. In Proceedings of ACL'97 Workshop on Intelligent, Scalable Text Summarisation, Madrid, Spain, 1997.
[2] Boguraev, B.K.B, Wong, Y.Y., Kennedy, C. Bellamy, R.K.E., Brawer, S. and Swartz, J. Dynamic presentation of document content for rapid on-line skimming. Proceedings of AAAI Spring Symposium on Intelligent Text Summarisation, Stanford, CA, 1998.
[3] Boguraev, B. K. B., Brawer, S. and Kennedy, C. An architecture for content analysis of documents and its use in information and knowledge management tasks. SIGCHI Bulletin, this volume.
[4] Wong, Y. Y. Temporal typography. characterization of time-varying typographic forms. Master's Thesis, MIT Media Lab, 1995.
Branimir Boguraev holds degrees in Computer Science and Applied Mathematics from the Higher Institute for Mechanical and Electrical Engineering in Sofia, Bulgaria. He obtained a Ph.D. in Computational Linguistics in the Computer Laboratory at the University of Cambridge, UK. Since then, he has held several research award positions, under the UK Information Technology Initiative, and has been broadly involved in work on developing a broad natural language technology base. In 1988, he joined IBM's T.J. Watson Research Center, focusing on data-intensive methods for constructing rich computational lexicons. More recently, he has managed research efforts in natural language processing, first at IBM Research, and following that, at Apple's Advanced Technology Group. His interests are in linguistically intensive methods for document content analysis, and their deployment in information and knowledge management tasks.
Rachel Bellamy has conducted research in user interface design since 1987. For the last five years, she has worked in Apple's Advanced Technology group. Her work at Apple has focused on the design and use of collaborative technologies to support K-12 and Organizational Learning.
Issue |
Article |
Vol.30 No.2, April 1998 |
Article |
Issue |