Making hypermedia adaptable for multiple forms of presentation involves enabling multiple distinct specifications for how a given collection of hypermedia can have its presentation generated. The Standard Reference Model for Intelligent Multimedia Presentation Systems describes how the generation of hypermedia presentation can be divided into distinct but cooperating layers. Earlier work has described how specifications for generating presentations can be divided into distinct modules of code corresponding to these layers. This paper explores how the modules for each layer of a presentation specification can be exchanged for another module encoded for that layer and result in the whole specification remaining well functioning. This capability would facilitate specifying presentation generation by allowing for the use of pre-programmed modules, enabling the author to focus on particular aspects of the presentation generation process. An example implementation of these concepts that uses current and developing Web standards is presented to illustrate how wide-spread modularized presentation generation might be realized in the near future. Mix'n'Match: Exchangeable Modules of Hypermedia Style 1999 179-188 ht99 Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman, Dick C.A. Bulterman. Mix'n'Match: Exchangeable Modules of Hypermedia Style, 1999 http://homepages.cwi.nl/$sim$media/conferences/DIR2003/I2RP.DIR2003.ppt The paper introduces a knowledge-based multimedia approach to multimedia information retrieval. The approach uses domain knowledge to augment a user' s query, performs automatic ontology mapping to search different multimedia databases, and combines the results in a multimedia presentation. The texts in the presentation are generated from the domain knowledge. Thus, the user can view a coherent multimedia presentation that contains the answer to his or her query. The paper describes an architecture for realizing the approach. The individual parts of the architecture have been implemented, but are not yet integrated in one system. Intelligent Information Retrieval and Presentation with Multimedia Databases semantic web technologies, multimedia presentations, ontology mapping, natural language generation 52-56 dir03 Floris Wiesman, Stefano Bocconi, Boban Arsenijevic, Yulia Bachvarova, Nico Roos, Lambert Schomaker. Intelligent Information Retrieval and Presentation with Multimedia Databases Hypermedia documents are most often created with a particular presentation environment in mind. This requires the authoring of one document per presentation platform. As pointed out in [], much implementation effort can be avoided by specifying how the same underlying document can be presented in different environments. A style sheet defines a mapping from a source document to a presentation for it. We discuss the existing use of style sheets as applied to text and discuss their application to the case of hypermedia, and in particular how they need to be extended. Style Sheet Support for Hypermedia Documents 216-217 ht97 Jacco van Ossenbruggen, Lynda Hardman, Lloyd Rutledge, Anton Eliens. Style Sheet Support for Hypermedia Documents Being able to author a hypermedia document once for presentation under a wide variety of potential circumstances requires that it be stored in a manner that is adaptable to these circumstances. Since the nature of these circumstances is not always known at authoring time, specifying how a document adapts to them must be a process that can be performed separately from its original authoring. These distinctions include the porting of the document to different platforms and formats and the adapting of the document s presentation to suit the needs of the user and of the current state of the presentation environment. In this paper we discuss extensions to our CMIF hypermedia authoring and presentation environment that provide adaptability through this distinction between authoring and presentation specification. This extension includes the use of HyTime for document representation and of DSSSL for presentation specification. We also discuss the Berlage architecture, our extension to HyTime that specifies the encoding of certain hypermedia concepts useful for presentation specification. A Framework for Generating Adaptable Hypermedia Documents 1997 Hypermedia, Open Hypermedia, HyTime, DSSSL, SGML. 121-130 mm97 Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman, Dick C.A. Bulterman. A Framework for Generating Adaptable Hypermedia Documents, 1997 This is a short version of CWI technical report INS-R0303 Multimedia presentation generation has to be able to balance the functional aspects of a presentation that address the information needs of the user and its aesthetic form. We demonstrate our approach using automatic colour design for which we integrate relevant aspects of colour theory. We do not provide a definition of the relative importance of form versus function, but seek to explore the roles of subjective elements in the generation process. Colour Picking - the Pecking Order of Form and Function mm2003 Frank Nack, Amit Manniesing, Lynda Hardman. Colour Picking - the Pecking Order of Form and Function http://www.cwi.nl/$sim$lynda/madrid2000/Time.PDF Most hypermedia documents don't incorporate time explicitly. This prevents authors from having direct control over the temporal aspects of a presentation. In this paper we discuss the concept of presentation time - the timing of the individual parts of a presentation and the temporal relations among them. We argue why time is necessary from a presentation perspective, and discuss its relationship with other temporal views of a presentation. We derive the requirements and present our solution for incorporating temporal and linking information in a model of time-based hypermedia. Do You Have the Time? Composition and Linking in Time-based Hypermedia 1999 189-196 ht99 Lynda Hardman, Jacco van Ossenbruggen, Lloyd Rutledge, K. Sjoerd Mullender, Dick C.A. Bulterman. Do You Have the Time? Composition and Linking in Time-based Hypermedia, 1999 Links in hypermedia: the requirement for context 1993 183-191 ht93 Lynda Hardman, Dick C.A. Bulterman, Guido van Rossum. Links in hypermedia: the requirement for context, 1993 Also for multimedia, there is a need to create presentations in presentation-independent forms and have the final form(s) generated from the base document. We can input our expertise on the specific requirements for multimedia in automatic document generation. In addition, we have experience in processing multimedia documents in a Web context in general and related issues within SMIL 2.0 in particular. We can explain the use of alternative media for devices where text is inappropriate. In addition, different sorts of meta-data can be incorporated into a presentation-independent base document, and device/user-specific presentations can be generated from this base document, using the information captured in the meta-data. In addition, we can explain the advantages of incorporating timing as one of the dimensions of variation in a presentation. For example, adding temporal variations within HTML, using e.g. HTML+SMIL, can allow different types of trade-offs in a presentation. Device Independent Multimedia Authoring diaworkshop Lynda Hardman, Jacco van Ossenbruggen. Device Independent Multimedia Authoring This is a revised version of CWI technical report INS-R0305 Traditionally, research in applying Semantic Web technology to multimedia information systems has focused on using annotations and ontologies to improve the retrieval process. This paper concentrates on improving the presentation of the retrieval results. First, our approach uses ontological domain knowledge to select and organize the content relevant to the topic the user is interested in. Domain ontologies are valuable in the presentation generation process, because effective presentations are those that succeed in conveying the relevant domain semantics to the user. Explicit discourse and narrative knowledge allows selection of appropriate presentation genres and creation of narrative structures, which are used for conveying these domain relations. In addition, knowledge of graphic design and media characteristics is essential to transform abstract presentation structures into real multimedia presentations. Design knowledge determines how the semantics and presentation structure are expressed in the multimedia presentation. In traditional Web environments, this type of design knowledge remains implicit, hidden in style sheets and other document transformation code. Our second use of Semantic Web technology is to model design knowledge explicitly, and to enable it to drive the transformations needed to turn annotated media items into structured presentations Towards Ontology-driven Discourse: From Semantic Graphs to Multimedia Presentations 597-612 iswc2003 Joost Geurts, Stefano Bocconi, Jacco van Ossenbruggen, Lynda Hardman. Towards Ontology-driven Discourse: From Semantic Graphs to Multimedia Presentations http://homepages.cwi.nl/$sim$media/conferences/HT03/ Generating hypermedia presentations requires processing constituent material into coherent, unified presentations. One large challenge is creating a generic process for producing hypermedia presentations from the semantics of potentially unfamiliar domains. The resulting presentations must both respect the underlying semantics and appear as coherent, plausible and, if possible, pleasant to the user. Among the related unsolved problems is the inclusion of discourse knowledge in the generation process. One potential approach is generating a discourse structure derived from generic processing of the underlying domain semantics, transforming this to a structured progression and then using this to steer the choice of hypermedia communicative devices used to convey the actual information in the resulting presentation. This paper presents the results of the first phase of the Topia project, which explored this approach. These results include an architecture for this more domain-independent processing of semantics and discourse into hypermedia presentations. We demonstrate this architecture with an implementation using Web standards and freely available technologies. Finding the Story -- Broader Applicability of Semantics and Discourse for Hypermedia Generation August 23-27 2003 Topia, Discourse, Narrative, Hypermedia, Semantics, Clustering, Concept Lattices, RDF, SMIL 67-76 ht03 Lloyd Rutledge, Martin Alberink, Rogier Brussee, Stanislav Pokraev, William van Dieten, Mettina Veenstra. Finding the Story -- Broader Applicability of Semantics and Discourse for Hypermedia Generation, August 23-27, 2003 yes http://homepages.cwi.nl/$sim$media/conferences/WWW2005/talk.html 1-59593-052-3 NWO/NASH, ToKeN/I2RP, MultimediaN This paper discusses generating document structure from annotated media repositories in a domain-independent manner. This approaches the vision of a universal RDF browser. We start by applying the search-and-browse paradigm established for the WWW to RDF presentation. Furthermore, this paper adds to this paradigm the clustering-based derivation of document structure from search returns, providing simple but domain-independent hypermedia generation from RDF stores. While such generated presentations hardly meet the standards of those written by humans, they provide quick access to media repositories when the needed document has not yet been written. The resulting system allows a user to specify a topic for which it generates a hypermedia document providing guided navigation through virtually any RDF repository. The impact for content providers is that as soon as new media items and their annotations are added to a repository, they become immediately available for automatic integration into subsequently requested presentations. Making RDF Presentable - Integrated Global and Local Semantic Web Browsing May 2005 NWO,BSIK,NAP 8 199-206 www2005 Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman. Making RDF Presentable - Integrated Global and Local Semantic Web Browsing, May, 2005 This is a revised version of CWI technical report INS-E0301 Time-based, media-centric Web presentations can be described declaratively in the XML world through the development of languages such as SMIL. It is difficult, however, to fully integrate them in a complete document transformation processing chain. In order to achieve the desired processing of data-driven, time-based, media-centric presentations, the text-flow based formatting vocabularies used by style languages such as XSL, CSS and DSSSL need to be extended. The paper presents a selection of use cases which are used to derive a list of requirements for a multimedia style and transformation formatting vocabulary. The boundaries of applicability of existing text-based formatting models for media-centric transformations are analyzed. The paper then discusses the advantages and disadvantages of a fully-fledged time-based multimedia formatting model. Finally, the discussion is illustrated by describing the key properties of the example multimedia formatting vocabulary currently implemented in the back-end of our Cuypers multimedia transformation engine. Towards a Formatting Vocabulary for Time-based Hypermedia 384-393 www2003 Jacco van Ossenbruggen, Joost Geurts, Lynda Hardman, Lloyd Rutledge. Towards a Formatting Vocabulary for Time-based Hypermedia Hypermedia structured in terms of the higher-level intent of its author can be adapted to a wider variety of final presentations. Many multimedia systems encode such high-level intent as constraints on either time, spatial layout or navigation. Once specified, these constraints are translated into specific presentations whose timelines, screen displays and navigational structure satisfy these constraints. This ensures that the desired spatial, temporal and navigation properties are maintained no matter how the presentation is adapted to varying circumstances. Rhetorical structure defines author intent at a still higher level. Authoring at this level requires that rhetorics can be translated to final presentations that properly reflect them. This paper explores how rhetorical structure can be translated into constraints, which are then translated into final presentations. This enables authoring in terms of rhetorics and provides the assurance that the rhetorics will remain properly conveyed in all presentation adaptation. Generating Presentation Constraints from Rhetorical Structure 2000 Topia 19-28 ht00 Lloyd Rutledge, Brian Bailey, Jacco van Ossenbruggen, Lynda Hardman, Joost Geurts. Generating Presentation Constraints from Rhetorical Structure, 2000 This is a revised version of CWI technical report INS-R0025 First generation Web-content encodes information in handwritten (HTML) Web pages. Second generation Web content generates HTML pages on demand, e.g. by filling in templates with content retrieved dynamically from a database or transformation of structured documents using style sheets (e.g. XSLT). Third generation Web pages will make use of rich markup (e.g. XML) along with metadata (e.g. RDF) schemes to make the content not only machine readable but also machine processable -- a necessary pre-requisite to the emphSemantic Web. While text-based content on the Web is already rapidly approaching the third generation, multimedia content is still trying to catch up with second generation techniques. Multimedia document processing has a number of fundamentally different requirements from text which make it more difficult to incorporate within the document processing chain. In particular, multimedia transformation uses different document and presentation abstractions, its formatting rules cannot be based on text-flow, it requires feedback from the formatting back-end and is hard to describe in the functional style of current style languages. We state the requirements for second generation processing of multimedia and describe how these have been incorporated in our prototype multimedia document transformation environment, emphCuypers. The system overcomes a number of the restrictions of the text-flow based tool sets by integrating a number of conceptually distinct processing steps in a single runtime execution environment. We describe the need for these different processing steps and describe them in turn (semantic structure, communicative device, qualitative constraints, quantitative constraints, final form presentation), and illustrate our approach by means of an example. We conclude by discussing the models and techniques required for the creation of third generation multimedia content. Towards Second and Third Generation Web-Based Multimedia 479-488 www10 Jacco van Ossenbruggen, Joost Geurts, Frank Cornelissen, Lloyd Rutledge, Lynda Hardman. Towards Second and Third Generation Web-Based Multimedia 51 Ercim News Hypermedia Presentation Generation on the Semantic Web The research goal of the Multimedia and Human-Computer Interaction group at CWI is to investigate the automated generation of Web-based hypermedia presentations tailored to the abilities, preferences and platform of the user. This requires the description and processing of different types of information in order to assemble semantically annotated media items into a coherent presentation, i.e. a presentation that communicates the intended semantic relations to the user. 2002 36 Jacco van Ossenbruggen, Lynda Hardman. Hypermedia Presentation Generation on the Semantic Web. Ercim News 51, 2002 Workshop "Digital Storytelling" DISTEL 2000 Arbeitsumgebungen fur die Entwicklung interaktiver Geschichten June 15-16 2000 This paper presents an integrated framework for new media production in which particular presentations in a variety of forms may be authored and generated using a range of tools interacting with a distributed media repository. The A4SM project demonstrates a number of elements of this overall framework. The emphasis in this work is on the provision of tools and technologies for the manual authorship of linear and interactive media productions. Semi-automated and interactive assembly of a variety of forms of production have been shown to be feasible by a number of research projects, and can be realised by tools interacting with this common repository and environment. ZGDV Darmstadt, Germany Frank Nack, Craig Lindley. Arbeitsumgebungen fur die Entwicklung interaktiver Geschichten. In: Workshop "Digital Storytelling" DISTEL 2000, June 15-16, 2000 Amsterdam CWI Internal Communication Multimedia Synchronization and Unix 1992 Dick C.A. Bulterman, Robert van Liere. Multimedia Synchronization and Unix. CWI Internal Communication, 1992 ACM Digital Libraries (DL'98) Practical Application of Existing Hypermedia Standards and Tools June 1998 191-199 Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman, Dick C.A. Bulterman. Practical Application of Existing Hypermedia Standards and Tools. In: ACM Digital Libraries (DL'98), June, 1998 Digital content creation A video annotation methodology for interactive video sequence generation The FRAMES project within the RDN CRC (Cooperative Research Centre for Research Data Networks) has developed an experimental environment for dynamic virtual video sequence synthesis from databases of video data. A major issue for the development of dynamic interactive video applications of this type is that of how to model and design a video annotation space in order to achieve specific design goals. Consideration of the behavior of the FRAMES algorithm for sequence synthesis, together with the structure of the video annotation space, can provide very clear guidelines for the design of the annotation space, and hence the nature of the video components needed. The resulting design methodology for interactive video is simple to use, and represents a straightforward extension of conventional pre-production activities in standard video production. This paper presents an analysis of the associative chaining algorithm as a search process, together with a systematic video semantic annotation methodology based upon that analysis that can be used to optimise possibilities for user interaction or to achieve specific aesthetic effects. The methodology can be used to create predictable video synthesis behavior, and provides a basis for interactive video production throughout the production lifecycle. 2001 163-183 New York, NY, USA Craig A. Lindley. A video annotation methodology for interactive video sequence generation. In: Digital content creation, Springer-Verlag New York, Inc., 2001 42 Bruno Bachimont, editor Documentaliste - Sciences de l'information Nouveaux outils et documents audiovisuels : les innovations du web semantique 6 2005 392-404 Raphael Troncy. Nouveaux outils et documents audiovisuels : les innovations du web semantique. Documentaliste - Sciences de l'information 42(6), 2005 IEEE Internet Computing SMIL 2.0: XML For Web Multimedia On 7 August, the World Wide Web Consortium (W3C) released version 2.0 of Synchronized Multimedia Integration Language, or SMIL (pronounced smile). Three years ago, SMIL 1.0 introduced a basic foundation for Web multimedia and, as the sidebar SMIL support describes, it quickly gained widespread use. With a specification document about 15 times as large as version 1.0, SMIL 2.0 builds on this foundation and marks an enormous step forward in multimedia functionality. SMIL's features fall into five categories: media content, layout, timing, linking, and adaptivity. The latter brings altogether new features to the Web, letting authors adapt content to different market groups, user abilities, system configurations, and run-time system delays. In this tutorial, I ll cover each feature category and its basic constructs using a simple SMIL presentation built with the SMIL 2.0 Language Profile, which is the flagship SMIL-defined language for multimedia browsers. September-October 2001 78-84 Lloyd Rutledge. SMIL 2.0: XML For Web Multimedia. IEEE Internet Computing, September-October, 2001 yes 2 IJWET Special Issue on Semantic Web Technologies for Data Integration and Multimedia Presentation. Flavius Frasincar, Geert-Jan Houben and Jacco van Ossenbruggen (guest editors). Article is a revised version of http://ftp.cwi.nl/CWIreports/INS//INS-E0303.pdf NWO NASH International Journal of Web Engineering and Technology (IJWET) Combining RDF Semantics with XML Document Transformations 2/3 December 2005 248-263 Jacco van Ossenbruggen, Lynda Hardman, Lloyd Rutledge. Combining RDF Semantics with XML Document Transformations. International Journal of Web Engineering and Technology (IJWET) 2(2/3), December, 2005 ACM Computing Surveys Hypermedia: The Link with Time December 1999 This essay presents a brief discussion of combining temporal aspects of multimedia presentations with hypertext links. Three ways of combining linking with temporally synchronized components of a presentation are described. We describe work that has been done to incorporate both temporal and linking information within the W3C language SMIL (Synchronized Multimedia Integration Language). We conclude with a discussion of future directions, namely providing support for linking within and among non-linear presentations and the ability to add temporal information to existing XML document languages. Lynda Hardman, Jacco van Ossenbruggen, Lloyd Rutledge, Dick C.A. Bulterman. Hypermedia: The Link with Time. ACM Computing Surveys, December, 1999 Proceedings of the 2nd Italian Semantic Web Applications and Perspectives Workshop (SWAP 05) oMAP: An Implemented Framework for Automatically Aligning OWL Ontologies December 14-16 2005 Trento, Italy Umberto Straccia, Raphael Troncy. oMAP: An Implemented Framework for Automatically Aligning OWL Ontologies. In: Proceedings of the 2nd Italian Semantic Web Applications and Perspectives Workshop (SWAP 05), December 14-16, 2005 11 Multimedia Systems Context aware guidance for multimedia authoring: harmonizing domain and discourse knowledge 3 This paper presents an approach to assist authors during the authoring of multimedia presentations. We extend the existing authoring support by integrating processes of topic identification, content collection and discourse structure building in a single environment. This integration allows identification of the context of the authoring process. Our approach combines this process context awareness with explicit domain and discourse knowledge to steer system suggestions. We evaluate our approach with an experimental system prototype. 2006 226-235 Kateryna Falkovych, Frank Nack. Context aware guidance for multimedia authoring: harmonizing domain and discourse knowledge. Multimedia Systems 11(3), 2006 Poster presentation The 2nd European Workshop on the Integration of Knowledge, Semantic and Digital Media Technologies (EWIMT) Semantic Timeline Interfaces for Annotated Multimedia Assets November 30 - December 1 2005 Jacco van Ossenbruggen, Lynda Hardman. Semantic Timeline Interfaces for Annotated Multimedia Assets. In: The 2nd European Workshop on the Integration of Knowledge, Semantic and Digital Media Technologies (EWIMT), November 30 - December 1, 2005 In earlier work, we have shown how clustering techniques can transform semantic markup into discourse, which style sheets can transform into hypermedia presentations. This paper discusses the different clustering techniques for deriving discourse from semantics. These techniques use properties, relations and numerical properties of information objects and semantic networks. We show how these clustering techniques contribute to generating the hierarchical and sequential components of discourse. Clustering Semantics for Discourse Generation August 30 2003 The Hypermedia and the Semantic Web Workshop at the Fourteenth Conference on Hypertext and Hypermedia (HyperText 2003) Martin Alberink, Lloyd Rutledge, Mettina Veenstra. Clustering Semantics for Discourse Generation, August 30, 2003 Dutch research project ToKeN2000 ("Toegankelijkheid en Kennisontsluiting") Jaap van den Herik et al.. ToKeN2000 ("Toegankelijkheid en Kennisontsluiting") Media Computing - Computational Media Aesthetics The Future of Media Computing - From Ontology-based Semiotics to Computational Intelligence 2002 159-196 Boston, Dordrecht, London F. Nack. The Future of Media Computing - From Ontology-based Semiotics to Computational Intelligence. In: Media Computing - Computational Media Aesthetics, Kluwer Academic Publishers, 2002 http://www.hiraeth.com/conf/ActiveWeb/ The Active Web - A British HCI Group Day Conference Adaptable Hypermedia with Web Standards and Tools January 20 1999 London Lloyd Rutledge, Lynda Hardman, Jacco van Ossenbruggen, Dick C.A. Bulterman. Adaptable Hypermedia with Web Standards and Tools. In: The Active Web - A British HCI Group Day Conference, January 20, 1999 work in progress "Canonical Process of Media Production", http://www.cwi.nl/$sim$media/project/canonical/ 2006 - work in progress Lynda Hardman, Frank Nack, Zeljko Obrenovic, Brigitte Kerherve, Kurt Piersol. "Canonical Process of Media Production", http://www.cwi.nl/$sim$media/project/canonical/, 2006 - work in progress Workshop on Narrative, Musical, Cinematic and Gaming Hyperstructure Generating Media Stories - Play It Again, Sam September 2005 New types of knowledge spaces, such as the Semantic Web, allow for yet uncharted forms of knowledge exploration and social relationships. Such interactive, open and multimodal environments sustain the activation of articulation expressions that form the basis of adaptive discourses. In this paper we are in particular interested what story generation requires, in such a context, from authors and how that reflects on the accessibility of the information to users. The examples are taken form the domain of documentary making. Salzburg, Austria Frank Nack, Stefano Bocconi. Generating Media Stories - Play It Again, Sam. In: Workshop on Narrative, Musical, Cinematic and Gaming Hyperstructure, September, 2005 Special session on Advances in semantic multimedia analysis for personalised content access IEEE International Symposium on Circuits and Systems Creating Meaningful Multimedia Presentations May 2006 Lynda Hardman, Jacco van Ossenbruggen. Creating Meaningful Multimedia Presentations. In: IEEE International Symposium on Circuits and Systems, May, 2006 Many Semantic Web applications provide access to their resources through text-based search queries, using explicit semantics to improve the search results. This paper provides an analysis of the current state of the art in semantic search, based on 35 existing systems. We identify different types of semantic search features that are used during query construction, the core search process, the presentation of the search results and user feedback on query and results. For each of these, we consider the functionality that the system provides and how this is made available through the user interface. An analysis of search-based user interaction on the Semantic Web INS-E0706 July 2007 Semantic Search, User Interaction, System Analysis, Evaluation Michiel Hildebrand, Jacco van Ossenbruggen, Lynda Hardman. An analysis of search-based user interaction on the Semantic Web, number INS-E0706, July, 2007 7 New Review of Hypermedia and Multimedia Denotative and Connotative Semantics in Hypermedia: Proposal for a Semiotic-Aware Architecture In this article we claim that the linguistic-centered view within hypermedia systems needs refinement through a semiotic-based approach before real interoperation between media can be achieved. We discuss the problems of visual signification for images and video in dynamic systems, in which users can access visual material in a non-linear fashion. We describe how semiotics can help overcome such problems, by allowing descriptions of the material on both denotative and connotative levels. Finally we propose an architecture for a dynamic semiotic-aware hypermedia system. 2001 7-37 Frank Nack, Lynda Hardman. Denotative and Connotative Semantics in Hypermedia: Proposal for a Semiotic-Aware Architecture. New Review of Hypermedia and Multimedia 7, 2001 SMIL 2.0 - Interactive Multimedia for Web and Mobile Devices 2004 Dick C.A. Bulterman, Lloyd Rutledge. SMIL 2.0 - Interactive Multimedia for Web and Mobile Devices, Springer, 2004 Poster Paper Proceedings of ACM Multimedia Evaluating SMIL: Three User Case Studies November 1999 This paper presents three user case studies of the multimedia standard SMIL. Each conveys a different kind of typical multimedia. The studies illustrate how SMIL can be used for these forms of multimedia. Analysis of these studies also show potential areas for extension of the language to better suit the needs of Web-based multimedia. Orlando, Florida, USA Lloyd Rutledge, Lynda Hardman, Jacco van Ossenbruggen. Evaluating SMIL: Three User Case Studies. In: Proceedings of ACM Multimedia, November, 1999 Handbook of Multimedia Information Management Document Model Issues for Hypermedia 1997 39 - 68 Lynda Hardman, Dick C.A. Bulterman. Document Model Issues for Hypermedia. In: Handbook of Multimedia Information Management, Prentice Hall, 1997 IEEE MultiMedia Play the Game This column is about the advances in game technology and how they might influence the research within multimedia. January - March 2001 8-10 Frank Nack. Play the Game. IEEE MultiMedia, January - March, 2001 1 Hypermedia Journal Evaluating the Usability of the Glasgow Online Hypertext 1 Although there are many hypertext systems currently on the market there is little advice available on how authors can create easy-to-use hypertexts. This paper addresses the usability of Glasgow Online -- a hypertext which contains tourist information on the city of Glasgow. The issues raised in the paper are based on an observational study in which a number of readers performed tasks representative of those undertaken by tourists visiting Glasgow. The usability issues raised by Glasgow Online are discussed in a wider context and their applicability to other styles of hypertext is considered. 1989 34-63 Lynda Hardman. Evaluating the Usability of the Glasgow Online Hypertext. Hypermedia Journal 1(1), 1989 1000 Computer Science Today: recent trends and developments Multimedia Authoring Tools: State of the Art and Research Challenges The integration of audio, video, graphics and text on the desktop promises to fundamentally challenge the centuries-old model of the printed document as the basis for information exchange. Before this potential can be realized, however, systems must be devised that enable the production and presentation of complex, inter-related media objects. These systems are generically called multimedia authoring tools. In this article, we consider the development of multimedia authoring tools, examine the current state of the art, and then discuss a set of research challenges that need to be addressed before the full potential of multimedia output technology can be effectively utilized to share information. Lecture Notes in Computer Science 1995 575-591 Dick C.A. Bulterman, Lynda Hardman. Multimedia Authoring Tools: State of the Art and Research Challenges 1000. In: Computer Science Today: recent trends and developments, Springer-Verlag, 1995 Visual Information Systems System Design for Structured Hypermedia Generation LNCS 1306 Springer-Verlag Lecture Notes in Computer Science 1997 Marcel Worring, Carel van den Berg, Lynda Hardman, Audrey Tam. System Design for Structured Hypermedia Generation. In: Visual Information Systems, number LNCS 1306, Springer Berlin / Heidelberg, 1997 University of Amsterdam Automatic multimedia presentation generation is applicable in a wide variety of circumstances because of its ability to adapt to different presentation contexts such as hardware platforms, user expertise and user interest. The process of generating a multimedia presentation takes a semantic description of the content and transforms this through a number of steps into a multimedia presentation tailored to the requirements of the individual user. This is a complex process which requires a system able to make trade-offs between the different presentation dimensions. This report shows that the use of constraints and constraint logic programming can be beneficial to the process of automatic presentation generation because of its ability to integrate different conceptual layers into one single execution environment. This approach enables an efficient revision of earlier made choices at any stage in the process. We introduce two types of constraints -- quantitative and qualitative -- and discuss what type of constraint can be beneficial at what stage. Constraints for Multimedia Presentation Generation 2002 Multimedia, Constraints, CLP, CHR, Cuypers, Topia Joost Geurts. Constraints for Multimedia Presentation Generation, 2002 In previous work, we proposed a unified approach for describing multimodal human-computer interaction and interaction constraints in terms of sensual, motor, perceptual and cognitive functions of users. In this paper, we extend this work by providing formalised vocabularies that express human functionalities and anatomical structures required by specific modalities. The central theme of our approach is to connect these modality representations with descriptions of user, device and environmental constraints that influence the interaction. These descriptions can then be used in a reasoning framework that will exploit formal connections among interaction modalities and constraints. The focus of this paper is on specifying a comprehensive vocabulary of necessary concepts. Within the context of an interaction framework, we describe a number of examples that use this formalised knowledge. Vocabularies for description of accessibility issues in multimodal user interfaces INS-E0702 2007 Multimodal interaction, universal accessibility, inclusive design, formal models Zeljko Obrenovic, Raphael Troncy, Lynda Hardman. Vocabularies for description of accessibility issues in multimodal user interfaces, number INS-E0702, 2007 A web browser provides a uniform user interface to different types of information. Making this interface universally accessible and more interactive is a long term goal still far from being achieved. Universally accessible browsers require novel interaction modalities and additional functionalities, for which existing browsers tend to provide only partial solutions. Although functionality for web accessibility can be found as open-source and free software components, their reuse and integration is complex because they were developed in diverse implementation environments, following standards and conventions incompatible with the web. To enable the integration of existing partial solutions within a mainstream web browser environment, we have developed a middleware infrastructure, AMICO:WEB. This enables browser access to a wide variety of open source and free software components. The main contribution of AMICO:WEB is in enabling the syntactic interoperability between web extension mechanisms and a variety of integration mechanisms used by open-source and free software components. It also bridges the semantic differences between the high-level world of web XML-based APIs and the low-level APIs of the device-oriented world. We discuss the design decisions made during the development of AMICO:WEB in the context of web accessibility, using two typical usage scenarios: one describing a disabled user using a mainstream web browser with additional interaction modalities; another describing a non-disabled user browsing in a suboptimal interaction situation. Integrating heterogeneous open-source software into web browsers using AMICO:WEB INS-E0703 2007 Web Accessibility, User Interfaces, Open Source Software, Software Platform, Middleware Zeljko Obrenovic, Jacco van Ossenbruggen. Integrating heterogeneous open-source software into web browsers using AMICO:WEB, number INS-E0703, 2007 MPEG-7 can be used to create complex and comprehensive metadata descriptions of multimedia content. Since MPEG-7 is defined in terms of an XML schema, the semantics of its elements has no formal grounding. In addition, certain features can be described in multiple ways. MPEG-7 profiles are subsets of the standard that apply to specific application areas and that aim to reduce this syntactic variability, but they still lack formal semantics. We propose an approach for expressing the semantics explicitly by formalizing the constraints of various profiles using ontologies and logical rules, thus enabling interoperability and automatic use for MPEG-7 based applications. We have implemented VAMP, a full semantic validation service that detects any inconsistencies of the semantic constraints formalized. Another contribution of this paper is an analysis of how MPEG-7 is practically used. We report on experiments about the semantic validity of MPEG-7 descriptions produced by numerous tools and projects and we categorize the most common errors found. VAMP: Semantic Validation for MPEG-7 Profile Descriptions INS-E0705 April 2007 VAMP, MPEG-7 Validation Service, MPEG-7 Semantic Constraints Raphael Troncy, Werner Bailer, Michael Hausenblas, Martin Hoffernig. VAMP: Semantic Validation for MPEG-7 Profile Descriptions, number INS-E0705, April, 2007 This is a revised version of CWI technical report INS-R0307 Conferentie Informatiewetenschap 2003 Sequence and Emphasis in Automated Domain-Independent Discourse Generation For humans to gain comprehensive views of large amounts of repository contents, they need to have insight into the relations among information objects. It is a challenge to automatically generate presentations of repository contents, through, for example, search results, which reveal such relations to readers. Such presentations must reflect properties of information objects such that large sets of information objects appear as a coherent whole. An approach to this is generation of discourse structures that convey such properties of information objects in presentations. Semantic Web technology provides a conceptual basis for generation of discourse in Web-based information environments. This paper describes automatic generation of sequence and emphasis in presentations of information objects. It shows generation of object sequences and emphasis in accordance with a user input of relevance of information attributes in our Topia architecture. The resulting presentations allow users to encounter information objects in decreasing order of relevance. This makes it easier to identify relevant information objects among many others, as well as to observe their relations with the other information objects. November 20 2003 3-18 Eindhoven, The Netherlands Martin Alberink, Lloyd Rutledge, Mettina Veenstra. Sequence and Emphasis in Automated Domain-Independent Discourse Generation. In: Conferentie Informatiewetenschap 2003, November 20, 2003 Proceedings of the ACM Workshop on Multimedia for Human Communication - From Capture to Convey (MHC 05) Capture and Transfer of Metadata During Video Production November 2005 Video production is a complex process that requires the capture and annotation of video sequences that become assembled together into a final presentation. At different stages of the process information is captured and associated with the video sequence, to be used by some other process at some later stage. In this paper we go through the video production chain step by step and identify where media or metadata is captured and express this in terms of an associated proposed canonical model. Frank Nack. Capture and Transfer of Metadata During Video Production. In: Proceedings of the ACM Workshop on Multimedia for Human Communication - From Capture to Convey (MHC 05), November, 2005 In this article we claim that the linguistic-centered view within hypermedia systems needs refinement through a semiotic-based approach before real interoperation between media can be achieved. We discuss the problems of visual signification for images and video in dynamic systems, in which users can access visual material in a non-linear fashion. We describe how semiotics can help overcome such problems, by allowing descriptions of the material on both denotative and connotative levels. Finally we propose an architecture for a dynamic semiotic-aware hypermedia system. Denotative and Connotative Semantics in Hypermedia: Proposal for a Semiotic-Aware Architecture INS-R0202 March 2002 Hypermedia, hypertext, multimedia, denotation, connotation, semiotics, ontology, conceptual information space Frank Nack, Lynda Hardman. Denotative and Connotative Semantics in Hypermedia: Proposal for a Semiotic-Aware Architecture, number INS-R0202, March, 2002 Edited by Michael Hausenblas, Susanne Boll, Tobias Burger, Oscar Celma, Christian Halaschek-Wiener, Erik Mannens and Raphael Troncy This document gives an overview on the state-of-the-art of multimedia metadata formats. Initially, practical relevant vocabularies for developers of Semantic Web applications are listed according to their modality scope. In the second part of this document, the focus is set on the integration of the multimedia vocabularies into the Semantic Web, that is to say, formal representations of the vocabularies are discussed. Multimedia Vocabularies on the Semantic Web July 24 2007 Multimedia Semantics Incubator Group Report (XGR) W3C. Multimedia Vocabularies on the Semantic Web, July 24, 2007 Semantic Design 2006 Poster presented at MultimediaN Day, June 2006 Alia Amin, Lynda Hardman, Jacco van Ossenbruggen. Semantic Design, 2006 Web publishing systems have to take into account a plethora of Web-enabled devices, user preferences and abilities. Technologies generating these presentations will need to be explicitly aware of the context in which the information is being presented. Semantic Web technology can be a fundamental part of the solution to this problem by explicitly modeling the knowledge needed to adapt presentations to a specific delivery context. We propose the development of a emphSmart Stylelayer which is able to process metadata that describes content and use this metadata to improve the presentation of the content to human users. In the paper, we derive the requirements of such a Smart Style layer by considering Web design from both the document engineering and graphic design perspectives. In addition, design trade-offs made by human designers have to be taken into account for the automated process. After stating the requirements for a Smart Style layer, we discuss to what extent the currently available Web technology can be used and what its limitations are. The limitations are illustrated with examples of potential future extensions. Smart Style on the Semantic Web INS-R0201 February 2002 Semantic Web, Device Independent Authoring, Document Engineering, Graphic Design Jacco van Ossenbruggen, Lynda Hardman. Smart Style on the Semantic Web, number INS-R0201, February, 2002 This article describes the current state of the art on representing the four essential conceptual facets of a multimedia unit, namely the form and substance of content and the form and substance of its expression, and points to the still unsolved problems regarding the syntax for media semantics. We first provide a brief overview of the general features of the MPEG-7 standard and its different parts. This serves as a description of the state of the art in content description for audio-visual media. We then analyse the ability of one of these parts for its capability to define structures for describing media semantics. We describe the problems of two currently conflicting MPEG-7 representations of expression-based media semantic, which should be equivalent. We then discuss high-level aspects of media semantics, namely the general problems of an ontology for media semantics. Finally, we talk about the problems of applying the theoretical concepts to real applications. Towards a Syntax for Multimedia Semantics INS-R0204 April 2002 Multimedia semantics, MPEG-7, DDL, MDS, Semantic Web, XML-Schema, information spaces Frank Nack, Lynda Hardman. Towards a Syntax for Multimedia Semantics, number INS-R0204, April, 2002 Edited by Oscar Celma, Stamatia Dasiopoulou, Michael Hausenblas, Suzanne Little, Chrisa Tsinaraki and Raphael Troncy. MPEG-7 and the Semantic Web August 14 2007 Multimedia Semantics Incubator Group Report Editor's Draft W3C. MPEG-7 and the Semantic Web, August 14, 2007 no ToKeN ISSN 1386-3681 The availability of semantically enriched data in repositories of larger content providers offers a means for new ways of multimedia presentation authoring. Existing (semi-)automatic content composition environments explore limited numbers of genres and concentrate on templatebased approaches for composing discourse structures for these genres. We analyze a number of genres based on identifiable genre characteristics. We show that existing template-based approaches to content composition support the essay and biography genres that belong to one characteristic genre category but fail in supporting genres of another category. We present our approach to overcome this limitation and apply this approach to composing newspaper articles in the domain of fine arts. Composing Discourse based on Genre Semantics INS-E0516 December 2005 Genre, Content composition, Discourse structure, News article composition NWO CHIME Kateryna Falkovych, Frank Nack. Composing Discourse based on Genre Semantics, number INS-E0516, December, 2005 The 8th IFIP 2.6 Working Conference on Database Semantics (DS-8): Semantic Issues in Multimedia Systems Supporting Adaptive and Adaptable Hypermedia Presentation Semantics January 1999 adaptive hypermedia, adaptable hypermedia, CMIF, SMIL, HyTime, DSSSL, SRM Having the content of a presentation adapt to the needs, resources and prior activities of a user can be an important benefit of electronic documents. While part of this adaptation is related to the encodings of individual data streams, much of the adaptation can/should be guided by the semantics in and among the objects of the presentation. The semantics involved in having hypermedia presentations adapt can be divided between adaptive hypermedia, which adapts autonomously, and adaptable hypermedia, which requires presentationexternal intervention to be adapted. Understanding adaptive and adaptable hypermedia and the differences between them helps in determining the best manner with which to have a particular hypermedia implementation adapt to the varying circumstances of its presentation. The choice of which type of semantics to represent can affect speed of the database management system processing them. This paper reflects on research and implementation approaches toward both adaptive and adaptable hypermedia and how they apply to specifying the semantics involved in hypermedia authoring and processing. We look at adaptive approaches by considering CMIF and SMIL. The adaptable approaches are represented by the SGML-related collection of formats and the Standard Reference Model (SRM) for IPMS are also reviewed. Based on our experience with both adaptive and adaptable hypermedia, we offer recommendations on how each approach can be supported at the data storage level. Rotorua, New Zealand, 5-8 January 1999 Dick C.A. Bulterman, Lloyd Rutledge, Lynda Hardman, Jacco van Ossenbruggen. Supporting Adaptive and Adaptable Hypermedia Presentation Semantics. In: The 8th IFIP 2.6 Working Conference on Database Semantics (DS-8): Semantic Issues in Multimedia Systems, January, 1999 Domain model enhanced search - A comparison of taxonomy, thesaurus and ontology 2005 Master Thesis Content and Knowledge Engineering, University of Utrecht Katharina Schwarz. Domain model enhanced search - A comparison of taxonomy, thesaurus and ontology, 2005 edited by Frank Nack IEEE MultiMedia Multimedia Standards: Building Blocks of the Web You've probably heard this joke before: The nice thing about standards is that there are so many to choose from. It used to mean that for any one task, there were many standards from which to choose. However, these days in multimedia, the joke takes on a whole new meaning there are many standards for composing multimedia on the Web, but now rather than choosing one, you use all of them together. July-September 2001 13-15 Lloyd Rutledge. Multimedia Standards: Building Blocks of the Web. IEEE MultiMedia, July-September, 2001 Proceedings 4th International Conference on the Electronic Document A flexible authoring process for collaborative authoring based on document fragments and contracts October 24-26 2001 Many research work on collaborative authoring focuses on providing an adequate framework for the authoring process. In this article, we propose document farms as a simple computer infrastructure from which it is possible to derive various authoring processes. We follow a structured document approach, with a document model completed by fragment and contract definitions. The result is a collaborative authoring process which can be constrained during the document class definition, or during the document instantiation. The general architecture of a document farm is described, as well as interfaces that promote more intimate interaction with the document structure. Toulouse, France Stephane Sire, Yassine Rekik, Christine Vanoirbeek. A flexible authoring process for collaborative authoring based on document fragments and contracts. In: Proceedings 4th International Conference on the Electronic Document, October 24-26, 2001 Based on INS-E0502 tech report. 11 Multimedia Systems Journal, Special issue on Multimedia System Technologies for Educational Tools, S. Acton, F. Kishino, R. Nakatsu, M. Rauterberg & J. Tang eds. Context Aware Guidance for Multimedia Authoring: Harmonizing Domain and Discourse Knowledge 3 This paper presents an approach to assist authors during the authoring of multimedia presentations. We extend existing authoring support by integrating processes of topic identification, content collection and discourse structure building in a single environment. This integration allows identification of the context of the authoring process. Our approach combines this process context awareness with explicit domain and discourse knowledge to steer system suggestions. We evaluate our approach with an experimental system prototype. 2006 226-235 Kateryna Falkovych, Frank Nack. Context Aware Guidance for Multimedia Authoring: Harmonizing Domain and Discourse Knowledge. Multimedia Systems Journal, Special issue on Multimedia System Technologies for Educational Tools, S. Acton, F. Kishino, R. Nakatsu, M. Rauterberg & J. Tang eds. 11(3), 2006 IEEE MultiMedia Migrating from mobile telephony to multipurpose gadgets This text looks at how mobile telephoning has turned into mobile media-based information exchange and the turmoil this transformation has caused. January - March 2003 08-11 Frank Nack. Migrating from mobile telephony to multipurpose gadgets. IEEE MultiMedia, January - March, 2003 3rd European Conference on Multimedia Applications, Services and Techniques Presenting Multimedia on the Web and in TV broadcast May 26-28 1998 Berlin-Germany Warner R.T. ten Kate, Patrick J. Deunhouwer, Dick C.A. Bulterman, Lynda Hardman, Lloyd Rutledge. Presenting Multimedia on the Web and in TV broadcast. In: 3rd European Conference on Multimedia Applications, Services and Techniques, May 26-28, 1998 Proceedings of the sixteenth ACM Conference on Hypertext and Hypermedia 2005 Supporting the Generation of Argument Structure within Video Sequences Browsing is a useful way of exploring annotated media repositories. Sets of links can be automatically created from the annotations associated with the media items in the repository. When there are also relationships among the annotations themselves, such as when the annotation terms are part of a thesaurus, these relations can also be used in the link generation process. Using structured annotations and a thesaurus for generating link sets has two advantages. The first is to evaluate the effectiveness of the terms in the thesaurus for classifying the media items in the repository. The second is to be able to control the links being generated by changing relationships within the thesaurus. The work is illustrated using video segments annotated with argument structures, but we show that the method used is independent of the media types and applicable to systems that use similar annotation structures and typed relations among them. September 2005 75-84 Stefano Bocconi, Frank Nack, Lynda Hardman. Supporting the Generation of Argument Structure within Video Sequences. In: Proceedings of the sixteenth ACM Conference on Hypertext and Hypermedia 2005, September, 2005 Proceedings of First International Workshop on Intelligence and Multimodality in Multimedia Interfaces Towards the Generation of Hypermedia Structure July 1995 Edinburgh, UK Lynda Hardman, Dick C.A. Bulterman. Towards the Generation of Hypermedia Structure. In: Proceedings of First International Workshop on Intelligence and Multimodality in Multimedia Interfaces, July, 1995 6 edited by Lynda Hardman New Review of Hypermedia and Multimedia The Link vs. the Event: Activating and Deactivating Elements in Time-Based Hypermedia Activation and deactivation of media items plays a fundamental role in the playing of multimedia and time-based hypermedia presentations. Activation and deactivation information thus has to be captured in an underlying document format. In this paper we show that a number of aspects of activation and deactivation information can be captured using both link structures and events in time-based hypermedia. In particular, we discuss how deactivation and activation can be specified, how the activations and deactivations can be initiated and potential (synchronization) relationships between the elements involved. We first introduce the notions of time-based scheduling and event-based scheduling and then present a short summary of linking. We discuss the similarities between event-based scheduling and linking. We describe a number of aspects of activation and deactivation that can be specified within a document. We then discuss how activation and deactivation information can be recorded in link structures and events. 2000 89-109 Lynda Hardman, Patrick Schmitz, Jacco van Ossenbruggen, Warner R.T. ten Kate, Lloyd Rutledge. The Link vs. the Event: Activating and Deactivating Elements in Time-Based Hypermedia. New Review of Hypermedia and Multimedia 6, 2000 22 Multimedia Tools and Applications Saying What It Means: Semi-Automated (News) Media Anotation 3 This paper considers the automated and semi-automated annotation of audiovisual media in a new type of production framework, A4SM (Authoring System for Syntactic, Semantic and Semiotic Modelling). We present the architecture of the framework, describe a prototypical camera, a handheld device for basic semantic annotation, and an editing suite to demonstrate how video material can be annotated in real time and how this information can not only be used for retrieval but also can be used during the different phases of the production process itself. We then outline the underlying XML Schema based content description structures of A4SM and discuss the pros and cons of our approach of evolving semantic networks as the basis for audio-visual content description. March 2004 263 - 302 Frank Nack, Wolfgang Putz. Saying What It Means: Semi-Automated (News) Media Anotation. Multimedia Tools and Applications 22(3), March, 2004 Edited by Vassilis Tzouvaras, Raphael Troncy and Jeff Z. Pan. Multimedia systems typically contain digital documents of mixed media types, which are indexed on the basis of strongly divergent metadata standards. This severely hamplers the inter-operation of such systems. Therefore, machine understanding of metadata comming from different applications is a basic requirement for the inter-operation of distributed Multimedia systems. In this document, we present how interoperability among metadata, vocabularies/ontologies and services is enhanced using Semantic Web technologies. In addition, it provides guidelines for semantic interoperability, illustrated by use cases. Finally, it presents an overview of the most commonly used metadata standards and tools, and provides the general research direction for semantic interoperability using Semantic Web technologies. Multimedia Annotation Interoperability Framework August 14 2007 Multimedia Semantics Incubator Group Report Editor's Draft W3C. Multimedia Annotation Interoperability Framework, August 14, 2007 Quality of Service (QoS) protocols and architectures play an important role in distributed multimedia systems since multimedia presentations need to be played for the user without stalls, necessitating guaranteed response times. However in the Internet no other QoS than best-effort is given. This report investigates some QoS architectures and protocols designed to overcome these limitations and looks at the applications of these protocols for multimedia systems. Given the recent research on (semi-)automatically generated multimedia presentations, this report investigates the opportunities to gear the generated multimedia presentations toward the specific network conditions of the client requesting a document, and dynamically adapting the presentation as network conditions change. Network quality of service for multimedia presentation generation systems INS-R0106 May 2001 Multimedia, QoS, network protocols, presentation generation Frank Cornelissen. Network quality of service for multimedia presentation generation systems, number INS-R0106, May, 2001 Proceedings of the International Conference on Multimedia Modeling 2001 (MMM01) Improving Media Fragment Integration in Emerging Web Formats The media components integrated into multimedia presentations are typically entire files. At times the media component desired for integration, either as a navigation destination or as coordinate presentation, is a part of a file, or what we call a fragment. Basic media fragment integration has long been implemented in hypermedia systems, but not to the degree envisioned by hypermedia research. The current emergence of several XML-based formats is beginning to extend the possibilities for media fragment integration on a large scale. This paper presents a set of requirements for media fragment integration, describes how standards currently meet some of these requirements and proposes extensions to these standards for meeting remaining requirements. November 5-7 2001 147-166 CWI, Amsterdam, The Netherlands Lloyd Rutledge, Patrick Schmitz. Improving Media Fragment Integration in Emerging Web Formats. In: Proceedings of the International Conference on Multimedia Modeling 2001 (MMM01), November 5-7, 2001 0-521-36553-8 People and Computers IV, Proceedings of Fourth Conference of the British Computer Society Human-Computer Interaction Specialist Group Hypertext Tips: Experiences in Developing a Hypertext Tutorial September 1988 437-451 Manchester, UK Lynda Hardman. Hypertext Tips: Experiences in Developing a Hypertext Tutorial. In: People and Computers IV, Proceedings of Fourth Conference of the British Computer Society Human-Computer Interaction Specialist Group, September, 1988 This is a revised version of CWI Report INS-R0107. Proceedings of the International Conference on Multimedia Modeling 2001 (MMM01) Application-Specific Constraints for Multimedia Presentation Generation The paper describes the advantages of the use of constraint logic programming to articulate transformation rules for multimedia presentation in combination with efficient constraint solving techniques. It demonstrates the need for two different types of constraints. Quantitative constraints are needed to verify whether the final form presentation meets all the numeric constraints that are required by the environment. Qualitative constraints are needed to facilitate high-level reasoning and presentation encoding. While the quantitative constraints can be handled by off-the-shelf constraint solvers, the qualitative constraints needed are specific to the multimedia domain and need to be defined explicitly. November 5-7 2001 Multimedia, Constraints, CLP, CHR, Cuypers 247-266 CWI, Amsterdam, The Netherlands Joost Geurts, Jacco van Ossenbruggen, Lynda Hardman. Application-Specific Constraints for Multimedia Presentation Generation. In: Proceedings of the International Conference on Multimedia Modeling 2001 (MMM01), November 5-7, 2001 MultimediaN E-Culture demonstrator: Objectives and Architecture 2006 (note: to be published) Alia Amin, Mark van Assem, Viktor de Boer, Lynda Hardman, Michiel Hildebrand, Laura Hollink, Zhisheng Huang, Janneke van Kersen, Marco de Niet, Borys Omelayenjko, Jacco van Ossenbruggen, Guus Schreiber, Ronny Siebes, Jos Taekema, Jan Wielemaker, Bob Wielinga. MultimediaN E-Culture demonstrator: Objectives and Architecture, 2006 (note: to be published) Proceedings of ACM Multimedia User-Centered Abstractions for Adaptive Hypermedia Presentations November 1998 145-150 Dick C.A. Bulterman. User-Centered Abstractions for Adaptive Hypermedia Presentations. In: Proceedings of ACM Multimedia, ACM Press, November, 1998 Proceedings ACM MM 2000 Workshops - Bridging the Gap: Bringing Together New Media Artists and Multimedia Technologists Production and maintenance environments for interactive audio-visual stories October 31 2000 21-24 Los Angeles, CA. Frank Nack, Craig Lindley. Production and maintenance environments for interactive audio-visual stories. In: Proceedings ACM MM 2000 Workshops - Bridging the Gap: Bringing Together New Media Artists and Multimedia Technologists, October 31, 2000 Proceedings of ACM Multimedia Structural Distinctions Between Hypermedia Storage and Presentation In order to facilitate adaptability of hypermedia documents a distinction is often made between the underlying conceptual structure of a document and the structure of its presentation. This distinction enables greater variety in how a presentation can be adapted to best convey these underlying concepts in a given situation. What is often confusing for those applying this distinction is that although both levels of structure often share similar components: transformation form the storage of a document to its presentation sometimes occurs directly between these similar components and sometimes does not. These similarities typically fall in the categories of space, time and relationships between document portions. This paper identifies some primary similarities between the structure of hypermedia storage and presentation. It also explores how the transformation from storage to presentation often does not follow these similarities. This discussion is illustrated with the Fiets hypermedia application, which addresses the issues of storage, presentation and transformation using public domain formats and tools. The intention is to help authors who separate storage from presentation to better understand this distinction. November 1998 Hypermedia, storage, generation, presentation, HyTime, DSSSL, SMIL. 145-150 Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman, Dick C.A. Bulterman. Structural Distinctions Between Hypermedia Storage and Presentation. In: Proceedings of ACM Multimedia, ACM Press, November, 1998 no ToKeN This paper presents an approach to assist authors during the authoring of multimedia presentations. We extend existing authoring support by integrating processes of topic identification, content collection and discourse structure building in a single environment. This integration allows identification of the context of the authoring process. Our approach combines this process context awareness with explicit domain and discourse knowledge to steer system suggestions. We evaluate our approach with an experimental system prototype. Context Aware Guidance for Multimedia Authoring: Harmonizing Domain and Discourse Knowledge INS-E0502 February 2005 Multimedia presentation authoring, semantics, discourse structure, context. NWO CHIME Kateryna Falkovych, Frank Nack. Context Aware Guidance for Multimedia Authoring: Harmonizing Domain and Discourse Knowledge, number INS-E0502, February, 2005 This is a revised version of CWI technical report INS-R0105 3 Journal of Digital Information Hypermedia and the Semantic Web: A Research Agenda 1 August 2002 Until recently, the Semantic Web was little more than a name for the next generation Web infrastructure as envisioned by its inventor, Tim Berners-Lee. Now, with the introduction of XML and RDF, and new developments such as RDF Schema and DAML+OIL, the Semantic Web is rapidly taking shape. In this paper, we first give an overview of the state-of-the-art in Semantic Web technology, the key relationships with traditional hypermedia research, and a comprehensive reference list to various sets of literature (Hypertext, Web and Semantic Web). We then present a research agenda by describing the open research issues in the development of the Semantic Web from the perspective of hypermedia research. Jacco van Ossenbruggen, Lynda Hardman, Lloyd Rutledge. Hypermedia and the Semantic Web: A Research Agenda. Journal of Digital Information 3(1), August, 2002 no ToKeN We use rhetorical annotations to specify a generation process that can assemble meaningful video sequences with a communicative goal and an argumentative progression. Our annotation schema encodes the verbal information contained in the audio channel, identifying the claims the interviewees make and the argumentation structures they use to make those claims. Based on this schema, we construct a semantic graph which is traversed by rhetoric-based strategies selecting video segments. The selected video segments are edited to form a meaningful video sequence. Using rhetorical annotations for generating video documentaries INS-E0501 January 2005 Media semantics, media rhetorics, automated video editing, multimedia presentation generation, video documentaries NAP,NWO I2RP Stefano Bocconi, Frank Nack, Lynda Hardman. Using rhetorical annotations for generating video documentaries, number INS-E0501, January, 2005 Poster Paper Proceedings Supplement of the seventh European Conference on Computer Supported Cooperative Work Collaborative authoring with document fragments and contracts September 18-20 2001 Research on collaborative authoring focuses on providing an adequate soft-ware environment for the authoring process. This poster describes our framework based on a web architecture as well as a modular document model and some interfaces from which it is possible to derive various authoring processes. We apply a structured document approach with a document model completed by fragment and contract definitions. This results in a check-out/check-in type of authoring processes where negotiation can take place during the document class definition, or during the document instantiation. We describe the general architecture of a document farm prototype which supports our framework. Bonn, Germany Stephane Sire. Collaborative authoring with document fragments and contracts. In: Proceedings Supplement of the seventh European Conference on Computer Supported Cooperative Work, September 18-20, 2001 no ToKeN Browsing is a useful way of exploring annotated media repositories. Sets of links can be automatically created from the annotations associated with the media items in the repository. When there are also relationships among the annotations themselves, such as when the annotation terms are part of a thesaurus, these relations can also be used in the link generation process. Using structured annotations and a thesaurus for generating link sets has two advantages. The first is to evaluate the effectiveness of the terms in the thesaurus for classifying the media items in the repository. The second is to be able to control the links being generated by changing relationships within the thesaurus. The work is illustrated using video segments annotated with argument structures, but we show that the method used is independent of the media types and applicable to systems that use similar annotation structures and typed relations among them. Supporting the generation of argument structure within video sequences INS-E0504 March 2005 Hypermedia, Automatic Linking, Structured Annotations, Thesaurus NWO I2RP Stefano Bocconi, Frank Nack, Lynda Hardman. Supporting the generation of argument structure within video sequences, number INS-E0504, March, 2005 Proceedings of the 6th International Conference on Web Information Systems Engineering (WISE 2005) oMAP: Combining Classifiers for Aligning OWL Ontologies This paper introduces a method and a tool for automatically aligning OWL ontologies, a crucial step for achieving the interoperability of heterogeneous systems in the Semantic Web. Different components are combined for finding suitable mapping candidates (together with their weights), and the set of rules with maximum matching probability is selected. Machine learning-based classifiers and a new classifier using the structure and the semantics of the OWL ontologies are proposed. Our method has been implemented and evaluated on an independent test set provided by an international ontology alignment contest. We provide the results of this evaluation with respect to the other competitors. November 20-22 2005 133-147 New York City, USA Umberto Straccia, Raphael Troncy. oMAP: Combining Classifiers for Aligning OWL Ontologies. In: Proceedings of the 6th International Conference on Web Information Systems Engineering (WISE 2005), November 20-22, 2005 37 Communications of the ACM The Amsterdam Hypermedia Model: Adding Time and Context to the Dexter Model 2 From Computing Reviews, by Jeanine Meyer The purpose of this paper is to convince the reader of the need for a general model for hypermedia and present the Amsterdam hypermedia model (AHM) as fulfilling that need. Hypertext, which is described by the authors as a "relatively mature discipline," has the Dexter model, but the authors show that enhancing that model for hypermedia is not a straightforward task. In particular, it requires attention to issues of synchronization. The AHM model includes attention to timing and composite objects as well as implementation issues such as channels and having the sources of components residing over a distributed system. The paper features one example and also describes an authoring and presentation environment called CMIFed. It is generally well written. The paper can be understood even if one has not studied the Dexter hypertext reference model or the CMIF multimedia document model and, in fact, this paper could serve as an introduction to the issues involved. Too much of the focus, however, is on other systems and not on what AHM actually is. The authors do not demonstrate the model by using it to express the featured example. Moreover, to really merit the term "model," AHM should be shown as serving a substantial role in describing and implementing applications in terms of two or more distinct authoring or runtime environments. This is not done, though it appears well within the experience and understanding of the authors. February 1994 50-62 Lynda Hardman, Dick C.A. Bulterman, Guido van Rossum. The Amsterdam Hypermedia Model: Adding Time and Context to the Dexter Model. Communications of the ACM 37(2), February, 1994 Workshop on Narrative, Musical, Cinematic and Gaming Hyperstructure Creating a Semantic-based Discourse Model for Hypermedia Presentations: (Un)discovered Problems September 2005 We address the problem of building a flexible discourse model that enables creation of relatively complex discourse structures by using a semantic framework and semantic descriptions of media items. With such a model we intend to support different approaches to hypermedia presentation authoring. We give a short overview of existing hypermedia presentation generation systems and discuss what type of authoring they support. We present our motivation for working towards a discourse model that improves on existing approaches. We describe in more detail our experiences with various strategies for building discourse structures for DISC and SampLe systems. Based on these experiences we come up with a set of requirements for a common discourse model. Salzburg, Austria Kateryna Falkovych, Stefano Bocconi. Creating a Semantic-based Discourse Model for Hypermedia Presentations: (Un)discovered Problems. In: Workshop on Narrative, Musical, Cinematic and Gaming Hyperstructure, September, 2005 Proceedings of the 3rd European Semantic Web Conference (ESWC 06) Towards Distributed Information Retrieval in the Semantic Web: Query Reformulation Using the oMAP Framework This paper introduces a general methodology for performing distributed search in the Semantic Web. We propose to define this task as a three steps process, namely resource selection, query reformulation/ontology alignment and rank aggregation/data fusion. For the second problem, we have implemented oMAP, a formal framework for automatically aligning OWL ontologies. In oMAP, different components are combined for finding suitable mapping candidates (together with their weights), and the set of rules with maximum matching probability is selected. Among these components, traditional terminological-based classifiers, machine learning-based classifiers and a new classifier using the structure and the semantics of the OWL ontologies are proposed. oMAP has been evaluated on international test sets. June 11-14 2006 378-392 Budva, Montenegro Umberto Straccia, Raphael Troncy. Towards Distributed Information Retrieval in the Semantic Web: Query Reformulation Using the oMAP Framework. In: Proceedings of the 3rd European Semantic Web Conference (ESWC 06), June 11-14, 2006 DECOR : Passage `a l'echelle des techniques de decouverte de correspondances Aligner automatiquement des ontologies avec oMAP January 23 2007 76-82 Namur, Belgium Raphael Troncy. Aligner automatiquement des ontologies avec oMAP. In: DECOR : Passage `a l'echelle des techniques de decouverte de correspondances, January 23, 2007 This is also published as Technical Report INS-E0307. The presentation generation area of hypermedia authoring contains different approaches to address the challenge of the presentation creation process. This paper presents the approach in which the author creates a presentation following the five stages process. These five stages reflect various facets in hypermedia authoring. Semantics in Multi-facet Hypermedia Authoring August 30 Hypermedia Authoring, Semantics, Intelligent Support, Narrative Structures 2003 The Hypermedia and the Semantic Web Workshop at the Fourteenth Conference on Hypertext and Hypermedia (HyperText 2003) Kateryna Falkovych, Frank Nack, Jacco van Ossenbruggen, Lloyd Rutledge. Semantics in Multi-facet Hypermedia Authoring, August 30, 2003 1-58113-931-4 First ACM Workshop on Story Representation, Mechanism and Context VOX POPULI: Automatic Generation of Biased Video Sequences We describe our experimental rhetoric engine Vox Populi that generates biased video-sequences from a repository of video interviews and other related audio-visual web sources. Users are thus able to explore their own opinions on controversial topics covered by the repository. The repository contains interviews with United States residents stating their opinion on the events occurring after the terrorist attack on the United States on the 11th of September 2001. We present a model for biased documentary statements, such as interviews, and explain in detail how this model facilitates the automatic generation of rhetorical arguments on a micro-level. We outline the required representations of relevant rhetorical structures and the way they can be processed. The processes are described via examples generated by our experimental engine. The first example shows how to logically counter an opinion using semantics contained in the audio tracks from the database, while the second example describes the generation of an emotional counterargument using visual material. October 2004 9-16 Stefano Bocconi. VOX POPULI: Automatic Generation of Biased Video Sequences. In: First ACM Workshop on Story Representation, Mechanism and Context, October, 2004 Proceedings International Conference on Media Futures The Rise and Fall of Multimedia Authoring For much the past decade, our research group has focused on developing multimedia authoring tools. We developed a research multimedia format and were involved in developing a multimedia standard. Our lab developed a research prototype of a multimedia editor and then spun off a company that turned it into a commercial editor for this standard. With several research concepts now having reached full maturity in the form of this standard and product, our lab is back where it started: at the beginning of the research idea life cycles, with new visions and hopes for what's next. May 8 2001 17-20 Florence, Italy Lloyd Rutledge, Lynda Hardman. The Rise and Fall of Multimedia Authoring. In: Proceedings International Conference on Media Futures, May 8, 2001 This is also published as CWI technical report INS-E0401 Proceedings of the 1st ACM WS on Experiential Telepresence (ETP 03) Capturing experience - a matter of contextualising events This paper explores the notion of experience in the context of dynamic and interactive environments, such as web-based musea, where neither the individual user requirements nor the requested material can be predicted in advance. A definition of experiences for the particular context is introduced on which the analysis of the what (events), why (context) and the how (presentation) are based. The paper tries to identify the essential aspects of representation for the three main fields of investigation, namely content and expression for the event; goal, task action and role for the context, and the influence of event and context for the presentation. The aim is that the system can find satisfactory solutions for upcoming questions (e.g. based on the content of an image), misunderstandings (rearrangement of the material) or non-understanding (creation of a new sequence). The intent of the paper is to provide a first step towards dynamic and adaptive knowledge structures that facilitate conceptual presentations. November, 7th, 2003 2003 53 - 64 Berkeley, California, USA Frank Nack. Capturing experience - a matter of contextualising events. In: Proceedings of the 1st ACM WS on Experiential Telepresence (ETP 03), November, 7th, 2003, 2003 Short position paper End User Semantic Web Interaction Workshop (colocated with ISWC 2005) Recommending Topics from Rated Objects November 7 2005 Galway, Ireland Lloyd Rutledge, Lora Aroyo, Peter Gorgels, Natalia Stash, Mettina Veenstra, Rogier Brussee. Recommending Topics from Rated Objects. In: End User Semantic Web Interaction Workshop (colocated with ISWC 2005), November 7, 2005 Much current research on hypermedia generation accepts user input only at the start of an otherwise fullyautomated process. However, since multimedia presentation creation is often a complex and creative process, it has multiple phases which would each benefit from human intervention. This paper presents a hypermedia generation model that lets the user influence all phases of this computer-assisted human-guided process. The main focus is on providing extra support for helping the user find relevant media items and combine them meaningfully into a rich and coherent multimedia presentation. Like fully-automated systems, our approach uses explicit knowledge about the presentation's topic domain, narrative structures, hypermedia presentation and distinctions between media modalities. This paper presents a motivating scenario that is used to derive a number of system requirements and to discusses the pros and cons of the presented approach. SampLe: Towards a Framework for System-supported Multimedia Authoring INS-E0302 August 2003 Multimedia semantics Kateryna Falkovych, Frank Nack, Jacco van Ossenbruggen, Lloyd Rutledge. SampLe: Towards a Framework for System-supported Multimedia Authoring, number INS-E0302, August, 2003 CHI 2006 Workshop on Mobile Social Software Affective Speech for Social Communication: Implementation Challenges in Text-to-Speech for Short Messages April 2006 The flexibility to choose from different modal content presentation will be an important feature in future ubiquitous application. Currently, short messages (e.g. SMS/MMS) are only available in visual form. However, in certain situations, users may like to have these messages presented in audio form. We explored the alternative of presenting short messages in affective synthetic text-to-speech form special for social communications between teens. Evaluation of this alternative presentation reveals that, for emotion recognition, it was easier to interpret emotion messages generated from affective synthetic speech. Although there is no actual difference in the way people think they were able to derive emotions from both types of messages. For teens, affective synthetic speech is sometimes fun to usei Montreal, Canada Alia Amin, Jimmy Wang. Affective Speech for Social Communication: Implementation Challenges in Text-to-Speech for Short Messages. In: CHI 2006 Workshop on Mobile Social Software, April, 2006 Proceedings of W3C Workshop on Metadata for Content Adaptation Discourse knowledge in device independent document formatting October, 12-13 2004 Most document structures define layout structures which implicitly define semantic relationships between content elements. While document structures for text are well established (books, reports, papers etc.), models for time based documents such as multimedia and hypermedia are relatively new and lack established document structures. Traditional document description languages convey domain-dependent semantic relationships implicitly, using domain-independent mark-up for expressing layout. This works well for textual documents a,s for example, CSS and HTML demonstrate. True device independence, however, sometimes requires a change of document model to maintain the content semantics. To achieve this we need explicit information about the discourse role of the content element. We propose a model in which content is marked-up with the discourse role it plays in the document. This way the formatter has knowledge about the function of a content element so it can make appropriate lay out choices. Dublin, Ireland Joost Geurts, Jacco van Ossenbruggen, Lynda Hardman. Discourse knowledge in device independent document formatting. In: Proceedings of W3C Workshop on Metadata for Content Adaptation, October, 12-13, 2004 http://www.cwi.nl/$sim$lloyd/Nile02Slides/ This paper explores the setting and processing of constraints over the metrics of narrative hierarchical structure of presentations as they are automatically generated from queries on stored media, annotations and semantic data. We describe how these functions can be implemented in our existing constraint-based multimedia generation system, Cuypers. The metrics we use are established measures of the node-edge graphs representing the navigational structure of hypermedia documents. This work applies graph restructuring techniques to Cuypers backtracking programming for maintaining constraints to enable the system to keep the narrative structure graph of the presentation within bounds as it is being built. The resulting graph is then processed to assemble media and place it in a generated narrative. We assert that the resulting well-shaped narrative hierarchies help the user keep track of where the presentation is and where it is going as each point during its progression. Toward Constraining Generated Large-scale Narrative Graph Metrics Narrative, constraints, Cuypers, graph theory, link metric Lloyd Rutledge. Toward Constraining Generated Large-scale Narrative Graph Metrics There should also be a SEN tech report. 1 ACM Trans. Multimedia Comput. Commun. Appl. Structured Multimedia Authoring 1 Authoring context sensitive, interactive multimedia presentations is much more complex than authoring either purely audiovisual applications or text. Interactions among media objects need to be described as a set of spatio-temporal relationships that account for synchronous and asynchronous interactions, as well as on-demand linking behavior. This article considers the issues that need to be addressed by an authoring environment. We begin with a partitioning of concerns based on seven classes of authoring problems. We then describe a selection of multimedia authoring environments within four different authoring paradigms: structured, timeline, graph and scripting. We next provide observations and insights into the authoring process and argue that the structured paradigm provides the most useful framework for presentation authoring. We close with an example application of the structured multimedia authoring paradigm in the context of our own structure-based system GRiNS. Jan-Mar 2005 89-109 Dick C.A. Bulterman, Lynda Hardman. Structured Multimedia Authoring. ACM Trans. Multimedia Comput. Commun. Appl. 1(1), Jan-Mar, 2005 no ToKeN ISSN 1386-3681 This paper provides a solution to discourse structure adaptation in the process of automatic hypermedia presentation generation. Existing approaches to discourse structure composition are based on the assumption that a user can comprehend relations between the elements in a discourse structure if the overall structure is semantically coherent. This assumption does not, so far, take into account specific user needs. In this paper we show that although discourse structure composition approaches significantly differ, a general model of the composition process can be derived. Within this general model we identify how adaptation can be applied. We formulate the problem of discourse adaptation with regard to the general model and present our proposed solution. We implement this solution within the process of composing discourse structures for newspaper articles. Combining Coherence and Adaptation in Discourse-Oriented Hypermedia Generation INS-E0601 February 2006 Adaptive content structuring, coherence, discourse structure, newsarticle composition NWO CHIME Kateryna Falkovych, Federica Cena, Frank Nack. Combining Coherence and Adaptation in Discourse-Oriented Hypermedia Generation, number INS-E0601, February, 2006 See http://www.cwi.nl/$\sim$media/Dynamo/sion99.html for a project description Semi-automatic Hypermedia Presentation Generation Lynda Hardman et al.. Semi-automatic Hypermedia Presentation Generation Ambient intelligence opens new possibilities for interactive multimedia, leading towards applications where the selection, generation and playback of multimedia content can be directed and influenced by multiple users in an ambient sensor network. In this paper, we derive the basic requirements for a flexible infrastructure that can support the integration of multimedia and ambient intelligence, and enable rapid tailoring of interactive multimedia applications. We describe our implementation of the proposed infrastructure, and demonstrate its functionality through several prototype applications. Designing interactive ambient multimedia applications: requirements and implementation challenges INS-E0605 2006 Adaptive multimedia, ambient intelligence, software platform Zeljko Obrenovic, Frank Nack, Lynda Hardman. Designing interactive ambient multimedia applications: requirements and implementation challenges, number INS-E0605, 2006 Facet browsing has become popular as a user friendly interface to data repositories. We extend facet browsing of Semantic Web data in four ways. First, users are able to select and navigate through facets of resources of any type and to make selections based on properties of other, semantically related, types. Second, we address a disadvantage of hierarchy-based navigation by adding a keyword search interface that dynamically makes semantically relevant suggestions. Third, the interface of our browser, /facet, allows the inclusion of facet-specific display options that go beyond the hierarchical navigation that characterizes current facet browsing. Fourth, the browser works on any RDFS dataset without any additional configuration. These properties make /facet an ideal tool for Semantic Web developers that need a instant interface to their complete dataset. The automatic facet configuration generated by the system can then be further refined to configure it as a tool for end users. The implementation is based on current Web standards and open source software. The new functionality we provide is motivated using a scenario from the cultural heritage domain. /facet: A Browser for Heterogeneous Semantic Web Repositories INS-E0604 2006 facet browsing, heterogeneous querying, cross relations Michiel Hildebrand, Jacco van Ossenbruggen, Lynda Hardman. /facet: A Browser for Heterogeneous Semantic Web Repositories, number INS-E0604, 2006 MPEG-7 can be used to create complex and comprehensive metadata descriptions of multimedia content. Since MPEG-7 is defined in terms of an XML schema, the semantics of its elements have no formal grounding. In addition, certain features can be described in multiple ways. MPEG-7 profiles are subsets of the standard that apply to specific application areas, which can be used to reduce this syntactic variability, but they still lack formal semantics. In this paper, we propose an approach for expressing semantics explicitly by formalizing the semantic constraints of a profile using ontologies and rules, thus enabling interoperability and automatic use for MPEG-7 based applications. We demonstrate the feasibility of the approach by implementing a validation service for a subset of the semantic constraints of the Detailed Audiovisual Profile (DAVP). Enabling Multimedia Metadata Interoperability by Defining Formal Semantics of MPEG-7 Profiles INS-E0606 August 2006 MPEG-7, Metadata interoperability, OWL/RDF, Semantic validation service Raphael Troncy, Werner Bailer, Michael Hausenblas, Philip Hofmair, Rudolf Schlatte. Enabling Multimedia Metadata Interoperability by Defining Formal Semantics of MPEG-7 Profiles, number INS-E0606, August, 2006 Multimedia presentation generation has to be able to balance the functional aspects of a presentation that address the information needs of the user, and its aesthetic form. We demonstrate our approach using automatic colour design for which we integrate relevant aspects of colour theory. Colour selection takes the relative importance of form and function into account through the use of weights in the generation process. We do not provide a definition of the relative importance of form versus function, but seek to explore the roles of subjective elements in the generation process. Colour Picking, the Pecking Order of Form and Function INS-R0303 June 2003 Multimedia semantics, automatic colour design, colour harmonisation, style-driven multimedia presentation generation Frank Nack, Amit Manniesing, Lynda Hardman. Colour Picking, the Pecking Order of Form and Function, number INS-R0303, June, 2003 A multimedia presentation can be viewed as a collection of multimedia items (such as image, text, video and audio), along with detailed information that describes the spatial and temporal placement of the items as part of the presentation. Manual multimedia authoring involves explicitly stating the placement of each media item in the spatial and temporal dimensions. The drawback of this approach is that resulting presentations are hard to adapt to different target platforms, network resources, and user preferences. An approach to solving this problem is to abstract from the low-level presentation details, for example by specifying the high-level semantic relations between the media items. The presentation itself can then be generated from the semantic relations along with a generic set of transformation rules, specifying how each semantic relation can be conveyed using multimedia constructs. These constructs may differ depending on the target platform, current network conditions or user preferences. We are thus able to automatically adapt the presentation to a wide variety of different circumstances while ensuring that the underlying message of the presentation remains the same. This approach requires an execution environment in which transformation rules, resulting in a set of constraints, are derived from a given semantic description. The resulting set of constraints can then be solved to create a final multimedia presentation. The paper describes the design and implementation of such a system. It explains the advantages of using constraint logic programming to realize the implementation of both the transformation rules and the constraints system. It also demonstrates the need for two different types of constraints. Quantitative constraints are needed to verify whether the final form presentation meets all the numeric constraints that are required by the environment. Qualitative constraints are needed to facilitate high-level reasoning and presentation encoding. While the quantitative constraints can be handled by off-the-shelf constraint solvers, the qualitative constraints needed are specific to the multimedia domain and need to be defined explicitly. Application-Specific Constraints for Multimedia Presentation Geneneration INS-R0107 May 31 2001 Multimedia, transformations, constraints, CLP, CHR, ECLiPSe. Joost Geurts, Jacco van Ossenbruggen, Lynda Hardman. Application-Specific Constraints for Multimedia Presentation Geneneration, number INS-R0107, May 31, 2001 Published as CWI technical report INS-R0205 Technical University of Catalonia Design dependencies within the automatic generation of hypermedia presentations June 30 2002 Hypermedia, Multimedia, Cuypers, Formatting Model, Attention Oscar Rosell Martinez. Design dependencies within the automatic generation of hypermedia presentations, June 30, 2002 IEEE MultiMedia Everything You Wanted to Know About MPEG-7: Part 2 Part 1 of this article provided an overview of the development, functionality, and applicability of MPEG-7. In Part 2 we discuss the description of MPEG-7 s concepts, terminology, and requirements. We then compare MPEG-7 with other approaches to multimedia content description. October - December 1999 64-73 Frank Nack, Adam T. Lindsay. Everything You Wanted to Know About MPEG-7: Part 2. IEEE MultiMedia, October - December, 1999 IEEE MultiMedia Everything You Wanted to Know About MPEG-7: Part 1 Part I of this article provides an overview of the development, functionality, and applicability of MPEG-7. We ll first present the role of MPEG-7 within the context of past MPEG standards. We then outline ideas of what should be possible using MPEG-7 technology. In Part II, we ll discuss the description of MPEG-7 s concepts, terminology, and requirements. We ll then compare MPEG-7 to other approaches on multimedia content description. July - September 1999 65-77 Frank Nack, Adam T. Lindsay. Everything You Wanted to Know About MPEG-7: Part 1. IEEE MultiMedia, July - September, 1999 18 Computer Standards and Interfaces Integrating the Amsterdam Hypermedia Model into the Standard Reference Model for Intelligent Multimedia Presentation Sytems 6-7 1997 497-508 Lynda Hardman, Marcel Worring, Dick C.A. Bulterman. Integrating the Amsterdam Hypermedia Model into the Standard Reference Model for Intelligent Multimedia Presentation Sytems. Computer Standards and Interfaces 18(6-7), 1997 yes 12 based on http://ftp.cwi.nl/CWIreports/INS//INS-E0309.pdf NAP,NWO NASH,ToKeN IEEE MultiMedia That Obscure Object of Desire: Multimedia Metadata on the Web (Part II) 1 This article discusses the state of the art in metadata for audio-visual media in large semantic networks, such as the Semantic Web. Our discussion is predominantly motivated by the two most widely known approaches towards machine-processable and semantic-based content description, namely the Semantic Web activity of the W3C and ISO's efforts in the direction of complex media content modeling, in particular the Multimedia Content Description Interface (MPEG-7). We explain that the conceptual ideas and technologies discussed in both approaches are essential for the next step in multimedia development. Unfortunately, there are still many practical obstacles that block their widespread use for providing multimedia metadata on the Web. Based on a scenario to explain our vision of a media-aware Semantic Web, we derive in Part I a number of problems regarding the semantic content description of media units. We then discuss the multimedia production chain, in particular emphasizing the role of progressive metadata production. As a result we distill a set of media-based metadata production requirements and show how current media production environments fail to address these. We then introduce those parts of the W3C and ISO standardization works that are relevant to our discussion. In Part II of this article, we analyze their abilities to define structures for describing media semantics, discuss syntactic and semantic problems, ontological problems for media semantics, and the problems of applying the theoretical concepts to real world problems. Part II concludes with implications of the findings for future action with respect to the actions the community should take. January - March 2005 Semantic Web, metadata production, multimedia production process, XML, XML Schema, RDF, RDF Schema, MPEG-4, MPEG-7, MPEG-21 I2RP 54-63 Frank Nack, Jacco van Ossenbruggen, Lynda Hardman. That Obscure Object of Desire: Multimedia Metadata on the Web (Part II). IEEE MultiMedia 12(1), January - March, 2005 Virtual Reality (VR) is sometimes used to give the user an immersive, three-dimensional sense of a real-world setting. VR is also sometimes used for information visualization, taking advantage of the perceptual characteristics of VR to convey information. This paper presents the Dam Square Virtual Context, a Web-based VR that is a combination of both these uses. This VR presents mock-ups of both Dam Square in the city of Amsterdam and the city's largest museum, the Rijksmuseum. This VR conveys abstract information that includes the relationships between objects in the museum, such as paintings of Amsterdam cityscapes, and the corresponding objects in Dam Square itself, such as the buildings and neighborhoods shown in these paintings. It is thus a multimedia combining VR wireframes and bitmaps with still images and interface scripts. The principle behind this is that the user learns how to walk through the museum to view objects that together convey a unified view about Amsterdam, which the user can then later walk through. Virtual context - relating media objects to their real world subjects INS-R0022 December 2000 Virtual Reality, museum objects, hypermedia Lloyd Rutledge, Alex R. van Ballegooij, Anton Eliens. Virtual context - relating media objects to their real world subjects, number INS-R0022, December, 2000 Proceedings of the Summer 1991 USENIX Conference A Structure for Transportable, Dynamic Multimedia Documents 1991 137-155 Nashville, TN Dick C.A. Bulterman, Guido van Rossum, Robert van Liere. A Structure for Transportable, Dynamic Multimedia Documents. In: Proceedings of the Summer 1991 USENIX Conference, 1991 A single multimedia document model needs to include a wide range of different types of information. In particular, information about space and time is essential for determining the spatial and temporal placement of elements within a presentation. Each information type included in a document model requires its own structuring mechanisms. The language used to express the document model has to be able to encapsulate the plurality of required structures. While this is a process that can be carried out relatively easily during the initial design of a language, it is more difficult in the case that a particular document language already exists and extra multimedia characteristics are required. For example, one could consider adding temporal information to an existing "static" document language. We investigate the underlying problems of superimposing a new document feature on an existing language and discuss possible strategies for integrating the required extra information in a modified document description language. Integrating Multimedia Characteristics in Web-based Document Languages INS-R0024 December 2000 multimedia document characteristics, time-based, spatial, text-flow, composition structures, Web markup languages Jacco van Ossenbruggen, Lynda Hardman, Lloyd Rutledge. Integrating Multimedia Characteristics in Web-based Document Languages, number INS-R0024, December, 2000 Poster ACM Hypertext Generalized Semantics-to-Document Derivation September 6-9 2005 284-285 Salzburg, Austria Lloyd Rutledge, Martin Alberink, Lynda Hardman, Mettina Veenstra. Generalized Semantics-to-Document Derivation. In: ACM Hypertext, September 6-9, 2005 Rejected by IHM-HCI 2001 TEd: a Treemap based Editor widget September 10-14 2001 Stephane Sire. TEd: a Treemap based Editor widget, September 10-14, 2001 Workshop on Computational Semiotics for New Media About the influence of computer semiotics on communal Intelligence June 29-30 2000 University of Surrey, Guildford, Surrey, UK Frank Nack. About the influence of computer semiotics on communal Intelligence. In: Workshop on Computational Semiotics for New Media, June 29-30, 2000 An investigation on the relationship between the user model and graphic representations for the automated generation of multimedia presentations 2003 Diploma Media System Design, University of Applied Sciences Darmstadt Katharina Schwarz. An investigation on the relationship between the user model and graphic representations for the automated generation of multimedia presentations, 2003 This is a revised version of CWI technical report INS-E0606 Semantic Multimedia - SAMT 2006 Enabling Multimedia Metadata Interoperability by Defining Formal Semantics of MPEG-7 Profiles MPEG-7 can be used to create complex and comprehensive metadata descriptions of multimedia content. Since MPEG-7 is defined in terms of an XML schema, the semantics of its elements have no formal grounding. In addition, certain features can be described in multiple ways. MPEG-7 profiles are subsets of the standard that apply to specific application areas, which can be used to reduce this syntactic variability, but they still lack formal semantics. In this paper, we propose an approach for expressing semantics explicitly by formalizing the semantic constraints of a profile using ontologies and rules, thus enabling interoperability and automatic use for MPEG-7 based applications. We demonstrate the feasibility of the approach by implementing a validation service for a subset of the semantic constraints of the Detailed Audiovisual Profile (DAVP). December 6-8 2006 MPEG-7, Metadata interoperability, OWL/RDF, Semantic validation service 41-55 Athens, Greece Raphael Troncy, Werner Bailer, Michael Hausenblas, Philip Hofmair, Rudolf Schlatte. Enabling Multimedia Metadata Interoperability by Defining Formal Semantics of MPEG-7 Profiles. In: Semantic Multimedia - SAMT 2006, December 6-8, 2006 7 New Review of Hypermedia and Multimedia The Role of High-level and Low-level Features in Style-based Retrieval and Generation of Multimedia Presentations In this article we argue that the automatic generation of dynamic multimedia presentation requires both low-level collections of objective measurements for media units representing prototypical style elements, and high-level conceptual descriptions supporting contextual and presentational requirements. Only the combination of both facilitates the retrieval of adequate material and its user-centered presentation. We discuss the problems of visual signification for images in dynamic systems and explain how a combined approach can help overcome such problems. We then propose an architecture for such a system and present its applicability for a museum-oriented multimedia system with a working example. 2001 39-65 Frank Nack, Menzo Windhouwer, Lynda Hardman, Eric Pauwels, Michele Huijberts. The Role of High-level and Low-level Features in Style-based Retrieval and Generation of Multimedia Presentations. New Review of Hypermedia and Multimedia 7, 2001 This is the description of the conference panel, organized by Chitra Dorai and Frank Nack Proceedings of the 10th ACM International Conference on Multimedia Media Semantics: Who needs it and why? December 1-6 2002 580-583 Juan-les-Pins, France Chitra Dorai, Andreas Mauthe, Frank Nack, Lloyd Rutledge, Thomas Sikora, Herbert Zettl. Media Semantics: Who needs it and why?. In: Proceedings of the 10th ACM International Conference on Multimedia, December 1-6, 2002 6 edited by Lynda Hardman New Review of Hypermedia and Multimedia 2000 Hybrid Narrative and Associative/Categorical Strategies for Interactive and Dynamic Video Presentation Generation There are a number of different approaches for automatically selecting video clips from a video database and sequencing them into meaningful presentations for viewers. The video database represents a multidimensional video hyperspace, and the sequencing algorithms function as (interactive) dynamic linking and path generation techniques within this hyperspace. Sequencing has been based upon either a narrative or a categorical model of video form. Each of these forms has its respective advantages and disadvantages, and varying suitability for different applications. The two primary forms may also be combined into several hybrid forms, both at the same level and at different levels of the syntactic composition of video sequences, to provide more options for authoring interactive dynamic video productions. Narrative, categorical, and hybrid sequence generation strategies can be applied to a variety of media modalities, including the automated generation of behaviour within virtual environments and computer animations. 2000 111-145 Craig Lindley, Frank Nack. Hybrid Narrative and Associative/Categorical Strategies for Interactive and Dynamic Video Presentation Generation. New Review of Hypermedia and Multimedia 2000 6, 2000 The report describes the architecture of emphCuypers, a system supporting second and third generation Web-based multimedia. First generation Web-content encodes information in handwritten (HTML) Web pages. Second generation Web content generates HTML pages on demand, e.g. by filling in templates with content retrieved dynamically from a database or transformation of structured documents using style sheets (e.g. XSLT). Third generation Web pages will make use of rich markup (e.g. XML) along with metadata (e.g. RDF) schemes to make the content not only machine readable but also machine processable -- a necessary pre-requisite to the emphSemantic Web. While text-based content on the Web is already rapidly approaching the third generation, multimedia content is still trying to catch up with second generation techniques. Multimedia document processing has a number of fundamentally different requirements from text which make it more difficult to incorporate within the document processing chain. In particular, multimedia transformation uses different document and presentation abstractions, its formatting rules cannot be based on text-flow, it requires feedback from the formatting back-end and is hard to describe in the functional style of current style languages. We state the requirements for second generation processing of multimedia and describe how these have been incorporated in our prototype multimedia document transformation environment, emphCuypers. The system overcomes a number of the restrictions of the text-flow based tool sets by integrating a number of conceptually distinct processing steps in a single runtime execution environment. We describe the need for these different processing steps and describe them in turn (semantic structure, communicative device, qualitative constraints, quantitative constraints, final form presentation), and illustrate our approach by means of an example. We conclude by discussing the models and techniques required for the creation of third generation multimedia content. Cuypers: a semi-automatic hypermedia generation system INS-R0025 December 2000 Multimedia, Transformations, CLP, XSLT, XML, SMIL, Topia Jacco van Ossenbruggen, Frank Cornelissen, Joost Geurts, Lloyd Rutledge, Lynda Hardman. Cuypers: a semi-automatic hypermedia generation system, number INS-R0025, December, 2000 Workshop colocated with ISWC 2005 Semantic Web Case Studies and Best Practices for eBusiness Multimedia Annotations and the Semantic Web November 7 2005 Jacco van Ossenbruggen, Giorgos Stamou, Jeff Z. Pan. Multimedia Annotations and the Semantic Web. In: Semantic Web Case Studies and Best Practices for eBusiness, November 7, 2005 Museums and the Web 2007 Searching and Annotating Virtual Heritage Collections with Semantic-Web Techniques April 11-14 2007 This paper describes ongoing work of a project aimed at exploiting Semantic Web techniques to support searching and annotating large cross-institutional digital-heritage collections. The project demonstrator contains multiple collections and multiple vocabularies. The architecture is fully based on Web standards. We show novel search and presentation techniques which make use of interoperability between the collections and between the vocabularies. Jacco van Ossenbruggen, Alia Amin, Lynda Hardman, Michiel Hildebrand, Mark van Assem, Borys Omelayenko, Guus Schreiber, Anna Tordai, Victor de Boer, Bob Wielinga, Jan Wielemaker, Marco de Niet, Jos Taekema, Marie-France van Orsouw, Annemiek Teesing. Searching and Annotating Virtual Heritage Collections with Semantic-Web Techniques. In: Museums and the Web 2007, April 11-14, 2007 13 John R. Smith, editor IEEE MultiMedia Multimedia Annotations on the Semantic Web 1 Multimedia in all forms (images, video, graphics, music, speech) is exploding on the Web. The content needs to be annotated and indexed to enable effective search and retrieval. However, recent standards and best practices for multimedia metadata don't provide semantically rich descriptions of multimedia content. On the other hand, the World Wide Web Consortium's (W3C's) Semantic Web effort has been making great progress in advancing techniques for annotating semantics of Web resources. To bridge this gap, a new W3C task force has been created to investigate multimedia annotations on the Semantic Web. This article examines the problems of semantically annotating multimedia and describes the integration of multimedia metadata with the Semantic Web. (Editor's note by John R. Smith). January-March 2006 86-90 Giorgos Stamou, Jacco van Ossenbruggen, Jeff Z. Pan, Guus Schreiber. Multimedia Annotations on the Semantic Web. IEEE MultiMedia 13(1), January-March, 2006 IEEE Intelligent Systems Media information spaces - a semantic challenge January-February 2002 81-83 Frank Nack. Media information spaces - a semantic challenge. IEEE Intelligent Systems, January-February, 2002 2 Multimedia Computing and Systems (icmcs), IEEE Computer Society Generic Film Forms for Dynamic Virtual Video Synthesis The FRAMES project within the RDN CRC (Cooperative Research Centre for Research Data Networks) is developing an experimental environment for video content-based retrieval and dynamic virtual video synthesis from archives of video data. The FRAMES research prototype is a video synthesis system incorporating an association engine for automatically generating associative sequences of video data. Authors preparing video descriptions, association specifications, and component video sequences require guidelines for developing the primitive video components and semantic representations required to create particular film forms. High level formal models of film and video provide such a methodology. In particular, categorical, associational, and abstract film forms can be generated based upon video component descriptors and specifications, while narrative and rhetorical forms require the introduction of independent rules or relations expressing causal and rhetorical relationships, respectively. 1999 97-101 Craig A. Lindley. Generic Film Forms for Dynamic Virtual Video Synthesis. Multimedia Computing and Systems (icmcs), IEEE Computer Society 2, 1999 Multimedia Modelling Models, Media and Motion: Using the Web to Support Multimedia Documents November 17-20 1997 227-246 Dick C.A. Bulterman. Models, Media and Motion: Using the Web to Support Multimedia Documents. In: Multimedia Modelling, November 17-20, 1997 Actes de la 4e conference internationale Hypertextes et Hypermedias -- realisations, outils & methodes Cooperative use of MHEG and HyTime in Hypermedia Environments (1)2-3-4 The standards MHEG-5 and HyTime are interchange formats for hypermedia information. While they may seem to compete, they actually play separate and complementary roles in a complete and open hypermedia environment. MHEG-5 is used for portable final-form hypermedia presentations. HyTime is used for the long-term, presentation-independent storage of hypermedia documents. Given these tasks, MHEG-5 can be used to encode presentations of HyTime documents. This paper explores these two standards, the cooperative roles they play and their application to the CMIF hypermedia environment architecture. The issues discussed include the semantic overlap between the hypermedia models each standard represents and how the use of each standard affects the cooperative use of the other. September Hypertextes et Hypermedias 1997 MHEG-5, HyTime, DSSSL, CMIF, Hypermedia Authoring 57-73 Paris Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman, Dick C.A. Bulterman. Cooperative use of MHEG and HyTime in Hypermedia Environments. In: Actes de la 4e conference internationale Hypertextes et Hypermedias -- realisations, outils & methodes, number (1)2-3-4, Editions HERMES, September, 1997 0 0-8186-8353-8 Database and Expert Systems Applications (DEXA'98), IEEE Computer Society The FRAMES Processing Model for the Synthesis of Dynamic Virtual Video Sequences The FRAMES project within the RDN CRC (Cooperative Research Centre for Research Data Networks) is developing an experimental environment for video content-based retrieval and dynamic virtual video synthesis from archives of video data. This paper describes the FRAMES dynamic virtual video synthesis process. The generation of dynamic virtual videos is based upon content descriptions of archived material, together with a specification of the videos that are to be created. Several different types of queries that refer to semantic features can be embedded within a specification; the execution of these queries determines the specific video content that will be displayed. This paper begins by describing the semantic descriptions that FRAMES uses, based upon a multi-level model of video semantics. The paper then describes the query types and associated execution processes used to generate dynamic virtual videos based upon prescriptions and content descriptions. 1998 119-122 Craig A. Lindley. The FRAMES Processing Model for the Synthesis of Dynamic Virtual Video Sequences. Database and Expert Systems Applications (DEXA'98), IEEE Computer Society, 1998 Electronic Proceedings of the ACM Workshop on Effective Abstractions in Multimedia Using the Amsterdam Hypermedia Model for Abstracting Presentation Behavior November 4 1995 We give a short description of the Amsterdam Hypermedia Model followed by examples of its use in a number of existing and planned applications. The main application to date has been as a basis of the multimedia authoring system, CMIFed, along with its ability to specify trade-offs for resource use. We discuss the models potential for generating differing document formats, followed by future work on using it as a goal format for generating multimedia documents. ACM Multimedia 1995 Workshop, San Francisco, CA Lynda Hardman, Dick C.A. Bulterman. Using the Amsterdam Hypermedia Model for Abstracting Presentation Behavior. In: Electronic Proceedings of the ACM Workshop on Effective Abstractions in Multimedia, November 4, 1995 ICCC/IFIP Conference -- Electronic Publishing '97 Generic Hypermedia Structure and Presentation Specification April 1997 Hypermedia, Document Processing, Dexter, AHM, SGML, DSSSL, HyTime We consider the generic hypermedia structure of a document to be a means of representing the document that allows it to be processed into a wide variety of presentations. Representing a document in this manner requires additional specification and resources to render it into any presentation. In this paper we discuss the relationship between the generic hypermedia structure of documents and the processing of this structure into presentation. Our discussion is expressed in terms of existing models for hypertext and hypermedia systems and also in terms of ISO standards for text and hypermedia document formatting and processing. The discussion and the resulting formalisms and the resulting formalisms are illustrated with extension designs for the hypermedia authoring and presentation environment developed at our laboratory. Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman, Dick C.A. Bulterman. Generic Hypermedia Structure and Presentation Specification. In: ICCC/IFIP Conference -- Electronic Publishing '97, April, 1997 8th European Digital Libraries Conference Next Generation Search Interfaces - Interactive Data Exploration and Hypothesis Testing To date, the majority of Web search engines have provided simple keyword search interfaces that present the results as a ranked list of hyperlinks. More recently researchers have been investigating interactive, graphical and multimedia approaches which use ontologies to model the knowledge space. Such systems use the semantic relationships to structure the assimilated search results into interactive semantic graphs or hypermedia presentations which enable the user to quickly and easily explore the results and detect previously unrecognized associations. More recently, the proliferation of eResearch communities has led to a demand for search interfaces which automate the discovery, analysis and assimilation of multiple information sources in order to prove or disprove a particular scientific theory or hypothesis. We believe that such semiautomated analysis, assimilation and hypothesis-driven approaches represent the next generation of search engines. In this paper we describe and evaluate such a search interface which we have developed for a particular eScience application. September 2004 86-98 Jane Hunter, Katya Falkovych, Suzanne Little. Next Generation Search Interfaces - Interactive Data Exploration and Hypothesis Testing. In: 8th European Digital Libraries Conference, September, 2004 DEMONSTRATION SESSION: Demos Proceedings of the sixteenth ACM Conference on Hypertext and Hypermedia 2005 Vox populi: a tool for automatically generating video documentaries Vox Populi is a system that automatically generates video documentaries. Our application domain is video interviews about controversial topics. Via a Web interface the user selects one of the possible topics and a point of view she would like the generated sequence to present, and the engine selects and assembles video material from the repository to satisfy the user request. September 2005 292-294 Stefano Bocconi, Frank Nack, Lynda Hardman. Vox populi: a tool for automatically generating video documentaries. In: Proceedings of the sixteenth ACM Conference on Hypertext and Hypermedia 2005, September, 2005 Workshop on Recommendation and Personalization in eCommerce (at the 2nd International Conference on Adaptive Hypermedia and Adaptive Web Based Systems) An explicit model for tailor-made eCommerce web presentations May 29-31 2002 Malaga, Spain Susanne G. Loeber, Lora M. Aroyo, Lynda Hardman. An explicit model for tailor-made eCommerce web presentations. In: Workshop on Recommendation and Personalization in eCommerce (at the 2nd International Conference on Adaptive Hypermedia and Adaptive Web Based Systems), May 29-31, 2002 Formal Ontologies Meet Industry Supporting the decision process for the choice of a domain modeling scheme June 9-10 2005 Enterprises are experiencing the need to model their domain in order to improve the communication and control over their intellectual assets. There are several different approaches to modeling a domain, some old-fashioned, some trendy; practitioners are at a loss when having to decide which approach best suits their situation. In this research we compare different domain modeling schemes to find out which problem each of them is best suited to solve. The modeling schemes we consider are taxonomy, thesaurus and ontology. We draw relevant information from evaluating case studies and interviewing practitioners in the field of domain modeling. We restrict the scope to those projects that aim at improving the retrieval of information. The aim of this research is to support practitioners when they want to model their domain to choose the right scheme, considering the trade-offs being made between requirements and effort. Verona, Italy Katharina Schwarz, Timo Kouwenhoven, Virginia Dignum, Jacco van Ossenbruggen. Supporting the decision process for the choice of a domain modeling scheme. In: Formal Ontologies Meet Industry, June 9-10, 2005 Poster ACM Hypertext Cruising the Semantic Web with Noadster September 6-9 2005 290-291 Salzburg, Austria Lloyd Rutledge, Jacco van Ossenbruggen. Cruising the Semantic Web with Noadster. In: ACM Hypertext, September 6-9, 2005 International Workshop on Personalization Enhanced Access to Cultural Heritage Recommendation with Semantics for Cultural Heritage June 25 2007 This paper describes how the combination of recommendation and Semantic Web techniques can enhance user interaction with digital cultural heritage collections. The interface that this paper describes adapts user interaction to individual interest in both artworks and abstract concepts related to them. This paper presents an overview of several studies that evaluate a recommender system for museum collections. The findings and requirements derived from them form a basic framework for personalized exhibition tours that present multiple cultural artifacts as a structured and cohesive whole. Corfu, Greece Lloyd Rutledge, Lora Aroyo, Brussee, Natalia Stash, Yiwen Wang, Henriette Cramer, Vanessa Evers, Satyan Ramlal. Recommendation with Semantics for Cultural Heritage. In: International Workshop on Personalization Enhanced Access to Cultural Heritage, June 25, 2007 2 IEEE MultiMedia Copyright - Copywrong 4 October - December 2000 Jana Dittmann, Frank Nack. Copyright - Copywrong. IEEE MultiMedia 2(4), October - December, 2000 13 Transactions on Information Systems Embedded Video in Hypermedia Documents: Supporting Integration and Adaptive Control 4 1995 440-470 Dick C.A. Bulterman. Embedded Video in Hypermedia Documents: Supporting Integration and Adaptive Control. Transactions on Information Systems 13(4), 1995 part of 2nd European Semantic Web Conference Workshop on Multimedia and the Semantic Web Requirements for practical multimedia annotation Applications that use annotated multimedia assets need to be able to process all the annotations about a specific media asset. At first sight, this seems almost trivial, but annotations are needed for different levels of description, these need to be related to each other in the appropriate way and, in particular on the Semantic Web, annotations may not all be stored in the same place. We distinguish between technical descriptions of a media asset from content-level descriptions. At both levels, the annotations needed in a single application may come from different vocabularies. In addition, the instantiated values for a term used from an ontology also need to be specified. We present a number of existing vocabularies related to multimedia, discuss the above problems then discuss requirements for and the desirability of a lightweight multimedia ontology. May 2005 4-11 Heraklion, Crete Joost Geurts, Jacco van Ossenbruggen, Lynda Hardman. Requirements for practical multimedia annotation. In: Workshop on Multimedia and the Semantic Web, May, 2005 1-871516-08-0 Hypertext: State of the Art User-Centred Hypertext Design: The Application of HCI Design Principles and Guidelines 1989 252-259 York, UK Lynda Hardman, Brian S. Sharratt. User-Centred Hypertext Design: The Application of HCI Design Principles and Guidelines. In: Hypertext: State of the Art, 1989 The main thrust of this work is determined on the development of design rules to guide the manual or automatic design of user centred web environments for museums of the fine arts. Within the broad field of art presentation we focus on the analysis of the-state-of-the-art of museum website design. We are in particular interested in websites that present actual works of art. This leaves out simpler websites that only offer general information without displaying artifacts. Furthermore, we discuss the various needs of museum visitors. However, our design rules focus on those topics that will satisfy average museum visitors. There is still interesting work to be done regarding the needs of expert users, to whom a website might present information as diverse as the chemical composition of paints or the insurance value of a painting. Our design rules facilitate a wide range of devices and presentation sizes. However,for small screens (e.g.PDA s,phones) additional or different rules would need to be defined to come to an acceptable presentation. Our design rules cover quite a range of fundamental layout requirements such as topography, color and text, as well as more specialized layout issues such as corporate identity and art style dependent presentation. Aspects that we only glance upon and that desire more at tention are, for example, the combination of temporal media types, use of navigation and information structure. In the area of color, our rule set is still quite general and could be improved with additional, and more specific, rules. Design of interactive and adaptive interfaces to exploit large media-based knowledge spaces in the domain of museums for the fine arts June 14 Topia 2002 Diploma Media System Design, University of Applied Sciences Darmstadt Karolina Czajka. Design of interactive and adaptive interfaces to exploit large media-based knowledge spaces in the domain of museums for the fine arts, June 14, 2002 Proceedings of the 1st Conference on Computational Semiotics for Games and New Media New media semiotics - computation and aesthetic function September 10-12 2001 3-8 Amsterdam, The Netherlands Craig Lindley, Frank Nack, Andy Clarke, Grethe Mitchell, Clive Fencott. New media semiotics - computation and aesthetic function. In: Proceedings of the 1st Conference on Computational Semiotics for Games and New Media, September 10-12, 2001 VU/CWI A HyTime-Compliant Interchange Format for CMIF Hypermedia Documents August 1995 Lynda Hardman, Jacco van Ossenbruggen, Dick C.A. Bulterman. A HyTime-Compliant Interchange Format for CMIF Hypermedia Documents, August, 1995 Until recently, the Semantic Web was little more than a name for the next generation Web infrastructure as envisioned by its inventor, Tim Berners-Lee. Now, with the introduction of XML and RDF, and new developments such as RDF Schema and DAML+OIL, the Semantic Web is rapidly taking shape. In this paper, we first give an overview of the state-of-the-art in Semantic Web technology, the key relationships with traditional hypermedia research, and a comprehensive reference list to various sets of literature (Hypertext, Web and Semantic Web). The core of the paper presents a research agenda by describing the open research issues in the development of the Semantic Web from the perspective of hypermedia research. Hypermedia and the Semantic web: A research agenda INS-R0105 May 31 2001 Hypermedia, Semantic Web, XML, RDF, DAML+OIL, SMIL Jacco van Ossenbruggen, Lynda Hardman, Lloyd Rutledge. Hypermedia and the Semantic web: A research agenda, number INS-R0105, May 31, 2001 7 IEEE MultiMedia All content counts - the future in digital media computing is meta 3 2000 10-13 Frank Nack. All content counts - the future in digital media computing is meta. IEEE MultiMedia 7(3), 2000 Proceedings of the 5th European Conference EuroITV 2007, Lecture Notes in Computer Science LNCS 4471, Springer An Architecture for Non-intrusive User Interfaces for Interactive Digital Television This paper presents an architecture that provides non-intrusive user interfaces in the interactive digital TV domain. The architecture is based on two concepts. First, the deployment of non-monolithic rendering for content consumption. This allows micro-level personalization of content delivery to be supported by utilizing different rendering components (e.g., sending video to the TV screen and extra information to a handheld device). Second, the definition of actions descriptions for user interaction, so that high-level user interaction intentions can be partitioned across a personalized collection of control components (e.g., handheld device). This paper introduces an over-all architecture to support micro-personalization and describes an implementation scenario developed to validate the architecture. May, 24-25 2007 11-20 Amsterdam, The Netherlands Pablo Cesar, Dick C.A. Bulterman, Zeljko Obrenovic, Julien Ducret, Samuel Cruz-Lara. An Architecture for Non-intrusive User Interfaces for Interactive Digital Television. In: Proceedings of the 5th European Conference EuroITV 2007, Lecture Notes in Computer Science LNCS 4471, Springer, May, 24-25, 2007 NAP,NWO NASH,ToKeN Creation, Use and Deployment of Digital Information From Syntactic Towards Semantic-driven Document Transformations March ACM H.5.1, I.7.2, H.5.4 2005 I2RP 55-72 Jacco van Ossenbruggen, Lynda Hardman. From Syntactic Towards Semantic-driven Document Transformations. In: Creation, Use and Deployment of Digital Information, Erlbaum, March, 2005 Semantic descriptions of non-textual media available on the web can be used to facilitate retrieval and presentation of media assets and documents containing them. While technologies for multimedia semantic descriptions already exist, there is as yet no formal description of a high quality multimedia ontology that is compatible with existing (semantic) web technologies. We explain the complexity of the problem using an annotation scenario. We then derive a number of requirements for specifying a formal multimedia ontology, including: compatibility with MPEG-7, embedding in foundational ontologies, and modularisation including separation of document structure from domain knowledge. We then present the developed ontology and discuss it with respect to our requirements. Adding Formal Semantics to MPEG7: Designing a Well-Founded Multimedia Ontology for the Web KU-N0407 January 2007 Multimedia Ontology, MPEG-7, DOLCE, Model Richard Arndt, Raphael Troncy, Steffen Staab, Lynda Hardman. Adding Formal Semantics to MPEG7: Designing a Well-Founded Multimedia Ontology for the Web, number KU-N0407, January, 2007 State of The Art Report in Eurographics '95 Authoring Support for Durable Interactive Multimedia Presentations August 29 - September 1 1995 Maastricht, The Netherlands Lynda Hardman, Dick C.A. Bulterman. Authoring Support for Durable Interactive Multimedia Presentations. In: State of The Art Report in Eurographics '95, August 29 - September 1, 1995 Proceedings of the IEEE International Conference on Multimedia and Expo (ICME) 2005 Using Rhetorical Annotations for Generating Video Documentaries July 2005 We use rhetorical annotations to specify a generation process that can assemble meaningful video sequences with a communicative goal and an argumentative progression. Our annotation schema encodes the verbal information contained in the audio channel, identifying the claims the interviewees make and the argumentation structures they use to make those claims. Based on this schema, we construct a semantic graph which is traversed by rhetoric-based strategies selecting video segments. The selected video segments are edited to form a meaningful video sequence. Stefano Bocconi, Frank Nack, Lynda Hardman. Using Rhetorical Annotations for Generating Video Documentaries. In: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME) 2005, July, 2005 The First ACM International Conference on Multimedia CMIFed: A Presentation Environment for Portable Hypermedia Documents August 1993 183-188 Guido van Rossum, Jack Jansen, K. Sjoerd Mullender, Dick C.A. Bulterman. CMIFed: A Presentation Environment for Portable Hypermedia Documents. In: The First ACM International Conference on Multimedia, August, 1993 Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A), ACM International Conference Proceeding Series, Vol. 225 Web browser accessibility using open source software A Web browser provides a uniform user interface to different types of information. Making this interface universally accessible and more interactive is a long term goal still far from being achieved. Universally accessible browsers require novel interaction modalities and additional functionalities, for which existing browsers tend to provide only partial solutions. Although functionality for Web accessibility can be found as open source and free software components, their reuse and integration is complex because they were developed in diverse implementation environments, following standards and conventions incompatible with the Web. To enable the integration of existing partial solutions within a mainstream Web browser environment, we have developed a middleware infrastructure, AMICO:WEB. This enables browser access to a wide variety of open source and free software components. The main contribution of AMICO:WEB is in enabling the syntactic interoperability between Web extension mechanisms and a variety of integration mechanisms used by open source and free software components. It also bridges the semantic differences between the high-level world of Web XML-based APIs and the low-level APIs of the device-oriented world. We discuss the design decisions made during the development of AMICO:WEB in the context of Web accessibility, using two typical usage scenarios: one describing a disabled user using a mainstream Web browser with additional interaction modalities; another describing a non-disabled user browsing in a suboptimal interaction situation. May, 7-8 2007 15-24 Banff, Canada Zeljko Obrenovic, Jacco van Ossenbruggen. Web browser accessibility using open source software. In: Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A), ACM International Conference Proceeding Series, Vol. 225, May, 7-8, 2007 16 Special issue on MPEG-7 Signal Processing: Image Communication Journal An Overview of the MPEG-7 Description Definition Language (DDL) Proposals 2000 271-293 Jane Hunter, Frank Nack. An Overview of the MPEG-7 Description Definition Language (DDL) Proposals. Signal Processing: Image Communication Journal 16, 2000 Also available on http://www.cwi.nl/$sim$media/group-www/theses/StefanoBocconi06.pdf Technische Universiteit Eindhoven 90-386-0824-1 The context of this research is one or more online video repositories containing several hours of documentary footage and users possibly interested only in particular topics of that material. In such a setting it is not possible to craft a single version containing all possible topics the user might like to see, unless including all the material, which is clearly not feasible. The main motivation for this research is, therefore, to enable an alternative authoring process for film makers to make all their material dynamically available to users, without having to edit a static final cut that would select out possible informative footage. Vox Populi: generating video documentaries from semantically annotated media repositories November 30 2006 Eindhoven, The Netherlands Stefano Bocconi. Vox Populi: generating video documentaries from semantically annotated media repositories, November 30, 2006 http://www.cwi.nl/$sim$lynda/talks/oratie/TUe_rede_slides_mei03.pdf Smart Style for Conveying Information May 2 hypermedia presentation generation, style dependencies, presentation structure 2003 Inaugural address given at the Technical University of Eindhoven Lynda Hardman. Smart Style for Conveying Information, May 2, 2003 Proceedings of the ACM Workshop on Multimedia for Human Communication - From Capture to Convey (MHC 05) Canonical Processes of Media Production November 2005 Creating compelling multimedia presentations is a complex task. It involves the capture of media assets, then editing and authoring these into one or more final presentations. Tools tend to concentrate on a single aspect of media production to reduce the complexity of the interface and to tailor support to a specific task. While these tools are suited to the task they are designed for, very often there is no consideration for input requirements for the next tool down the line. Each tool in the multimedia production chain has the potential for adding semantic annotations to the media asset at hand, describing relevant aspects of the asset and why it is being used for a particular purpose. These annotations need to be included in the information handed over to the next tool in the production chain. We specify inputs and outputs to a number of canonical processes we identify in multimedia production. We do not specify the intricate workings of the processes, but concentrate on the information flow between them. To validate our model we describe a number of workflows in terms of the processes and specify the inputs and outputs. Our claim is that by specifying explicitly the input and output required for processes that occur in widely differing uses of media we can provide the tool-building community with a small set of building blocks that can be supported for semantically aware media production tools. Lynda Hardman. Canonical Processes of Media Production. In: Proceedings of the ACM Workshop on Multimedia for Human Communication - From Capture to Convey (MHC 05), November, 2005 IEEE MultiMedia Aesthetics of contradiction As a culture, the multimedia computing community may also perpetuate myths arising from the inherent contradictions between human and machine. These contradictions influence the shape of our work -- a good enough motivation to step into the minefield of myth identification. There might be more myths to be addressed, but I'm merely discussing multimedia myths under the following categories, all of which are interrelated: handling content that's relevant, learning to deal with meaningful content that's concealed, and the meaning of content through the user's perspective. January - March 2003 14-17 Frank Nack. Aesthetics of contradiction. IEEE MultiMedia, January - March, 2003 This article discusses the state of the art in metadata for audio-visual media in large semantic networks, such as the Semantic Web. Our discussion is predominantly motivated by the two most widely known approaches towards machine-processable and semantic-based content description, namely the Semantic Web activity of the W3C and ISO's efforts in the direction of complex media content modeling, in particular the Multimedia Content Description Interface (MPEG-7). We explain that the conceptual ideas and technologies discussed in both approaches are essential for the next step in multimedia development. Unfortunately, there are still many practical obstacles that block their widespread use for providing multimedia metadata on the Web. Based on a scenario to explain our vision of a media-aware Semantic Web, we derive in Part I a number of problems regarding the semantic content description of media units. We then discuss the multimedia production chain, in particular emphasizing the role of progressive metadata production. As a result we distill a set of media-based metadata production requirements and show how current media production environments fail to address these. We then introduce those parts of the W3C and ISO standardization works that are relevant to our discussion. In Part II of this article, we analyze their abilities to define structures for describing media semantics, discuss syntactic and semantic problems, ontological problems for media semantics, and the problems of applying the theoretical concepts to real world problems. Part II concludes with implications of the findings for future action with respect to the actions the community should take. That Obscure Object of Desire: Multimedia Metadata on the Web (Part II) INS-E0309 December 3 2003 Semantic Web, metadata production, multimedia production process, XML, XML Schema, RDF, RDF Schema, MPEG-4, MPEG-7, MPEG-21 Jacco van Ossenbruggen, Frank Nack, Lynda Hardman. That Obscure Object of Desire: Multimedia Metadata on the Web (Part II), number INS-E0309, December 3, 2003 Multi-Perspective Requirements for a Common Multimedia Ontology Framework 2006 Response to call to define requirements for a multimedia ontology framework, see http://www.acemedia.org/aceMedia/reference/multimedia_ontology/ Raphael Troncy, Jacco van Ossenbruggen, Christian Halaschek-Wiener, Giorgos Stamou, Vassilis Tzouvaras. Multi-Perspective Requirements for a Common Multimedia Ontology Framework, 2006 ED-MEDIA 99 - World Conference on Educational Multimedia, Hypermedia & Educational Telecommunications GRiNS: An Authoring Environment for Web Multimedia June 19-24 1999 Seattle, Washington USA Dick C.A. Bulterman, Lloyd Rutledge, Lynda Hardman, Jack Jansen, K. Sjoerd Mullender. GRiNS: An Authoring Environment for Web Multimedia. In: ED-MEDIA 99 - World Conference on Educational Multimedia, Hypermedia & Educational Telecommunications, June 19-24, 1999 Proceedings of the K-Cap Workshop on Integrating Ontologies (IntOnt 2005) oMAP: Results of the Ontology Alignment Contest This paper summarizes the results of the emphoMAPsystem for the 2005 Campaign tests of the Ontology Alignment Evaluation Initiative. First, it describes the system and its main components. The results of the experiments for the three tasks follow. Then, we have a short discussion and interpretation of the results. Finally, we sketch some ideas to improve our system and provide the link to our current results. October 2 2005 92-96 Banff, Canada Umberto Straccia, Raphael Troncy. oMAP: Results of the Ontology Alignment Contest. In: Proceedings of the K-Cap Workshop on Integrating Ontologies (IntOnt 2005), October 2, 2005 Proceedings of the 9th ACM International Conference on Multimedia Designing Annotation Before It's Needed This paper considers the automated and semi-automated annotation of audiovisual media in a new type of production framework, A4SM (Authoring System for Syntactic, Semantic and Semiotic Modelling). We present the architecture of the framework and outline the underlying XML-Schema based content description structures of A4SM. We then describe tools for a news and demonstrate how video material can be annotated in real time and how this information can not only be used for retrieval but also can be used during the different phases of the production process itself. Finally, we discuss the pros and cons of our approach of evolving semantic networks as the basis for audio-visual content description. September 30 - October 5 2001 251-260 Ottawa, Ontario, Canada Frank Nack, Wolfgang Putz. Designing Annotation Before It's Needed. In: Proceedings of the 9th ACM International Conference on Multimedia, September 30 - October 5, 2001 1-58603-325-5 Knowledge Transformation for the Semantic Web UML for the Semantic Web: Transformation-Based Approaches The perspective role of UML as a conceptual modelling language for the Semantic Web has become an important research topic. We argue that UML could be a key technology for overcoming the ontology development bottleneck thanks to its wide acceptance and sophisticated tool support. Transformational approaches are a promising way of establishing a connection between UML and web-based ontology languages. We compare some proposals for defining transformations between UML and web ontology languages and discuss the different ways they handle the conceptual differences between these languages. We identify commonalities and differences of the approaches and point out open questions that have not or not satisfyingly been addressed by existing approaches. 2003 92-106 Kateryna Falkovych, Martha Sabou, Heiner Stuckenschmidt. UML for the Semantic Web: Transformation-Based Approaches. In: Knowledge Transformation for the Semantic Web, IOS Press, 2003 Proceedings International Conference on Media Futures Categorical, Narrative, and Hybrid Behavior Generation in the GENIE Environment for Interactive Narrative Virtual Worlds The GEneric Narrative Interaction Environment (GENIE) is a conceptual architecture for the exploration of interactive narrative generation in virtual worlds. Specific research demonstrators implement different aspects of the GENIE architecture, with the longer term goal of implementing the full system. A particular feature of GENIE is the adoption of narrative, categorical, and hybrid sequencing strategies for model-based presentation generation in virtual worlds. Narrative generation is concerned with the preservation of spatio-temporal continuity of action within the agent behaviors and events in the virtual world. Categorical sequencing is concerned with the generation of associative sequences of events based upon patterns of semantic similarity and dissimilarity. May 8 2001 65-68 Florence, Italy Craig Lindley, Frank Nack. Categorical, Narrative, and Hybrid Behavior Generation in the GENIE Environment for Interactive Narrative Virtual Worlds. In: Proceedings International Conference on Media Futures, May 8, 2001 A model of hypermedia is presented that allows the combination of hyper-structured information with dynamic multimedia information. The model is derived by extending the Dexter hypertext reference model and the CMIF multimedia model. The Amsterdam hypermedia model allows the following, in addition to the model provided by Dexter: beginitemize item the composition of multiple dynamic media, in order to specify a collection of time-based media making up a complete multimedia presentation; item the definition of channels for specifying default presentation information, allowing the specification of the presentation characteristics of nodes at a more general level than that for an individual node; item the composition of existing presentations into larger presentations, taking into account possible clashes of resource usage; item the inclusion of temporal relations while maintaining the separation of structure and presentation information, where time-based relationships are treated as presentation information; item the specification of context for anchors, in order to determine whether on following a link a part, or the whole, of the presentation is replaced by the information at the destination of the link. enditemize The Amsterdam hypermedia model enables the description of structured multimedia documents, incorporating time at a fundamental level, and extending the hypertext notion of links to timebased media and compositions of different media. The paper is organised as follows. The Dexter hypertext model and the CMIF multimedia model are summarised, and their limitations for use as a more general hypermedia model are discussed. The extensions included in the Amsterdam hypermedia model are described and a summary of the resulting model is given. The Amsterdam Hypermedia Model: Extending Hypertext to Support REAL Multimedia CS-R9306 January 1993 Lynda Hardman, Dick C.A. Bulterman, Guido van Rossum. The Amsterdam Hypermedia Model: Extending Hypertext to Support REAL Multimedia, number CS-R9306, January, 1993 This is a preliminary version of an article that will be published in IEEE MultiMedia That Obscure Object of Desire: Multimedia Metadata on the Web (Part I) INS-E0308 December 3 2003 Semantic Web, metadata production, multimedia production process, XML, XML Schema, RDF, RDF Schema, MPEG-4, MPEG-7, MPEG-21 Jacco van Ossenbruggen, Frank Nack, Lynda Hardman. That Obscure Object of Desire: Multimedia Metadata on the Web (Part I), number INS-E0308, December 3, 2003 Traditionally, research in applying Semantic Web technology to multimedia information systems has focused on using annotations and ontologies to improve the retrieval process. This paper concentrates on improving the presentation of the retrieval results. First, our approach uses ontological domain knowledge to select and organize the content relevant to the topic the user is interested in. Domain ontologies are valuable in the presentation generation process, because effective presentations are those that succeed in conveying the relevant domain semantics to the user. Explicit discourse and narrative knowledge allows selection of appropriate presentation genres and creation of narrative structures, which are used for conveying these domain relations. In addition, knowledge of graphic design and media characteristics is essential to transform abstract presentation structures in real multimedia presentations. Design knowledge determines how the semantics and presentation structure are expressed in the multimedia presentation. In traditional Web environments, this type of design knowledge remains implicit, hidden in style sheets and other document transformation code. Our second use of Semantic Web technology is to model design knowledge explicitly, and to let it drive the transformations needed to turn annotated media items into structured presentations. In this article we argue that for the automatic generation of adaptive multimedia presentations we are in need of expandable, adaptable style descriptions which provide both high-level conceptual and low-level feature extraction information. Only the combination of both facilitates the retrieval of adequate material and its user-centred presentation. We discuss the requirements for an adaptable Web-based environment for museums presenting visual artefacts. We then present the framework of our prototype multimedia generation environment which transforms a high-level user query into a concrete multimedia final-form encoding that is playable on an end-users' platform. We describe the underlying architecture and provide a working example. Towards Ontology-driven Discourse: From Semantic Graphs to Multimedia Presentations INS-R0305 May 2003 Semantic Web, Discourse, Narrative, multimedia presentation, automatic presentation generation, document structure Joost Geurts, Stefano Bocconi, Jacco van Ossenbruggen, Lynda Hardman. Towards Ontology-driven Discourse: From Semantic Graphs to Multimedia Presentations, number INS-R0305, May, 2003 This is also published in the 1st International Workshop on Scholarly Hypertext at the fourteenth conference on Hypertext and Hypermedia (HyperText 2003) Automatic hypermedia presentation generation uses an information source semantic network first to select the content and then to compose it in the presentation so that the semantic relations between the information items are conveyed to the user. A hypermedia presentation can be considered as an example of strongly-guided navigation through a hypermedia structure. This can provide a high level interface for semantic network navigation, for example aiding the user in browsing a semantic network of scholarly claims. Automatic Presentation Generation for Scholarly Hypermedia INS-E0305 October 2003 scholarly claims, hypermedia, automatic presentation generation, semantic network, narrative Stefano Bocconi. Automatic Presentation Generation for Scholarly Hypermedia, number INS-E0305, October, 2003 With the rapid growth of interest in the Internet as a means for accessing multimedia presentations for education, entertainment and commerce, comes a corresponding need for systems to supply automatically generated, personalised presentations. Multimedia is a rich and complex genre of resources and the interrelated effects of content, style and structure ensure that automatic presentation generation is in itself a complicated and challenging task. Integrating a model for user personalisation adds a further layer of complication and ensuring that the requirements of user, supplier and platform are all met is a demanding undertaking. This project investigates the influence of information about a user in the process of generating a multimedia presentation. As a result an architecture taking into account these trade-offs is proposed. To evaluate this combined architecture, a framework has been implemented which adjusts colour choices based on the different influences involved. This paper describes the integration of a user modelling approach within an existing system architecture, discusses some of the issues involved in applying user modelling to multimedia presentation generation and describes the prototype implementation and how it addresses some of these issues. Cuypers Meets Users: Implementing a User Model Architecture for Multimedia Presentation Generation INS-E0306 November 2003 multimedia presentation generation; user modelling Suzanne Little, Lynda Hardman. Cuypers Meets Users: Implementing a User Model Architecture for Multimedia Presentation Generation, number INS-E0306, November, 2003 The presentation generation area of hypermedia authoring contains different approaches to address the challenge of the presentation creation process. This paper presents the approach in which the author creates a presentation following the five stages process. These five stages reflect various facets in hypermedia authoring. Semantics in Multi-facet Hypermedia Authoring INS-E0307 November 2003 Hypermedia Authoring, Semantics, Intelligent Support, Narrative Structures Kateryna Falkovych, Frank Nack, Jacco van Ossenbruggen, Lloyd Rutledge. Semantics in Multi-facet Hypermedia Authoring, number INS-E0307, November, 2003 The Semantic Web - ISWC 2006 MultimediaN E-Culture Demonstrator November 2006 951-958 Guus Schreiber, Alia Amin, Mark van Assem, Viktor de Boer, Lynda Hardman, Michiel Hildebrand, Laura Hollink, Zhisheng Huang, Janneke van Kersen, Marco de Niet, Borys Omelayenjko, Jacco van Ossenbruggen, Ronny Siebes, Jos Taekema, Jan Wielemaker, Bob Wielinga. MultimediaN E-Culture Demonstrator. In: The Semantic Web - ISWC 2006, November, 2006 Time-based, media-centric Web presentations can be described declaratively in the XML world through the development of languages such as SMIL. It is difficult, however, to fully integrate them in a complete document transformation processing chain. In order to achieve the desired processing of data-driven, time-based, media-centric presentations, the text-flow based formatting vocabularies used by style languages such as XSL, CSS and DSSSL need to be extended. The paper presents a selection of use cases which are used to derive a list of requirements for a multimedia style and transformation formatting vocabulary. The boundaries of applicability of existing text-based formatting models for media-centric transformations are analyzed. The paper then discusses the advantages and disadvantages of a fully-fledged time-based multimedia formatting model. Finally, the discussion is illustrated by describing the key properties of the example multimedia formatting vocabulary currently implemented in the back-end of our Cuypers multimedia transformation engine. Towards a multimedia formatting vocabulary INS-E0301 June 2003 Document transformation formatting objects multimedia hypermedia Cuypers Jacco van Ossenbruggen, Lynda Hardman, Joost Geurts, Lloyd Rutledge. Towards a multimedia formatting vocabulary, number INS-E0301, June, 2003 This is a revised version of CWI technical report INS-R0201 Semantic Web Workshop, WWW2002 Smart Style on the Semantic Web May 2002 Semantic Web, Device Independent Authoring, Document Engineering, Graphic Design Web publishing systems have to take into account a plethora of Web-enabled devices, user preferences and abilities. Technologies generating these presentations will need to be explicitly aware of the context in which the information is being presented. Semantic Web technology can be a fundamental part of the solution to this problem by explicitly modeling the knowledge needed to adapt presentations to a specific delivery context. We propose the development of a emphSmart Stylelayer which is able to process metadata that describes content and use this metadata to improve the presentation of the content to human users. In the paper, we derive the requirements of such a Smart Style layer by considering Web design from both the document engineering and graphic design perspectives. In addition, design trade-offs made by human designers have to be taken into account for the automated process. After stating the requirements for a Smart Style layer, we discuss to what extent the currently available Web technology can be used and what its limitations are. The limitations are illustrated with examples of potential future extensions. Jacco van Ossenbruggen, Lynda Hardman. Smart Style on the Semantic Web. In: Semantic Web Workshop, WWW2002, May, 2002 The "Document Web" has established itself through the creation of an impressive family of XML and related languages. In addition to this, the "Semantic Web" is developing its own family of languages based primarily on RDF. Although these families were both developed specifically for "the Web", each language family has been developed from different premises with specific goals in mind. The result is that combining both families in a single application is surprisingly difficult. This is unfortunate, since the combination of semantic processing with document processing provides advantages in both directions -- namely using semantic inferencing for more intelligent document processing and using document processing tools for presenting semantic representations to an end-user. In this paper, we investigate this integration problem, focusing on the role of (RDF) semantics in selecting, structuring and styling (XML) content. We analyze the approaches taken by two example architectures and use our analysis to derive a more integrated alternative. Towards Smart Style: Combining RDF Semantics with XML Document Transformations INS-E0303 October 2003 Semantic Web, Semantic Transformations, XSLT, RDF Jacco van Ossenbruggen, Lynda Hardman, Lloyd Rutledge. Towards Smart Style: Combining RDF Semantics with XML Document Transformations, number INS-E0303, October, 2003 Poster Proceedings of the 15th International Conference on World Wide Web Determining User Interests About Museum Collections May 2006 855-856 Edinburgh, Scotland Lloyd Rutledge, Lora Aroyo, Natalia Stash. Determining User Interests About Museum Collections. In: Proceedings of the 15th International Conference on World Wide Web, ACM Press, May, 2006 Already published at ICME 2005 Proceedings of the Conferentie Informatiewetenschap 2005 Using Rhetorical Annotations for Generating Video Documentaries November 2005 We use rhetorical annotations to specify a generation process that can assemble meaningful video sequences with a communicative goal and an argumentative progression. Our annotation schema encodes the verbal information contained in the audio channel, identifying the claims the interviewees make and the argumentation structures they use to make those claims. Based on this schema, we construct a semantic graph which is traversed by rhetoric-based strategies selecting video segments. The selected video segments are edited to form a meaningful video sequence. Stefano Bocconi, Frank Nack, Lynda Hardman. Using Rhetorical Annotations for Generating Video Documentaries. In: Proceedings of the Conferentie Informatiewetenschap 2005, November, 2005 Revised version of CWI Report CS-R9304 ACM Multimedia Structured Multimedia Authoring 1993 283 - 289 Anaheim, CA Lynda Hardman, Guido van Rossum, Dick C.A. Bulterman. Structured Multimedia Authoring. In: ACM Multimedia, 1993 Designing User Interfaces for Hypermedia Experiences in Authoring Hypermedia: Creating Better Presentations 1995 Lynda Hardman. Experiences in Authoring Hypermedia: Creating Better Presentations. In: Designing User Interfaces for Hypermedia, Springer, 1995 Also available on http://www.cwi.nl/$sim$jrvosse/thesis/ Vrije Universiteit With the introduction of the World Wide Web in the early nineties, hypermedia has become the uniform interface to the wide variety of information sources available over the Internet. The full potential of the Web, however, can only be realized by building on the strengths of its underlying research fields. This book describes the areas of hypertext, multimedia, electronic publishing and the World Wide Web and points out fundamental similarities and differences in approaches towards the processing of information. It gives an overview of the dominant models and tools developed in these fields and describes the key interrelationships and mutual incompatibilities. In addition to a formal specification of a selection of these models, the book discusses the impact of the models described on the software architectures that have been developed for processing hypermedia documents. Two example hypermedia architectures are described in more detail: the DejaVu object-oriented hypermedia framework, developed at the VU, and CWI's Berlage environment for time-based hypermedia document transformations. Processing Structured Hypermedia -- A Matter of Style April 10 2001 ISBN 90 6196 502 0 Amsterdam, The Netherlands Jacco van Ossenbruggen. Processing Structured Hypermedia -- A Matter of Style, April 10, 2001 Visual Information Systems '96 System Design for Structured Hypermedia Generation In this contribution we consider the design of a hypermedia information system which not only includes standard functionality of storage and presentation, but also the automatic generation of hypermedia presentations on the basis of a domain-dependent knowledge base. We identify and describe the data sources required and the processes involved. 1996 254-261 Melbourne Marcel Worring, Carel van den Berg, Lynda Hardman. System Design for Structured Hypermedia Generation. In: Visual Information Systems '96, 1996 Media computing - Computational media aesthetics From ontology-based semiosis to computational intelligence June 2002 159-196 Frank Nack. From ontology-based semiosis to computational intelligence. In: Media computing - Computational media aesthetics, Kluwer Academic Publishers, June 2002 Edited by Raphael Troncy, Jacco van Ossenbruggen, Jeff Z. Pan and Giorgos Stamou. Many applications that process multimedia assets make use of some form of metadata that describe the multimedia content. The goals of this document are to explain the advantages of using Semantic Web languages and technologies for the creation, storage, manipulation, interchange and processing of image metadata. In addition, it provides guidelines for Semantic Web-based image annotation, illustrated by use cases. Relevant RDF and OWL vocabularies are discussed, along with a short overview of publicly available tools. Image Annotation on the Semantic Web August 14 2007 Multimedia Semantics Incubator Group Report (XGR) W3C. Image Annotation on the Semantic Web, August 14, 2007 This paper explores the interdependencies between userdetermined interaction in the authoring process of the complex multimedia presentation and the semantic descriptions of tasks and content that allow active system support for the user during the various process phases. The domain we have chosen is the one of the fine arts. The process phases in which we are particularly interested include events such as decision making about form and function, establishing the logical structure, material collection and selection. We introduce the design of an experimental framework for semi-automatic authoring, SampLe. This exploits large mediaaware semantic spaces through semantic-sensitive authoring methods to support users mainly during the early stages of presentation design. We discuss the relations between interactionbased interface design for the system and the underlying semantic structures. Semantic-based Support for the Semi-automatic Construction of Multimedia Presentations May 18 2004 Position paper for the First International Workshop on Interaction Design and the Semantic Web held in conjunction with the World Wide Web Conference, 2004, NYC Kateryna Falkovych, Jana Werner, Frank Nack. Semantic-based Support for the Semi-automatic Construction of Multimedia Presentations, May 18, 2004 Proceedings of Multimedia Modeling 99 (MMM 99) The Use of SMIL: Multimedia Research Currently Applied on a Global Scale This paper describes the current use of the multimedia standard SMIL. SMIL features that relate to active areas of multimedia research are discussed. SMIL current implementation in existing browsers is described. Examples from the Web of SMIL applications representing different types of multimedia are presented. These discussions together provide an overview of how SMIL currently addresses the needs of multimedia distributed on the Web. October 4-6 1999 1-17 Ottawa, Canada Lloyd Rutledge, Lynda Hardman, Jacco van Ossenbruggen. The Use of SMIL: Multimedia Research Currently Applied on a Global Scale. In: Proceedings of Multimedia Modeling 99 (MMM 99), World Scientific, October 4-6, 1999 8 ISSN 1463-662X interChange - Newsletter of the International SGML/XML Users' Group SMIL 2.0 Implementation Overview 1 Like most W3C Recommendations, SMIL 2.0 is being released with strong industrial support. We'll go over the tools that are now available for each member of the SMIL family. Keep in mind, however, that now is changing very fast - this writing is as of early December, 2001. More tools may be available as you read this. March 2002 SMIL 20-24 Lloyd Rutledge. SMIL 2.0 Implementation Overview. interChange - Newsletter of the International SGML/XML Users' Group 8(1), March, 2002 3-540-44178-6 6th European Conference on Research and Advanced Technology for Digital Libraries Dynamic Generation of Intelligent Multimedia Presentations through Semantic Inferencing This paper first proposes a high-level architecture for semi-automatically generating multimedia presentations by combining semantic inferencing with multimedia presentation generation tools. It then describes a system, based on this architecture, which was developed as a service to run over OAI archives - but is applicable to any repositories containing mixed-media resources described using Dublin Core. By applying an iterative sequence of searches across the Dublin Core metadata, published by the OAI data providers, semantic relationships can be inferred between the mixed-media objects which are retrieved. Using predefined mapping rules, these semantic relationships are then mapped to spatial and temporal relationships between the objects. The spatial and temporal relationships are expressed within SMIL files which can be replayed as multimedia presentations. Our underlying hypothesis is that by using automated computer processing of metadata to organize and combine semantically-related objects within multimedia presentations, the system may be able to generate new knowledge by exposing previously unrecognized connections. In addition, the use of multilayered information-rich multimedia to present the results, enables faster and easier information browsing, analysis, interpretation and deduction by the end-user. September 2002 Topia 158-189 Pontifical Gregorian University, Rome, Italy Suzanne Little, Joost Geurts, Jane Hunter. Dynamic Generation of Intelligent Multimedia Presentations through Semantic Inferencing. In: 6th European Conference on Research and Advanced Technology for Digital Libraries, Springer, September, 2002 ISBN: 90-74795-93-5, also available at http://www.cwi.nl/$\sim$lynda/thesis/ University of Amsterdam Modelling and Authoring Hypermedia Documents 1998 Lynda Hardman. Modelling and Authoring Hypermedia Documents, 1998 We discuss access to e-documents from three different perspectives beyond the plain keyword websearch of the entire document. The first one is situation-depending delivery of multimedia documents adapting the preferred form (picture, text, speech) to the available information capacity or need exemplified by documents from the annotated media database of the Rijksmuseum. It goes beyond Quality of Service methods which insist on delivering information in the same form even if that is no longer effective. Secondly, we discuss the use of ontologies to provide access across diverse library categorizations as part of the W3C semantic web. The system translates codes in the one catalogue system into a set of codes in another expanding the potential access to digital heritage knowledge across all library systems in the ontology, such as AAT, WordNet and IconClass. Thirdly, we discuss access to the pictorial contents of paintings by computer vision techniques, here showing in examples of Pieter de Hoogh and Johannes Vermeer which one of the two consistently painting photometric realistic in addition to adhering to the geometric realism as they both did. It is concluded access is the key issue in digital cultural heritage - be it access by situational delivery of e-document of cultural heritage, be it access to diverse knowledge systems, or be it access to the pictorial content of the picture. An integrated multimedia approach to cultural heritage e-documents December 6 2002 4th Intl. Workshop on Multimedia Information Retrieval, in conjunction with ACM Multimedia 2002 Arnold W.M. Smeulders, Lynda Hardman, Guus Schreiber, Jan-Mark Geusebroek. An integrated multimedia approach to cultural heritage e-documents, December 6, 2002 Seventh International World Wide Web Conference GRiNS: A GRaphical INterface for Creating and Playing SMIL Documents April 14-18 1998 Brisbane, Australia Dick C.A. Bulterman, Lynda Hardman, Jack Jansen, K. Sjoerd Mullender, Lloyd Rutledge. GRiNS: A GRaphical INterface for Creating and Playing SMIL Documents. In: Seventh International World Wide Web Conference, April 14-18, 1998 Ielka van der Sluis et al. (eds.): MOG 2007- CTIT Proceedings of the Workshop on Multimodal Output Generation Vocabularies for Description of Accessibility Issues in Multimodal User Interfaces In previous work, we proposed a unified approach for describing multimodal human-computer interaction and interaction constraints in terms of sensual, motor, perceptual and cognitive functions of users. In this paper, we extend this work by providing formalised vocabularies that express human functionalities and anatomical structures required by specific modalities. The central theme of our approach is to connect these modality representations with descriptions of user, device and environmental constraints that influence the interaction. These descriptions can then be used in a reasoning framework that will exploit formal connections among interaction modalities and constraints. The focus of this paper is on specifying a comprehensive vocabulary of necessary concepts. Within the context of an interaction framework, we describe a number of examples that use this formalised knowledge. January, 25-26 2007 117-128 Aberdeen, UK Zeljko Obrenovic, Raphael Troncy, Lynda Hardman. Vocabularies for Description of Accessibility Issues in Multimodal User Interfaces. In: Ielka van der Sluis et al. (eds.): MOG 2007- CTIT Proceedings of the Workshop on Multimodal Output Generation, January, 25-26, 2007 Proceedings of The Eighth International World Wide Web Conference (WWW8) Anticipating SMIL 2.0: The Developing Cooperative Infrastructure for Multimedia on the Web May 1999 SMIL is the W3C recommendation for bringing synchronized multimedia to the Web. Version 1.0 of SMIL was accepted as a recommendation in June. Work is expected to be soon underway for preparing the next version of SMIL, version 2.0. Issues that will need to be addressed in developing version 2.0 include not just adding new features but also establishing SMIL's relationship with various related existing and developing W3C efforts. In this paper we offer some suggestions for how to address these issues. Potential new constructs with additional features for SMIL 2.0 are presented. Other W3C efforts and their potential relationship with SMIL 2.0 are discussed. To provide a context for discussing these issues, this paper explores various approaches for integrating multimedia information with the World Wide Web. It focuses on the modeling issues on the document level and the consequences of the basic differences between text-oriented Web-pages and networked multimedia presentations. Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman, Dick C.A. Bulterman. Anticipating SMIL 2.0: The Developing Cooperative Infrastructure for Multimedia on the Web. In: Proceedings of The Eighth International World Wide Web Conference (WWW8), May, 1999 NWO Research proposal Semi-automatic Hypermedia Presentation Generation Lynda Hardman, Martin Kersten, Geert-Jan Houben, Paul De Bra, Warner R.T. ten Kate, Emile Aarts. Semi-automatic Hypermedia Presentation Generation Based on http://ftp.cwi.nl/CWIreports/INS//INS-E0601.pdf tech report Proceedings of the Fourth International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH2006) Combining Coherence and Adaptation in Discourse-Oriented Hypermedia Generation This paper provides a solution to discourse structure adaptation in the process of automatic hypermedia presentation generation. Existing approaches to discourse structure composition are based on the assumption that a user can comprehend relations between the elements in a discourse structure if the overall structure is semantically coherent. This assumption does not, so far, take into account specific user needs. In this paper we show that although discourse structure composition approaches significantly differ, a general model of the composition process can be derived.Within this general model we identify how adaptation can be applied. We formulate the problem of discourse adaptation with regard to the general model and present our proposed solution. June 2006 274-278 Dublin, Ireland Kateryna Falkovych, Federica Cena, Frank Nack. Combining Coherence and Adaptation in Discourse-Oriented Hypermedia Generation. In: Proceedings of the Fourth International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH2006), June, 2006 1386-3681 The Semantic Web envisions a Web that is both human readable and machine processible. In practice, however, there is still a large conceptual gap between annotated content repositories on the one hand, and coherent, human readable Web pages on the other. To bridge this conceptual gap, one needs to select the appropriate content from the repository, structure and order the material, and design a Web page that effectively conveys the selected content and the chosen structure. In addition to this conceptual gap, there is also a technological gap. On one side of this gap, we find the semantic-oriented technology deployed to build annotated content repositories. This includes RDF, RDF Schema and OWL. On the other side of the gap is the syntax-oriented technology deployed to build Websites. This includes XML, XSLT, CSS, XHTML and SMIL. In this paper, we discuss the conceptual relationships between the world of explicit metadata semantics and the world of Web presentations and their underlying syntactic formats. We also explore to what extent this gap can be bridged automatically, and how current Web technologies can be used to support this process. Structuring and Presenting Annotated Media Repositories INS-E0402 February 2004 Algorithms, Documentation, Design, Experimentation, Human Factors, Standardization, Languages, Theory Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman. Structuring and Presenting Annotated Media Repositories, number INS-E0402, February, 2004 Workshop on Narrative and Interactive Learning Environments Catch me, said the meaning, and turned out to be a hydra August 30 - September 1 2000 This paper presents an integrated framework for new media production in which particular presentations in a variety of forms may be authored and generated using a range of tools interacting with a distributed media repository. The A4SM project demonstrates a number of elements of this overall framework. The emphasis in this work is on the provision of tools and technologies for the manual authorship of linear and interactive media productions. Semi-automated and interactive assembly of a variety of forms of production have been shown to be feasible by a number of research projects, and can be realised by tools interacting with this common repository and environment. Edinburgh, Scotland Frank Nack. Catch me, said the meaning, and turned out to be a hydra. In: Workshop on Narrative and Interactive Learning Environments, August 30 - September 1, 2000 This is also published as Technical Report INS-E0305. Automatic hypermedia presentation generation uses an information source semantic network first to select the content and then to compose it in the presentation so that the semantic relations between the information items are conveyed to the user. A hypermedia presentation can be considered as an example of strongly-guided navigation through a hypermedia structure. This can provide a high level interface for semantic network navigation, for example aiding the user in browsing a semantic network of scholarly claims. Automatic Presentation Generation for Scholarly Hypermedia August 30 2003 1st International Workshop on Scholarly Hypertext at the fourteenth conference on Hypertext and Hypermedia (HyperText 2003) Stefano Bocconi. Automatic Presentation Generation for Scholarly Hypermedia, August 30, 2003 Author's Master Thesis, Technische Universiteit Eindhoven Automatically generating presentations from content available on the World Wide Web, mainly through multimedia databases, is made possible with the advent of the Semantic Web and the increasing availability of multimedia repositories. The Semantic Web enables machines to work with assumptions on the meaning and interrelations of multimedia content. This makes it possible to present information in a more coherent manner, paving the way for the field of automatic multimedia presentation generation. Among the different issues to be solved in this field is layout generation. When combining separate parts together into a single presentation, new options and dangers present themselves. In this report we look at two options that present themselves and the related dangers. These options are analysis of the presentation tree in order to find interesting structures and adding to the system the option to use emphasis elements when presenting media elements. Transforming Abstract Document Structure to a Hypermedia Presentation INS-E0407 July 2004 Presentation generation; structure analysis Rob van Velthoven. Transforming Abstract Document Structure to a Hypermedia Presentation, number INS-E0407, July, 2004 1386-3681 We describe our experimental rhetoric engine Vox Populi that generates biased video-sequences from a repository of video interviews and other related audio-visual web sources. Users are thus able to explore their own opinions on controversial topics covered by the repository. The repository contains interviews with United States residents stating their opinion on the events occurring after the terrorist attack on the United States on the 11th of September 2001. We present a model for biased documentary statements, such as interviews, and explain in detail how this model facilitates the automatic generation of rhetorical arguments on a micro-level. We outline the required representations of relevant rhetorical structures and the way they can be processed. The processes are described via examples generated by our experimental engine. The first example shows how to logically counter an opinion using semantics contained in the audio tracks from the database, while the second example describes the generation of an emotional counterargument using visual material. VOX POPULI: Automatic Generation of Biased Video Sequences INS-E0405 June 2004 Media semantics, media rhetorics, automated video editing, multimedia presentation generation, video documentaries Stefano Bocconi, Frank Nack. VOX POPULI: Automatic Generation of Biased Video Sequences, number INS-E0405, June, 2004 no NASH NWO In this paper, we report our experiences with the use of Semantic Web technology for annotating digital video material. Web technology is used to transform a large, existing video ontology embedded in an annotation tool into a commonly accessible format. The recombination of existing video material is then used as an example application, in which the video metadata enables the retrieval of video footage based on both content descriptions and cinematographic concepts, such as establishing and reaction shots. The paper focuses on the practical issues of porting ontological information to the Semantic Web, the multimedia-specific issues of video annotation, and requirements for Semantic Web query and access patterns. It thereby explicitly aims at providing input to the two new W3C Semantic Web Working Groups (Best Practices and Deployment; Data Access). Video on the Semantic Web - Experiences with Media Streams INS-E0404 2004 Video, Search, Query, Rules, Ontology Porting, Media Streams Joost Geurts, Jacco van Ossenbruggen, Lynda Hardman, Marc Davis. Video on the Semantic Web - Experiences with Media Streams, number INS-E0404, 2004 978-3-540-73288-4 Proceedings of the 12th International Conference on Human-Computer Interaction User Interaction with User-Adaptive Information Filters July 22-27 2007 Beijing, China Henriette Cramer, Vanessa Evers, Maarten Van Someren, Bob Wielinga, Sam Besselink, Lloyd Rutledge, Natalia Stash, Lora Aroyo. User Interaction with User-Adaptive Information Filters. In: Proceedings of the 12th International Conference on Human-Computer Interaction, July 22-27, 2007 Semantic annotations of media repositories make relationships among the stored media and relevant concepts explicit. However, these relationships and the media they join are not directly presentable as hypermedia. Previous work shows how clustering over the annotations in the repositories can determine hypermedia presentation structure. Here we explore the application of different clustering techniques to generating hypermedia interfaces to media archives. This paper also describes the effect of each type of clustering on the end user's experience. We then generalize and unify these techniques with the use of proximity measures in further improving generated presentation structure. Clustering Semantics for Hypermedia Presentation INS-E0409 November 2004 Hypermedia, Presentation Generation, Clustering, Semantics, Style, Document Structure Martin Alberink, Lloyd Rutledge, Lynda Hardman, Mettina Veenstra. Clustering Semantics for Hypermedia Presentation, number INS-E0409, November, 2004 This is a revised version of CWI technical report INS-E0604 The Semantic Web - ISWC 2006 /facet: A Browser for Heterogeneous Semantic Web Repositories Facet browsing has become popular as a user friendly interface to data repositories. The Semantic Web raises new challenges due to the heterogeneous character of the data. First, users should be able to select and navigate through facets of resources of any type and to make selections based on properties of other, semantically related, types. Second, where traditional facet browsers require manual configuration of the software, a semantic web browser should be able to handle any RDFS dataset without any additional configuration. Third, hierarchical data on the semantic web is not designed for browsing: complementary techniques, such as search, should be available to overcome this problem. We address these requirements in our browser, /facet. Additionally, the interface allows the inclusion of facet-specific display options that go beyond the hierarchical navigation that characterizes current facet browsing. /facet is a tool for Semantic Web developers as an instant interface to their complete dataset. The automatic facet configuration generated by the system can then be further refined to configure it as a tool for end users. The implementation is based on current Web standards and open source software. The new functionality is motivated using a scenario from the cultural heritage domain. November 2006 facet browsing, heterogeneous querying, cross relations 272-285 Michiel Hildebrand, Jacco van Ossenbruggen, Lynda Hardman. /facet: A Browser for Heterogeneous Semantic Web Repositories. In: The Semantic Web - ISWC 2006, November, 2006 11 based on http://ftp.cwi.nl/CWIreports/INS//INS-E0308.pdf IEEE MultiMedia That Obscure Object of Desire: Multimedia Metadata on the Web (Part I) 4 October - December 2004 Semantic Web, metadata production, multimedia production process, XML, XML Schema, RDF, RDF Schema, MPEG-4, MPEG-7, MPEG-21 38-48 Jacco van Ossenbruggen, Frank Nack, Lynda Hardman. That Obscure Object of Desire: Multimedia Metadata on the Web (Part I). IEEE MultiMedia 11(4), October - December, 2004 5 Revised version of CWI Report CS-R9306 Hypermedia Journal The Amsterdam Hypermedia Model: Extending Hypertext to Support REAL Multimedia 1 July 1993 47-69 Lynda Hardman, Dick C.A. Bulterman, Guido van Rossum. The Amsterdam Hypermedia Model: Extending Hypertext to Support REAL Multimedia. Hypermedia Journal 5(1), July, 1993 The International Conference on Multimedia Modeling Implementing Adaptability in the Standard Reference Model for Intelligent Multimedia Presentation Systems This paper discusses the implementation of adaptability in environments that are based on the Standard Reference Model for Intelligent Multimedia Presentation Systems. This adaptability is explored in the context of style sheets, which are represented in such formats as DSSSL. The use of existing public standards and tools for this implementation of style sheet-based adaptability is described. The Berlage environment is presented, which integrates these standards and tools into a complete storage-to-presentation hypermedia environment. The integration of the SRM into the Berlage environment is introduced in this work. This integration illustrates the issues involved in implementing adaptability in the model. October 12-15 1998 12-20 Lloyd Rutledge, Lynda Hardman, Jacco van Ossenbruggen, Dick C.A. Bulterman. Implementing Adaptability in the Standard Reference Model for Intelligent Multimedia Presentation Systems. In: The International Conference on Multimedia Modeling, October 12-15, 1998 1-871516-02-1 Hypertext: Theory in Practice Lost in Hyperspace: Cognitive Mapping and Navigation in a Hypertext Environment March 1988 105-125 Aberdeen, UK Deborah M. Edwards, Lynda Hardman. Lost in Hyperspace: Cognitive Mapping and Navigation in a Hypertext Environment. In: Hypertext: Theory in Practice, March, 1988 Poster The 5th International Semantic Web Conference Interactive User Profiling in Semantically Annotated Museum Collections November 2006 Athens, GA, USA Lloyd Rutledge, Lora Aroyo, Natalia Stash. Interactive User Profiling in Semantically Annotated Museum Collections. In: The 5th International Semantic Web Conference, November, 2006 Proceedings of the 10th International Multi-Media Modelling Conference 2004 SampLe: Towards a Framework for System-supported Multimedia Authoring January 5-7 2004 362 Brisbane, Australia Kateryna Falkovych, Frank Nack, Jacco van Ossenbruggen, Lloyd Rutledge. SampLe: Towards a Framework for System-supported Multimedia Authoring. In: Proceedings of the 10th International Multi-Media Modelling Conference 2004, January 5-7, 2004 5 Journal of Intelligent Systems An Interactive Multimedia Business Game 2-4 The playing of computer-based business games has changed little with recent technological improvements. Teams make decisions which are written down on paper and handed to a tutor who then has to enter these into the computer. To make it a richer and more real-life experience we have brought a business simulation into a multimedia environment. In order to create a sense of the passing of time the simulation is continuous, and players can enter input values throughout the game. Important parameters, such as product sales, are displayed as continuously updated graphs. Background information such as news items is provided in multimedia form. Players also have direct access to online tools, such as spreadsheets, from the game environment. We describe the prototype built to illustrate our approach. 1995 139-150 Lynda Hardman, Guido van Rossum, Aart van Bolhuis. An Interactive Multimedia Business Game. Journal of Intelligent Systems 5(2-4), 1995 ICCC/IFIP Conference -- Electronic Publishing '98 Addressing Publishing Issues with Hypermedia Distributed on the Web The content and structure of an electronically published document can be authored and processed in ways that allow for flexibility in presentation on different environments for different users. This enables authors to craft documents that are more widely presentable. Electronic publishing issues that arise from this separation of document storage from presentation include (1) respecting the intent and restrictions of the author and publisher in the document's presentation, and (2) applying costs to individual document components and allowing the user to choose among alternatives to control the price of the document's presentation. These costs apply not only to the individual media components displayed but also to the structure created by document authors to bring these media components together as multimedia. A collection of ISO standards, primarily SGML, HyTime and DSSSL, enable the representation of presentation-independent documents and the creation of environments that process them for presentation. SMIL is a W3C format under development for hypermedia documents distributed on the World Wide Web. Since SMIL is SGML-compliant, it can easily be incorporated into SGML/HyTime and DSSSL environments. This paper discusses how to address these issues in the context of a presentation-independent hypermedia storage. It introduces the Berlage environment, which uses SGML, HyTime, DSSSL and SMIL to store, process, and present hypermedia data. This paper also describes how the Berlage environment can be used to enforce publisher restrictions on media content and to allow users to control the pricing of document presentations. Also explored is the ability of both SMIL and HyTime to address these issues in general, enabling SMIL and HyTime systems to consistently process documents of different document models authored in different environments. April 20-22 1998 Hypermedia, Electronic Payment, Authoring, SGML, HyTime, DSSSL, SMIL, XML. 78-93 Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman, Dick C.A. Bulterman. Addressing Publishing Issues with Hypermedia Distributed on the Web. In: ICCC/IFIP Conference -- Electronic Publishing '98, April 20-22, 1998 Museums and the Web 2007 Personalized Museum Experience: The Rijksmuseum Use Case April 11-14 2007 San Francisco, USA Lora Aroyo, Rogier Brussee, Lloyd Rutledge, Peter Gorgels, Natalia Stash, Yiwen Wang. Personalized Museum Experience: The Rijksmuseum Use Case. In: Museums and the Web 2007, April 11-14, 2007 Proceedings of the 2nd Conference on Computational Semiotics for Games and New Media The emergence of the labyrinth: Intrinsic computational aesthetic form September 2-4 2002 3-6 Augsburg, Germany Craig Lindley, Frank Nack, Elisabeth Andre, Andy Clarke, Grethe Mitchell, Clive Fencott. The emergence of the labyrinth: Intrinsic computational aesthetic form. In: Proceedings of the 2nd Conference on Computational Semiotics for Games and New Media, September 2-4, 2002 In this article we argue that for the automatic generation of adaptive multimedia presentations we are in need of expandable, adaptable style descriptions which provide both high-level conceptual and low-level feature extraction information. Only the combination of both facilitates the retrieval of adequate material and its user-centred presentation. We discuss the requirements for an adaptable Web-based environment for museums presenting visual artefacts. We then present the framework of our prototype multimedia generation environment which transforms a high-level user query into a concrete multimedia final-form encoding that is playable on an end-users' platform. We describe the underlying architecture and provide a working example. The Role of High-level and Low-level Features in Semi-automated Retrieval and Generation of Multimedia Presentations Presentation Geneneration INS-R0108 June 2001 Multimedia semantics, feature grammars, multimedia presentation generation, multimedia retrieval Frank Nack, Menzo Windhouwer, Eric Pauwels, Michele Huijberts, Lynda Hardman. The Role of High-level and Low-level Features in Semi-automated Retrieval and Generation of Multimedia Presentations Presentation Geneneration, number INS-R0108, June, 2001 24 IEEE Software Open Source Software: All You Do Is Put It Together 5 The authors propose an infrastructure for rapidly prototyping applications from open source software components. The Adaptable Multi-Interface Communicator infrastructure (AMICO) is based on ideas of middleware platforms for component integration, but it focuses on pragmatic aspects of OSS integration, often absent from many existing integration platforms. The authors also identify the key requirements of middleware for rapid prototyping with OSS components and illustrate their approach through two examples in complex scenarios. 2007 86-95 Zeljko Obrenovic, Dragan Gasevic. Open Source Software: All You Do Is Put It Together. IEEE Software 24(5), 2007 1-58113-893-8 12th ACM International Conference on Multimedia - Doctoral Symposium Semantic-Aware Automatic Video Editing One of the challenges of multimedia applications is to provide user-tailored access to information encoded in different media. Particularly, previous research has not yet fully explored how to automatically compose different video segments according to a communicative goal. We propose a rhetoric-based method to support the selection and automatic editing of user-requested content from video footage. The method is applied to the domain of video documentaries to create biased sequences about a user selected subject. October 2004 971-972 Stefano Bocconi. Semantic-Aware Automatic Video Editing. In: 12th ACM International Conference on Multimedia - Doctoral Symposium, October, 2004 Proceedings of the International Conference on Multimedia Modeling 2000 (MMM00) Inter-dimensional Hypermedia Communicative Devices for Rhetorical Structure Hypermedia communicative devices are patterns of hypermedia presentation structure with specific affects on the user. Some devices have been determined for affecting the user with abstract authoring constructs, such as rhetorical structure. However, these devices each typically focus on particular aspects of the structure of hypermedia presentation. This paper presents a framework for hypermedia communicative devices that simultaneously affects multiple dimensions of presentation structure. In discussing this, we focus on the document's rhetorical structure as the input to be communicated through these devices. November 13-15 2000 Rhetoric, adaptation, presentation generation, hyper-media, Topia 89-105 Nagano, Japan Lloyd Rutledge, Jim Davis, Jacco van Ossenbruggen, Lynda Hardman. Inter-dimensional Hypermedia Communicative Devices for Rhetorical Structure. In: Proceedings of the International Conference on Multimedia Modeling 2000 (MMM00), November 13-15, 2000 Rhetorical structure theory (RST) provides a model of textual function based upon rhetoric. Initially developed as a model of text coherence, RST has been used extensively in text generation research, and has more recently been proposed as a basis for multimedia presentation generation. This paper investigates the use of RST for generating video presentations having a rhetorical form, using models of the rhetorical roles of video components, together with rules for selecting components for presentation on the basis of their rhetorical functions. An RST model can provide a predefined link structure providing viewers with options for obtaining and dynamically modifying rhetorically coherent video presentations from video archives and databases. The use of an RST analysis for interactive presentation generation may provide a more powerful rhetorical device than conventional linear video presentation. Conversely, making alternative RST analyses of the same video data available to users can have the effect of encouraging closer and more independent viewer analysis of the material, and discourage taking any particular rhetorical presentation at face value. The application of rhetorical structure theory to interactive news program generation from digital archives INS-R0101 January 2001 Interactive presentation generation, rhetorical structure theory, news, video, video archive Craig Lindley, Jim Davis, Frank Nack, Lloyd Rutledge. The application of rhetorical structure theory to interactive news program generation from digital archives, number INS-R0101, January, 2001 iX: Magazin fur professionelle Informationstechnik SMIL - Synchronized Multimedia Integration Language: Moving to the beat 10 October 1999 Lloyd Rutledge. SMIL - Synchronized Multimedia Integration Language: Moving to the beat. iX: Magazin fur professionelle Informationstechnik, number 10, October, 1999 IEEE MultiMedia Pictures at an exhibition This column envisaging the impact of information technology for museum environments. April - June 2002 14-17 Frank Nack. Pictures at an exhibition. IEEE MultiMedia, April - June, 2002 This article advocates the use of SGML technology for the creation, dissemination and display of Web documents. It presents a software architecture that allows for defining the operational interpretation of arbitrary document types by means of style sheets, written in a scripting language. Our approach has been motivated by a desire to extend the functionality of the Web with support for multimedia and active documents. Although growing in complexity, HTML is still lacking in functionality. We prefer a more flexible and generic approach, as enabled by the employment of SGML. After a brief introduction of SGML, we will illustrate how our approach accommodates (extensions of) HTML as well as arbitrary SGML documents containing multimedia data such as video and audio. We will then briefly sketch the software components used in the realization of our approach and discuss some topics for further research. Web Applications and SGML 1996 51-62 EP96 Jacco van Ossenbruggen, Anton Eliens, Bastiaan Schonhage. Web Applications and SGML, 1996 The paper describes the extension of a hypermedia class library with music as a new component type, but will focus on the development of a software wrapper object as an application programmers interface to the Csound software sound synthesis program. This wrapper provides the flexible, interactive and object-oriented interface needed by a hypermedia system. Additionally, some consequences of the fundamental difference between static and time-based media will be discussed. Music in Time-based Hypermedia 224-270 echt94 Jacco van Ossenbruggen, Anton Eliens. Music in Time-based Hypermedia Proceedings of the International Conference on 3D and Multimedia on the Internet, WWW and Networks, 17-18 April 1996, Bradford. Animating the Web -- An SGML-based Approach April 1996 DV:Animating-BOOK Anton Eliens, Jacco van Ossenbruggen, Bastiaan Schonhage. Animating the Web -- An SGML-based Approach. In: Proceedings of the International Conference on 3D and Multimedia on the Internet, WWW and Networks, 17-18 April 1996, Bradford., British Computer Society, April, 1996 Proceedings are published as Technical Report CS-98-01 of Aalborg University, Denmark Proceedings of the 4th Workshop on Open Hypermedia Systems The Role of XML in Open Hypermedia Systems June 19-20 1998 The World Wide Web Consortium has recently issued XML 1.0 as a Recommendation. XML, the Extensible Markup Language, is a data format for structured document interchange on the Web. In this paper, we put forward the position that XML, together with (future) standards based on XML, will play an important role in the development of open hypermedia systems. Many of objectives of the XML community are similar to the objectives of the open hypermedia systems (OHS) community. This paper discusses these similarities, but also stresses the differences between the architecture and protocol-oriented approach which is typical for the OHS research community versus the document oriented approach of the XML community. Hypertext '98 Pittsburgh, PA ohs4 Jacco van Ossenbruggen, Anton Eliens, Lloyd Rutledge. The Role of XML in Open Hypermedia Systems. In: Proceedings of the 4th Workshop on Open Hypermedia Systems, June 19-20, 1998 Euromedia 96: Telematics in a Multimedia Environment, December 19-21, London. United Kingdom Support for Business Process Redesign -- Simulation, Hypermedia and the Web A major success factor of business process redesign projects is the extent to which the organisation members themselves - both management and staff - are actively involved in the redesign project. One way to assess different redesign alternatives with respect to the project goals is to model these alternatives and subject them to a simulation study. Visualization and animation of the simulation models can provide insight into both the structure and the dynamics of the modelled alternatives. Effectively presented, visualization and animation enhance the involvement of the project members in the assessment and validation of the proposed redesigns. To support communication among all participants in the BPR project, we advocate an integrated presentation of redesign alternatives, visualized simulation experiments and the results of simulation studies in a single active document. In this paper we describe a framework enabling such integrated presentations by allowing the embedding of simulation models and experiments in hypermedia documents. December 1996 Business Process Redesign, Logistics-based Business Modelling, Discrete Event Simulation, Hypermedia, Active Documents, Web. 193-200 Anton Eliens, Frank Niessink, Bastiaan Schonhage, Jacco van Ossenbruggen, Paul Nash. Support for Business Process Redesign -- Simulation, Hypermedia and the Web. In: Euromedia 96: Telematics in a Multimedia Environment, December 19-21, London. United Kingdom, December, 1996 The Internet in 3D -- Information, Images and Interaction Animating the Web -- An SGML-based Approach The DejaVu framework offers access to the Web as one of the facilities in a collection of software components for developing multimedia user interfaces and hypermedia applications. It provides a vehicle for experimenting with true hypermedia extensions to the Web. Our approach, which is essentially object-oriented, allows for integrating the Web in applications and vice versa, for composing a Web browser with support for sound synthesis, digital video, 3D graphics and virtual reality. On a software level this is realized by extending HTML to allow for embedded scripts, written in a script language that is extensible with application-defined functionality. However, in addition to this rather low-level provision for active documents, our Web browser component allows for SGML-defined document types capturing the syntax and semantics of multimedia extensions. This paper gives a brief overview of the various components constituting the DejaVu framework and discusses briefly the architectural notions underlying our framework. We will show how to employ these components to develop a Web browser with (client-side) support for animation, multimedia and virtual reality. By giving some examples, we will illustrate our solution to defining the (browsing) semantics of arbitrary document types by means of script-based style sheets. As a conclusion we will discuss the requirements that must be met for developing a HyTime compliant browser for the Web. 1997 75-96 Anton Eliens, Jacco van Ossenbruggen, Bastiaan Schonhage. Animating the Web -- An SGML-based Approach. In: The Internet in 3D -- Information, Images and Interaction, 1997 6 SIGLINK Newsletter Style Sheet Languages for Hypertext 3 October 1997 16 - 20 Jacco van Ossenbruggen, Lloyd Rutledge, Lynda Hardman, Anton Eliens. Style Sheet Languages for Hypertext. SIGLINK Newsletter 6(3), October, 1997 30 The Seventh International World Wide Web Conference (WWW7) Requirements for Multimedia Markup and Style Sheets on the World Wide Web The unequivocal acceptance of Cascading Style sheets by all major browser vendors clearly indicates the need for separating content and style issues in HTML documents. However, in search of ever-richer content, providers rely on platform-specific plug-ins and browser-specific extensions such as dynamic HTML and scripting to achieve the desired (multimedia) functionality. We consider such an ad-hoc approach to delivering multimedia content as inadequate, and henceforth propose a multimedia document markup language (SMIL, which has recently been submitted to W3C) and discuss the requirements for an associated style sheet language supporting temporal and spatial constraints, hyperlinking among continuous media objects and adaptivity with respect to Quality of Service. In this paper we propose extensions to current style sheet languages to meet the requirements imposed by the capability to specify the presentation of multimedia content in a declarative manner. April Computer Networks and ISDN Systems 1998 Synchronized hypermedia; Style sheets; SMIL 694-696 Jacco van Ossenbruggen, Anton Eliens, Lloyd Rutledge, Lynda Hardman. Requirements for Multimedia Markup and Style Sheets on the World Wide Web 30. In: The Seventh International World Wide Web Conference (WWW7), Elsevier Science B.V., April, 1998 1994 Groningen Student Conference on Computer Science Music in Time-based Hypermedia Hypermedia can be regarded as a merge between multimedia and hypertext. The paper will describe the extension of a hypermedia system with music as a new document type, and the development of a software wrapper to give an existing, music generating program the flexible, interactive and object-oriented interface needed by a hypermedia system. Additionally, some consequences of the fundamental difference between static and time-based media will be discussed. January 1994 127-130 Jacco van Ossenbruggen. Music in Time-based Hypermedia. In: 1994 Groningen Student Conference on Computer Science, January, 1994 Proceedings of the Fourth International World Wide Web Conference -- The Web Revolution Bringing Music to the Web To reduce the amount of resources needed for high quality music on the WWW, we advocate the use of client-side sound synthesis techniques. The paper will describe the design of the hymne class library, which is used by the hush web widget to synthesize the sound of musical scripts embedded in HTML pages. December 1995 309-314 Jacco van Ossenbruggen, Anton Eliens. Bringing Music to the Web. In: Proceedings of the Fourth International World Wide Web Conference -- The Web Revolution, OReilly and Associates, Inc., December, 1995 Proceedings of the 6th International World Wide Web Conference -- Everone, Everything Connected Jamming (on) the Web The Web has become a dominant medium for the dissemination of information and, more recently, for collaborative work as well. The focus has been mainly on textual and graphical information, hardly addressing topics related to musical information. We present a framework that makes musical works accessible for Web users by offering high level support for the display of musical material on the Web as well as for live jam sessions. Our framework consists of a browser plug-in that supports the display and editing of scores as well as playing scores by connecting to a shared MIDI server. To participate in a jam session, clients of the MIDI server can also send data in real time, for instance by using a keyboard embedded in a Web page. We discuss the issues involved in displaying musical material on the Web and we sketch the technical architecture of our framework. April 7-11 1997 419-426 Santa Clara, California USA Anton Eliens, Martijn van Welie, Jacco van Ossenbruggen, Bastiaan Schonhage. Jamming (on) the Web. In: Proceedings of the 6th International World Wide Web Conference -- Everone, Everything Connected, OReilly and Associates, Inc., April 7-11, 1997 Andy Clarke Andrew Dillon Ray McAleese Jean-Pierre Balbe Catherine Green Jana Werner Jos Taekema Andreas Mauthe Martin Alberink Lynda Hardman Nico Roos Steffen Staab Mettina Veenstra Martin Kersten Jacco van Ossenbruggen Klein, M. Kurt Piersol Anton Eliens Martin Hoffernig Mark van Assem Umberto Straccia Imad Saleh Natalia Stash Marc Davis Maarten Van Someren Dragan Gasevic Christian Halaschek-Wiener Thomas Sikora Brian Bailey Christine Vanoirbeek Deborah M. Edwards Jana Dittmann Sam Besselink Frank Nack Carel van den Berg Giorgos Stamou Martijn van Welie W3C Jan Wielemaker Rajiv Mehrotra Peter Gorgels Susanne G. Loeber N. Streitz William van Dieten Julien Ducret Adam T. Lindsay Aart van Bolhuis Samuel Cruz-Lara Michael Hausenblas Stephane Sire Jane Hunter Amit Manniesing Clive Fencott Guus Schreiber Rudolf Schlatte Ramesh Jain Craig Lindley Lora Aroyo Satyan Ramlal Brussee Frank Niessink Jim Davis Chitra Dorai Richard Arndt K. Sjoerd Mullender Kateryna Falkovych Alain Lelu Janneke van Kersen Stefano Bocconi Vassilis Tzouvaras Borys Omelayenko Oscar Rosell Martinez Henriette Cramer Menzo Windhouwer Emile Aarts Dick C.A. Bulterman William I. Grosky Marc Nanard Victor de Boer Alex R. van Ballegooij Alia Amin Arnold W.M. Smeulders Jaap van den Herik et al. Philip Hofmair Annemiek Teesing Patrick J. Deunhouwer Yassine Rekik Lambert Schomaker Raphael Troncy W. Schuler Dylan M. Jones Virginia Dignum Lora M. Aroyo Bastiaan Schonhage Boban Arsenijevic J. Hannemann Floris Wiesman Grethe Mitchell Paul De Bra Jan van Leeuwen John Vince Rae Earnshaw Jack Jansen Martha Sabou Zeljko Obrenovic Yulia Bachvarova Brian S. Sharratt R. Winder Bob Wielinga F. Nack Heiner Stuckenschmidt Lloyd Rutledge Timo Kouwenhoven Borys Omelayenjko Joost Geurts Federica Cena Jimmy Wang Eric Pauwels Karolina Czajka Jan-Mark Geusebroek Herbert Zettl Frank Cornelissen Leen Breure C. Leung Wolfgang Putz Marcel Worring Omelayenko, B. Svetha Venkatesh Lynda Hardman et al. Pablo Cesar Ronny Siebes Katya Falkovych Marco de Niet Rogier Brussee Viktor de Boer Warner R.T. ten Kate Suzanne Little Yiwen Wang Michiel Hildebrand Zhisheng Huang Michele Huijberts Rob van Velthoven Craig A. Lindley Stanislav Pokraev Jeff Z. Pan Audrey Tam Elisabeth Andre Herre van Oostendorp Marie-France van Orsouw Laura Hollink Werner Bailer Katharina Schwarz Paul Nash Brigitte Kerherve Patrick Schmitz Anna Tordai Vanessa Evers Guido van Rossum Geert-Jan Houben Robert van Liere Kluwer Academic Publishers British Computer Society Erlbaum Springer IOS Press Prentice Hall VU/CWI/UvA Springer-Verlag New York, Inc. World Scientific CWI Springer-Verlag KU and CWI Springer Berlin / Heidelberg Elsevier Science B.V. ACM Press CWI and JRS Editions HERMES OReilly and Associates, Inc.