The Cuypers Multimedia Transformation Engine

Cuypers is a research prototype system, developed to experiment with the automatic generation of Web-based presentations as an interface to semi-structured multimedia databases. It differs from XSL-based transformations in that it explores a set of abstractions, both on the document and on the presentation level, that are geared towards interactive, time-based and media centric presentations, rather than presentations that are based on text-flow.

Cuypers deploys a set of extensible, high-level transformation rules specified in Prolog, exploiting Prolog's built-in support for backtracking. It facilitates easy feedback between the higher and lower level parts of the transformation process by executing both within the same environment. Instead of a strict separation between the transformation engine and the constraint solver, our system is implemented in ECLⁱPS^e, a Prolog environment with embedded constraint solvers. This allows Cuypers to backtrack immediately when the transformation process applies a rule that generates a multimedia fragment with insolvable constraints.

We are currently experimenting with implementing a formatting vocabulary that is geared towards time-based and media centric formatting. To implement these formatting objects, we use the Logtalk object-oriented extension to Prolog. Logtalk allows us to cleanly model the object behavior of hypermedia formatting objects (including inheritance, information hiding and predicate polymorphism) in a constraint logic programming environment.

Generating multimedia presentations is an inherently complex and knowledge-intensive task. We aim at explicit modeling the required knowledge and are researching the use of Semantic Web technology to encode it in a common, interoperable format. For example, the DISC demo uses Sesame to store and access the knowledge needed during the transformation. This includes RDF-encoded metadata associated with the underlying multimedia database, ontological knowledge representing narrative and discourse structures and knowledge about the characteristics and behavior of the various media types used.

On-line Demo

An on-line demo of Cuypers, based on the multimedia database of the Rijksmuseum Amsterdam is available from http://homepages.cwi.nl/~media/demo/cuypers/.

Implementation

The core of the Cuypers engine (cuypers/engine/) is implemented using:

• Logtalk, an open source object-oriented extension to Prolog
• The ECLⁱPS^e Constraint Logic Programming System

The Cuypers web interface (cuypers/www/) is built on top of:

• The Sesame Open Source RDF Schema-based Repository and Querying facility
• The Apache Cocoon XML/XSLT publishing framework
• The Apache Tomcat Java servlet container

Resources

• Cuypers source code documentation is available on-line.
• Example HFO serializations: the "Lincoln" and "Chiaroscuro" demo.
• XSLT transformation sheets for converting HFOs to SMIL 2.0, SMIL 1.0 and HTML+TIME.
• Java classes implementing a Sesame/XSLT connection as an XSLT extension in Xalan.

Publications

Publications about Cuypers include:

• High level description of the Cuypers architecture: Jacco van Ossenbruggen, Joost Geurts, Frank Cornelissen, Lloyd Rutledge, and Lynda Hardman. Towards Second and Third Generation Web-Based Multimedia. In: The Tenth International World Wide Web Conference (pages 479-488), May 1-5, 2001, Hong Kong, application/pdf text/html Additional comments: this is a revised version of CWI technical report INS-R0025.
  
• Use of Semantic Web technology within Cuypers:
  • Joost Geurts, Stefano Bocconi, Jacco van Ossenbruggen, and Lynda Hardman . Towards Ontology-driven Discourse: From Semantic Graphs to Multimedia Presentations . In: Second International Semantic Web Conference (ISWC2003), October 20-23, 2003, Sanibel Island, Florida, USA, Note: To be published. Additional comments: This is a revised version of CWI technical report INS-R0305 .
• Use of hypermedia formatting objects within Cuypers:
  • Jacco van Ossenbruggen, Joost Geurts, Lynda Hardman, and Lloyd Rutledge . Towards a Formatting Vocabulary for Time-based Hypermedia . In: The Twelfth International World Wide Web Conference (pages 384-393), ACM Press, Budapest, Hungary, May 20-24, 2003
  • Oscar Rosell Martinez: Design dependencies within the automatic generation of hypermedia presentations . Master's Thesis, Technical University of Catalonia, June 30, 2002 application/pdf Note: Published as CWI technical report INS-R0205
• Use of constraint logic programming within Cuypers:
  • Joost Geurts, Jacco van Ossenbruggen, and Lynda Hardman. Application-Specific Constraints for Multimedia Presentation Generation . In: Proceedings of the International Conference on Multimedia Modeling 2001 (MMM01) (pages 247-266), November 5-7, 2001, CWI, Amsterdam, The Netherlands. Additional comments: this is a revised version of CWI Report INS-R0107.
    
  • A detailed version of the previous paper: Joost Geurts. Constraints for Multimedia Presentation Generation. Master's Thesis, University of Amsterdam, 2002.

$Id: index.html,v 1.23 2004/10/07 11:29:00 jrvosse Exp $