Authoring Multimedia Documents for the Web
Opera Project (www.inrialpes.fr/opera)
The WWW consortium defined the first version of Smil [W3C SMIL 1998]: a
"tag-based" standard to describe synchronized multimedia documents on the web.
Smil 1.0 allows to describe multimedia documents which are mainly predictive
: the temporal organization of the document is known before its presentation
(only some durations of continuous object may be unknown) and readers'
interactions can only trigger hyperlinks either inside or outside the
document. The second version of Smil [W3C SMIL Boston 2000] (whose
recommendation is expected in june 2000) will merge predictive and
unpredictive behaviors, to be able to describe synchronization such as
starting a video when a user click occurs.
These two standards will make the delivery of synchronized multimedia
documents on the web possible, but this delivery will be really effective if
and only if friendly authoring tools are designed to help authors to handle
the intrinsic complexity of the temporal synchronization specification task.
Indeed, two features make this one very complex: the dynamic nature of a
multimedia document (its presentation is closer to a program execution than to
a static document visualization) and the incremental process of the design
task (it is a cyclic action/visualization process). Thus the challenging
issues are :
- finding an equivalent to the WYSIWYG paradigm used in word-processor
(not valuable in a dynamic context) but avoiding to design something too
close to a programming environment interface which will be only used by
- helping the author to propagate local changes (change of an object
duration for instance) on the whole document without loosing some existing
- finding technics to analyze the document to detect well-known temporal
errors in every temporal synchronization tasks like deadlocks (waiting for
an event that will never occurs) and integrating these technics (used
mainly in real-time programming environments) in a document authoring
Solutions proposed by the Opera project to overcome these difficulties are
based on the following principles:
- providing the author with multiple views interface and direct
manipulation capabilities on these views [JRT 1998c]. Multiple views are
necessary to show the different dimensions of the document (logical,
spatial, temporal and hyperlink dimensions) and its execution. Direct
manipulation is a friendly way to interact with objects : for instance,
the spatial organization can be directly achieved in the presentation
view by moving objects. To experiment this idea the Opera project built
the Kaomi toolkit [JRT 1999b] which eases the design of such multiple
views authoring tools whatever the paradigm use at the interface levels is
(conditions/actions, hierarchical operators, constraints). We already test
the Kaomi toolkit in different contexts : editing SMIL1.0 document [JRTV
1999], MHEG like documents, or multimedia document expressed by using
temporal and spatial constraints. Multimedia documents designed with such
authoring tools are mainly predictive. Our next challenge will be to
extend the toolkit for the design of unpredictive documents.
- designing "intelligent" temporal view. The aim of this view is twofold :
- firstly to visualize the temporal positions of the document objects
deduced from the authoring actions the author has already performed.
The difficulty here is that there is often several possible objects
positions (due to unpredictive behaviors or solutions space introduced
with constraints formalism).
- secondly to help the author to make the link between such positions
and its authoring actions, since some temporal positions are the
result of joint effects of several authoring actions.
An "intelligent" temporal view is a view which provide authors with
good supports to handle these two difficulties. In [JRT 1998c], we present
such a view for a constraint-based authoring tool (namely Madeus) based
on the following principle : one solution of the constraints set is shown
and the author can get through the solutions space by selecting and moving
objects. Moreover some graphical representations of each authoring
actions are displayed in the view. The major difficulty we still have to
face here is to find the best way to draw this kind of graph : roughly
speaking lines connect objects which are temporally related by an
authoring action, knowing that this drawing is not a static one : the
author will move objects. Simultaneously, we have to think about the
design of such an intelligent view in a more unpredictive context.
- using as much as possible constraint technologies in the authoring tool
we proposed. A constraint defines some relations between two variables
which must be always maintained. In a multimedia document context,
constraints can be used to express the relations between either objects
begin and end points or objects width and height. Their are at least two
strong advantages of handling such relations by using constraint-based
formalisms. Firstly constraints algorithms can be used both to detect some
errors in the document and to automatically propagate some local changes
in the whole document while maintaining valid the other constraints.
Secondly, since constraints can be used to describe not only one solution
but a set of solutions, we can take benefit of this flexibility to adapt a
document to some presentation conditions just before its presentation. For
instance, it is possible to compute a solution such that the maximum
duration for reading the document is less than a given value. With Madeus
[JLRST 1998], the Opera project experiments the idea of a constraint-based
authoring tool for 5 years. One biggest difficulties we still have to
face is to find a better compromise beween a powerful constraints solver
(in term of expressivity and capabilities to control which solution is
computed) and a fast constraints solver [Tardif 00]. The other challenge
consists in findind some ways to extend classical contraints technics
(consistency, solution search) when variables values cannot be controlled
in order to be able to specify unpredictive behaviors [LS 1996a].
Finally, we are currently experimenting how constraints can be used to
internally handle temporal synchronizations between objects, even if the
formalism used by the author is not based on constraints. This experiment
takes place in our Smil 1.0 authoring tool [JRTV 1999].
- considering the twofold issues of error analysis and diagnosis making,
as crucial ones even if a document is not a "critical" process like some
real-time programs. Nevertheless, multimedia documents will be really used
as a new medium to convey information if and only if authors (and people
which pay authors) have some strong confidence in the document execution
they designed. Little work has been already done in this direction. The
intensive use of constraints to handle temporal synchronizations in a
multimedia documents as promoted by the Opera project, provides the author
with a way to detect some temporal inconsistencies but the diagnosis issue
(how helping the author to correct this error) still stays an open and
difficult problem. Moreover, in more unpredictive contexts real-time
formal checking methods based on states diagram and events would be
certainly more suitable to detect errors. Our plan in this direction is to
begin by defining something like an "error typology" both for predictive
and unpredictive documents.
In conclusion, future directions of Opera work on multimedia authoring can
be summed up by :
- use constraint technics "at the right place" inside authoring
- extend our work to unpredictive behaviors,
- pay more attention to the error analysis and diagnosis making
M. Jourdan, C. Roisin, L. Tardif, ``Multiviews Interfaces for Multimedia
Authoring Environments'', Proceedings of the 5th Conference on Multimedia
Modeling, pp. 72-79, IEEE Computer Society, Lausanne, octobre 1998.
M. Jourdan, N. Layaïda, C. Roisin, L. Sabry-Ismail, L. Tardif, ``Madeus, an
Authoring Environment for Interactive Multimedia Documents'', Proc. of the ACM
Multimedia'98 Conference, pp. 267-272, Bristol (UK), septembre 1998.
M. Jourdan, C. Roisin, L. Tardif, ``A Scalable Toolkit for Designing
Multimedia Authoring Environments'', numéro spécial `Multimedia Authoring and
Presentation: Strategies, Tools, and Experiences' de Multimedia Tools and
Applications Journal, à paraître, 1999.
M. Jourdan, C. Roisin, L. Tardif, L. Villard, ``Authoring SMIL documents by
direct manipulations during presentation'', World Wide Web, Balzer Science
Publishers, vol 2. num 4. 1999.
N. Layaïda, L. Sabry-Ismail, ``Maintaining Temporal Consistency of Multimedia
Documents using Constraint Networks'', Multimedia Computing and Networking
1996, M. Freeman, P. Jardetzky, H. M. Vin, ed., pp. 124-135, SPIE 2667,
Laurent Tardif, Frédéric Bes and Cécile
Roisin,"Constraints for multimedia documents", in proceedings of PACLP2000,
Manchester (UK), april 2000
[W3C SMIL 1998]
W3C Recommendation, Synchronized Multimedia Integration Language (SMIL)
1.0 Specification, http://www.w3.org/TR/REC-smil, 15-June 1998.
[W3C SMIL Boston 2000]
W3C Working Draft,, Synchronized Multimedia Integration Language (SMIL)