SMIL 2.0 — Interactive Multimedia on the Web

http://www.cwi.nl/~media/SMIL/Tutorial/{SMIL-4hr.html}

Includes material from the book "SMIL 2.0 — Interactive Multimedia for Web and Mobile Devices"

Copyright © 2006, Lloyd Rutledge

Outline

	Media		Layout		Timing		Interaction		Adaptivity
Beginner	Integration	<ref>	Containing Boxes	box elements	Timelines	composition	Linking	<a> and href=	Selectivity	principles
		URIs		fit=		time boundaries		playstates		system test
		modes and types		z-index=		time types		volume		<switch>
Intermediate	Transitions	fades and wipes	Beyond Boxes	hierarchical regions	Animation	targets	Exclusion	<excl>	Customization	custom test
		alteration		registration alignment		values		<priorityClass>		skip-content=
		<transitionFilters>		opening and closing, sound		functions		time graphs		<prefetch>
Expert	Alteration	fragmentation	CSS	CSS in native SMIL	Beyond Timelihes	fill and restart	User Events	principles	Accessibility	core attributes
		<param>		text-flow		synchronization behavior		keyboard interaction		metadata
		time manipulation		XHTML+SMIL		min= and max=		DOM events		principles

Synchronized Multimedia Integration Language (SMIL)

Main Points

Pronounced smile

Multimedia for the Web — for multimedia what HTML is for hypertext

Integration format for presentable mono-medium formats

Structure

SMIL 1.0 — W3C Recommendation (1998)

SMIL 2.0 "meta-language" W3C Recommendation (2001)

SMIL 2.0 family formats SMIL Profile, SMIL Basicand XHTML+SMIL still lingering ...

SMIL 2.1 recently released with minor additions

SMIL 3.0 could start soon

Main Themes

Powerful timing and synchronization

Adaptive to users and systems

Models a flexible but consistent presentation and user interface

SMIL Isn't

Flash — Flash is mono-medium animation on steriods

MPEG-{4 / 7 / 21} — MPEG looks at content and coding, and player architecture and a whole lot more, but is more media centric than web centric

D-HTML — D-HTML uses scripted definitions of local behaviors, without a notion of the presentation's context

SMIL 2.0 Profiles

What is a Profile?

A language for which a browser can be built

A combination of modules from the SMIL 2.0 "meta-language"

Possibly non-SMIL constructs with SMIL constructs

SMIL 2.0 Language Profile (SMIL Profile)

What is typically thought of as SMIL 2.0

Most of SMIL 2.0 features in one profile

SMIL 2.0 Basic Language Profile (SMIL Basic)

Intented for mobile devices

Assumes restricted processing ability

XHTML+SMIL

Applies timing to text-based display

XHMTL-based layout

SMIL 1.0

Backwards-compatable — can be played on SMIL Profile browsers

SMIL's "Success"

SMIL looses, Flash wins

The dominant desktop Web multimedia format is Flash, by far

Reason: big industry support, cheap and simple authoring

Consequences

Web multimedia is "owned" by Macromedia

Web multimedia is not adaptive or adaptable

Web multimedia is not integrated into the rest of the Web infrastructure

SMIL's quiet success

Unseen format for much of Real media

Very widespread use in MMS

Pretty widespread use in SVG

Important for the accessibility community, such as Talking Books

Constructs and syntax used in media toolkits

SMIL Implementors

RealNetworks

RealOne for SMIL 2.0

Clear leader for SMIL players

Ambulant: current open source player up-to-date with 2.1 (and 3.0?)

Oratrix: GRiNS player and editor

Microsoft

Internet Explorer 5.5 and up plays XHMTL+SMIL

Adobe

Adobe's SVG Viewer supports SMIL animation in SVG

Mobile Industry

3GPP — Next generation mobile framework officially uses SMIL Basic

MMS — Uses SMIL Basic constructs

X-Smiles

Academic product uniting many XML formats in one browser

What's Next?

SMIL 2.0 Profile

Out for years, not used much

Academia exercise interest in development, as in Ambulant

Other long-standing SMIL languages

XHTML+SMIL will probably disappear

SVG grows in adoption (and in SMIL features integrated?)

Mobile use increases, such as with MMS

SMIL Ongoing Developments

SMIL 2.1 has just filled in some feature gaps

SMIL 3.0 may start soon to add new features

Timed Text uses SMIL to make books accessible

What We Need to Specify

		Media
Adaptivity					Layout
	Interaction			Timing

miniFiets 2.0 — The Layout

<!DOCTYPE smil PUBLIC "-//W3C//DTD SMIL 2.0//EN"
                        "http://www.w3.org/TR/REC-smil/SMIL20.dtd">
<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
 <head>
  <layout>
   <topLayout title="Fiets Amsterdam Tour"
              backgroundColor="black"
 width="1010" height="665">
    <region regionName="splashScreen"  top="5" left="5" bottom="5" right="5"/>
    <region regionName="buildingImage" top="5" right="5" width="875" height="655"/>
    <region regionName="closedCaptioning" bottom="5" left="5" right="5" height="60"/>
    <region title="Thumbnail Bar" top="5" left="5" bottom="5" width="120">
     <region regionName="museumThumb"     fit="meet" height="90" top="65"/>
     <region regionName="weighhouseThumb" fit="meet" height="90" top="280"/>
     <region regionName="CWI-INSThumb"    fit="meet" height="90" top="495"/>
    </region>
   </topLayout>
  </layout>
  <transition id="fade1s" type="fade" dur="1s" />
 </head>

miniFiets 2.0 — The Greeting Section

<body>
 <seq>
  <par title="Greeting Section" end="greet.end+1s">
   <img src="FietsLogo.jpg" region="splashScreen" end="greet.end"
     transIn="fade1s" transOut="fade1s" alt="Logo for Fiets: a bicycle zone sign"
/>
   <par id="greet" begin="1s">
    <switch>
     <par systemlanguage="en"
>
      <audio src="welcome.wav"  region="buildingImage"
        alt="Welcome to Fiets, your self-guided tour of Amsterdam (spoken)" 
/>
      <text  src="welcome.html" region="closedCaptioning" systemcaptions="on"
        alt="Welcome to Fiets, your self-guided tour of Amsterdam (captions)"
/>
     </par>
     <par systemlanguage="nl"
>
      <audio src="welkom.wav"  region="buildingImage"
        alt="Welkom bij Fiets, uw eigen stadswandeling door Amsterdam (gesproken)"
/>
      <text  src="welkom.html" region="closedCaptioning" systemcaptions="on"
        alt="Welkom bij Fiets, uw eigen stadswandeling door Amsterdam (ondertiteling)"
/>
     </par>
    </switch>
   </par>
  </par>

miniFiets 2.0 — The Thumbnail Section

   <par title="Thumbnail Section" dur="indefinite">
    <par>
     <a href="#museum"     alt="Show the Rijksmuseum"
>
      <img src="museum.jpg"     region="museumThumb" alt="Rijksmuseum thumbnail"
/>
     </a>
     <a href="#weighhouse" alt="Show the Weighhouse"
>
      <img src="weighhouse.jpg" region="weighhouseThumb"
        alt="Weighhouse (Waag) thumbnail"
/>
     </a>
     <a href="#CWI-INS"    alt="Show the CWI-INS building"
>
      <img src="CWI-INS.jpg"    region="CWI-INSThumb"    alt="CWI-INS building thumbnail"
/>
     </a>
    </par>
    <excl dur="indefinite"
>
     <img src="museum.jpg"       id="museum"
          region="buildingImage" alt="Rijksmuseum"      />
     <img src="weighhouse.jpg"   id="weighhouse"
          region="buildingImage" alt="Weighhouse (Waag)"
/>
     <img src="CWI-INS.jpg"      id="CWI-INS"
          region="buildingImage" alt="CWI-INS building" />
    </excl>
   </par>
  </seq>
 </body>
</smil>

SMIL Structure

SMIL Document Classes

<!DOCTYPE smil PUBLIC "-//W3C//DTD SMIL 2.0//EN"
  "http://www.w3.org/TR/REC-smil/SMIL20.dtd">
<smil xmlns="http://www.w3.org/2001/SMIL20/Language">

<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN"
  "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd"
<svg xmlns="http://www.w3.org/2000/svg">

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+SMIL //EN"
  "http://www.w3.org/2001/SMIL20/WD/xhtmlplussmil.dtd"
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:smil="http://www.w3.org/2001/SMIL20">

The <smil> Element

Contains all of a native SMIL document

The <head> Element

Layout and meta-information, as in HTML

The <body> Element

The content and linking (as in HTML), and timing

Integration — <ref>

Media Object Elements

The Waag is a medieval buiding in Amsterdam					The Waag is a medieval buiding in Amsterdam. It was originally a gate into the city through the city wall. Later it became a weighhouse for goods being brought
<text>	<image>	<video>	<audio>	<animation>	<textstream>

Brush Media — Draw a rectangle on the screen

The <brush> Element

The color= Attribute

Mostly replicable with <region>

Integration — URIs

The src= attribute holds a URI locating the media

Local file system pathname URIs

relative and absolute

Remove URIs

protocol, server, path

xml:base=

base for relative URIs

inherited down subtree

Integration — Modes and Types

Media Typing

The mimetype Construct

Communicating mimetypes through HTTP

Filename suffixes

The type= Attribute

Captured vs. Structured Media

General Types

Photographic Images

JPEG, GIF, RealPix, PNG

Video

MPEG, RealVideo

Audio

WAV, RealAudio

Graphics and Animation

GIF, SVG

Text

HTML, Timed Text?

Containing Boxes — Box Elements

Presentation Windows

The <topLayout> Element — General-purpose Window

The <root-layout> Element — The Main or Only Window

The <region> Element

Each media object instance contains a region reference:

allows author to know where object will be played

<video src="anchor.mpg" region="V-main"/>

The region is defined by:

A regionName= or id= attribute for each region is required

Region positioning attributes

Length values are percentage values or pixels

The unit "px" may be omitted

Containing Boxes — fit=

Containing Boxes — z-index=

When regions overlap, the z-index attribute determines which image appears on top

z-index= determines the stacking order

highest integer stacks on top

Stack level tie-breakers

most recently started is on top

Stacking context is inherited through nested regions

Stack levels are inherited by default

Timelines — Composition

<seq> (sequence) — each child starts when previous ends 1 2

<par> (parallel) — all children start together 1 2

<par>'s and <seq>'s can be nested

endsync=

(1)<par> can end when the ="first" element to finish ends

<par endsync="first">
 <text src="leader_title.html" region="m_title" dur="5s"/>
 <video id="v1" src="cnn.mpg" region="V-Main" begin="1.4s"/>
 <audio src="cnn.aiff" region="music" begin="id(v1)(0.5s)"/>
</par>

(2) <par> can end when the referenced element ends: id( Id-value )

(3) <par> can end when the ="last" element to finish ends (default)

Timelines — Time Boundaries

begin= — in terms of default begin

end= — in terms of default begin

dur= — explicit duration overrides implicit

repeatDur= — loop for specified period

repeatCount= — loop specified number of times

="indefinite" — nothing from element or its content triggers this time boundary

Timelines — Which Time?

Durations

Intrinsic

Explicit — dur= clock-value, "indefinite"

Simple

Active

Perceived

Activation/Termination

Resolved times

Unresolved times

Definite times

Indefinite times

Linking — <a> and href=

The <a> Element

Contains media that is link trigger

The href= Attribute

URI value refers to link destination

Linking within SMIL Presentations

The href="# idref " Attribute Assignment

Moving back and forth along the timeline (instead of scrolling)

Linking — Playstates

Play Spaces for the Link Destination

The show= Attribute

The external= Attribute — open with external program

The target= Attribute — region or other display space to show in

Play States

The sourcePlaystate= Attribute

The destinationPlaystate= Attribute

Linking — Sound

The sourceLevel= Attribute

The destinationLevel= Attribute

Selectivity — Principles

Each element must pass a "playability" test

Those not selected are ignored and not played

Example: captions not played if user does not desire them

Selectivity — System Test Attributes

System Test Attributes

Standard collection of conditions to test for for selectivity

Test attributes can be combined

Adaptation to User

The systemLanguage= Attribute

The systemCaptions= Attribute

The systemOverdubOrSubtitle= Attribute

The systemAudioDesc= Attribute

Adaptation to Hardware

The systemBitrate= Attribute

The systemCPU= Attribute

The systemScreenSize= Attribute

The systemScreenDepth= Attribute

Adaptation to Software

The systemOperatingSystem = Attribute

The systemComponent= Attribute

The systemRequired= Attribute

Selectivity — <switch>

At most one of the children of a switch element is played

The first acceptable element is chosen

Thus, ordering should be more preferred first

Last child can be always-accepted default

Acceptability based on anything the browser wants

Example: versions for different languages

Transitions — Fades and Wipes

The <transition> Element

In document head

Defines one transition for use in presentation

Types of Transitions

The type= Attribute — Which of the fixed set of transition types to use

The subtype= Attribute

Some transition types have subtypes

Such as type ="barWipe" with subtypes ="leftToRight" and ="topToBottom"

Transitions — Alteration

Controlling the Transition

The direction= Attribute — ="forward" (default) or ="reverse"

The fadeColor= Attribute — color value for what is faded from/to

The startProgress= and endProgress= Attributes

How far through default transition to start/end

Transition Modifiers

The horzRepeat= and vertRepeat= Attributes

Repeating the transition in each direction

The borderColor= and borderWidth= Attributes

Changing the appearance of wipe borders

Transitions — <transitionFilter>

Selecting a Transition for the Presented Element

The transIn= and transOut= Attributes — ID of Transition to Use

Inline Transitions — Transitions Defined in the Presentation Body

The <transitionFilter> Element

Had transition-defining attributes

Applies to it's parent, by default

The targetElement= Attribute

ID of element other than parent to apply this transition to

The mode= Attribute

The parent/target element transitions ="in" or ="out"

Beyond Boxes — Hierarchical Regions

Beyond Boxes — Registration Alignment

Sub-regions — Precise Image Positioning within Regions

regPoint= and regAlign= Alignment

Beyond Boxes — Opening and Closing, Sound

Opening and Closing Regions and Windows

Active Regions and Windows

The showBackground= Attribute for Regions

The "always" and "whenActive" values

The open= Attribute — Opening Windows

The "onStart" and "whenActive" values

The close= Attribute — Closing Windows

The "onRequest" and "whenNotActive" values

The soundLevel= attribute

Percentage value, purely relative

Animation — Targets

Animation Elements

The <animate> Element — change an attribute value over time

The <animateMotion> Element — move an object across the screen

The <animateColor> Element — change an object's color

The <set> Element — assign an attribute a value for a period of time

Specifying the Animation Target

The href= or targetElement= Attribute — ID element with attribute to animate

The attributeName= Attribute

The attributeType= Attribute — ="XML", ="CSS", or ="auto"matically determine which

Animation — Values

Listing the Animation Values

The from= and to= Attributes — starting and ending values of attribute in animation

The by= Attribute — incremental value change

The values= Attribute — list of values the attribute goes through

Animation — Functions

Defining the Animation Function

The calcMode= Attribute

="discrete", ="linear" or ="paced"

The accumulate= Attribute

Does repeating animation build on top of previous iterations?

The additive= Attribute

="sum" with the assigned value or ="replace" it?

The origin= Attribute

Position <animateMotion> starts from

Spline Animation

The path= Attribute

SVG path syntax for specifying a curve

The keyTimes= Attribute

A time offset for each value

The keySplines= Attribute

A curve for smooth value changing between each value