Linked Media:
Weaving non-textual content into the Semantic Web
Raphaël Troncy
Researcher at CWI,
Media Fragments Co-Chair
This slide set was presented at the MozCamp, in Utrecht, The Netherlands, 5, March 2009.
What's on the Internet tonight?
The Internet is becoming consumers' primary entertainment source […]
— Saul Berman, IBM Media & Entertainment Strategy and Change
© Fox Broadcasting Company
- There is a decline of TV as Primary Media Device.
- Nearly 75% of US Internet users watched online video
- Video producers are looking at ways to increase their reach.
Video Forecast
Cisco's Global Consumer Internet Traffic Forecast
Source: Cisco, 2007
- Video is driving IP traffic growth + Increase in volume of video content
- In 2011, nearly 30% is Internet Video, additional 30% due to Video P2P
W3C Workshop
W3C Video on the Web Workshop
12-13 December 2007, San Jose, California and Brussels, Belgium
Hosted by Cisco Systems
Motto: Make video a first-class object on the Web
Workshop Laundry list
<video> HTML 5.0- Codecs and formats
- Metadata ⇒ Media Annotations WG
- Content protection
- Fragment ⇒ Media Fragments WG
- Accessibility (still an issue)
- Tagging
- Searchability
- Synchronization, Temporal properties for styling
- Distribution: cost, QoS, tracking, mobile, etc.
- Playlist files
- …
W3C Media Fragments Working Group
Media Fragments WG Goal
Provide URI-based mechanisms for uniquely identifying fragments for media objects on the Web, such as video, audio, and images.
Media Fragments URI is a hot topic
Every key player does it ... but they all use a different syntax!
Technologies Survey (cont.)
Other time-clipping and spatial clipping methods:
- MPEG-21: time, space, track, id (http://www.example.com/myVideo.mp4#mp(/CD/track/~time(%27npt%27,%270%27,%2760%27)))
- SVG: space (http://www.example.com/myImage.svg#svgView(14.64,15.73,146.98,147.48))
- SMIL: time (
clipBegin, clipEnd), space (panZoom, area), media markers
- MPEG-7: time, space, modality
Media Fragments 1.0
- Syntax:
http://www.example.com/movie.mov#t=12.33,21.16
http://www.example.com/movie.mov#t=smpte-25:00:12:33:06,00:21:16:00
http://www.example.com/movie.mov#xywh=20,20,40,40
http://www.example.com/movie.mov#xywh=pixel:20,20,40,40
http://www.example.com/movie.mov#track='audio1'
http://www.example.com/movie.mov#id='the%20kiss%20scene'
http://www.example.com/movie.mov#track='audio2'&t=12.33,21.16
- Communication between the User Agent and the Server:
General principle is that smart UA will strip out the fragment definition and encode it into custom http headers ...
- 2 way-handshakes: use custom HTTP range units, defined along the different axis suitable to describe media fragments
- 4 way-handshakes: custom resources that will direct the client on how and where to get all the data needed to construct a fragment
- (Media) Servers will handle the request, slice the media content and serve just the fragment
... while old ones will serve the whole resource
- Server, caches and proxies: use byte/time ranges to cache (merge) fragments
UA → Server Communication
- User → UA (1): A user requests a media fragment URI, for example using a web browser:
http://www.example.org/myPodcast.mp3#t=15,45
- UA → Proxy (2) → Origin Server (3): UA chops off the fragment and turns it into a HTTP GET request with a time range header:
GET http://www.example.org/myPodcast.mp3
Accept: application/mp3
Range: seconds=15-45
- Origin Server → Proxy (4) → UA (5): The server has a module for slicing on demand multimedia resources, that is, establishing the relationship between seconds and bytes,
extract the bytes corresponding to the requested fragment, and add the new container headers in order to serve a playable resource. The
server will then reply with the closest inclusive range in a 206 HTTP response:
HTTP/1.1 206 Partial Content
Accept-Ranges: bytes, seconds
Content-Length: 1201290
Content-Type: audio/mpeg
Content-Range: seconds 14.875-45.01/321
- The user agent will then have to skip 0.125s to start playing the multimedia fragment as 15s.
Call for Implementations
- Client side: browser plugin/extension that convert Media Fragment URI into suitable HTTP requests
- Server side: server extensions that slice on demand media resources depending on the codec and container formats:
- Example: the Apache module (mod_annodex) for the MIME types 'application/annodex'
- The TemporalURI spec uses both the fragment (client-side offsets / filtering) and the query (server-side resource composition)
- The module provides temporal and clip URI query handling by recomposing valid Annodex resource for Ogg files!
W3C Media Annotations Working Group
Media Annotations WG: Core Ontology
- Many standards: EXIF, ID3, MPEG 7, SMIL, iTunes XML, Yahoo! MediaRSS, CableLabs VOD Metadata Content, Video sitemaps, IPTC, EBU, etc.
- Related initiative: Metadata Working Group (MWG) = Canon, Sony, Nokia, Microsoft, Adobe
- Develop a simple lingua-franca ontology
Work in progress: mapping table
Media Annotations WG: Metadata API
Client side read-only media annotations API
var vid = document.getElementById("vid");
vid.getTitle();
vid.getAuthor();
vid.getRights();
vid.getLicense();
vid.getCreationDate();
See: Demo
Example 1: Cultural Heritage
See: Demo
The Web of Data
- Expose open datasets in RDF
- Set RDF links among the data items for different datasets
- Over 4,5 billion triples, 5 millions links(March 2009)
DBpedia
DBpedia is a community effort to extract structured "infobox" information from Wikipedia
DBpedia - Geonames interlinking
DBpedia is interlinked with other datasets on the Web ... so processors can switch automatically from one to the other
Linked Data Cloud (1) - March 2007
Linked Data Cloud (2) - August 2007
Linked Data Cloud (3) - March 2008
Linked Data Cloud (4) - September 2008
Linked Data Cloud (5) - March 2009
Example 2: Browsing Multimedia News
What are the dimensions generally used for searching news items?
News Metadata Enrichment
News Metadata and Linked Data
Example 2: Browsing Multimedia News (cont.)
Take Home Message
- Videos are and will be more and more present on the web
... but it is still very difficult to search for, interact with and re-use them
- Need for pointing to, bookmarking, downloading parts of videos:
- Need for describing and searching parts of videos:
){kind=link}
