Role media items play in the 5 basic discourse layers

by Frank and Stefano - 12-08-2004
Blue note following the Discourse reading club

What is in the world?

The world contains two types of objects:
  1. Concepts expressed in a formal way (e.g. with an ontology).
  2. Digitalized media items. Each media item represents a particular point of view on a concept/event/object/..... expressed using the characteristic of the chosen medium (e.g. a photo (medium) representing a close-up (point of view) of a singer(concept or object), or the audio recording (medium) of a song (the singer's interpretation is the point of view) of the song (concept or object).
Media items are annotated with machine understandable (i.e. expressed formally) attributes. Relations exist between these two types, i.e between media items and between concepts. Across these two types there are relations of many to many which we call instatiations, i.e. a particular media items is the instantiation of zero or more concepts, and a concept has zero or more instantiations in media items. All the elements in this world created with a particular point of view.
Please note that here all is given. However, this world can be extended, for example by the user (annotating or using an interactive system) and possibly by the system itself (Frank is sure, Stefano not so much).

What is in the fabula?

The fabula is a selection of content in the world and depends on how the process decides to develop the plot and the story, as explained in the following.

Plot and Story

Structure (Plot)

The user provides the generation process with the subject, and sometimes with the narrative structure (like in DISC where the user selects the biography genre or in SampLe where the user chooses the structure of the presentation). We distinguish two approaches, representing two extreems for a generation process:
  1. The process selects all media items and concepts from the world and then generate a plot based on the given subject.
  2. The process chooses or is given a plot and tries to fill that in with media items from the world.
The first approach will fit all the retrieved media items in a structure, no matter what has been retrieved, while the second approach can fail to fill the chosen strucure because there are not enough media items in the world (or the existing ones can not be manipulated to represent the missing ones). In case of failure, the process can try another structure (or do nothing and go on with a partially filled structure).
In this paragraph we are concerned with the structure or narrative of the presentation to be generated, and not so much with the form of the presentation. This means that, in case we want to have a presentation about Rembrandt, and we need a portrait of the painter,  if we find one we are happy, without being concerned about the fact that stylistically the portrait can not be used in the presentation.
The generation process can be more or less concerned with the structure of the presentation, and follow more the first approach or the second one or something in between. For ex, Topia follows the first approach while DISC is more on the second one, even if DISC can not fail on this level (but it should).

Form (Story)

After the structure is determined and a fabula has been created which can potentially fill the structure, we distinguish the following approaches:
  1. No form requirements: the media items are included in the presentation with no further analysis
  2.  Form requirements:
    1. The media items are selected based on their form features (e.g. if I need a close-up or a long shot, cold/warm color, audio/video, etc.)
    2. The media items are selected if they satisfy to the form requirements, otherwise they are manipulated to make them satisfy the requirements
Please note that almost no process uses purely the first approach, because the presentation generation is usually aware that it is presenting a picture or text, and this is a form requirement. No presentation generation in this group is doing the manipulation in 2.2, yet.

Again, the first approach can not fail while the second can (even 2.2 because manipulation can not do miracles). In this case the process can go on anyway or try to change the structure described in the previous paragraph. The fact that the constraint solving phase can fail and the process change the layout of the presentation is not really a change of structure from the narrative point of view.

User experience

This part should determine what approaches is used between the ones presented in the previous paragraphs. What we want the user to experience should dictate the genre for style (educational, entertaining) and content (events, actions) and the way (emotional/rational) of the presentation and then give a weight to the different approaches. For example, a more emotional goal, if based on visual and audio, would possibly not be very strong in structure but for sure carefull in form.