A hypermedia communicative device model

Lynda Hardman, Jacco van Ossenbruggen

Time to pin the communicative device down. This is possibly a little premature, since Marcos is still working on CD's from his perspective, but I hope that nothing fundamental will change, only that we may get a few more CD's.

This document describes the basic CD structure, lists some CD's, including which CD's can be used with which, and gives an example CD tree for the chiaroscuro demo. Until further notice, the CD structure is hierarchical, since I'm not sure whether we need a DAG, and so we'll start simple.

A Newer Communicative Device skeleton (9 Aug 2002)

A-B : exs: figure with image(A) and optional caption(B);
          table with content(A) and optional caption(B).
      children A B
      ordering: default: ordered
      default layout: A above B, A left-of B, A before B, A link-to B, A only.

A1-A2 : exs: figure with image(A1) and important caption(A2);
          table with content(A1) and important caption(A2).
      children A1 A2
      ordering: default: ordered
      default layout: A1 left-of A2, A1 above A2, A1 before A2, A1 link-to A2

A1-A2-..-An : ex slideshow.
      children N times A
      ordering: default ordered
      default layout: hbox, vbox, tbox (bookshelf perhaps)


A-B1-B2 : ex. title with examples of definition
      children A with 2 times B
      ordering: default A, B1, B2
      default layout: A above (B1 left-of B2),
		      A above B1 above B2,
		      (A above B1) left-of B2,
		      B1 left-of (A above B2),
		      A above (B1 before B2),
		      (A above B1) before B2,
		      ... (and the rest)


A-B1-B2-..-Bn: ex. ??


Semantically equivalent A1-..-An : ex. text definition and corresponding voiceover
      children N times A
      ordering: default unordered
      default layout: A1 or A2 or .. An (based on user profile/platform)


----------
Misc. notes:

(No RST for "table" or "grid".)

Where is knowledge about the media used?
Global constraints for too many audios or videos.


All the multi-nuclear relations the same? (Joint, Sequence, Contrast)
In Sequence the order matters, in joint, not.

----------

A Communicative Device skeleton

list of children, each with importance factor. A child is another CD, or an atomic media item. Media items need a MIME type (for the attention stuff later).
list of zero or more orderings among the children
style info. I'm not sure what this is yet, but it needs to be here somewhere. E.g. for a "collection of scenes" CD, we would want each child scene to be slightly visually different from another, e.g. background colour?? But then this should point to a item in a "collection of scene styles". Does this mean that I need a design store with designs labelled for use with certain sets of CD's?
Some of the style information can be deduced from the relative importance level of the completed CD tree. Not all of it, since there are aspects of the discourse structure that aren't related to importance, e.g. the grouping - but that is reflected in the CD structure itself.

Some Communicative Devices

Name: name of the communicative device, the same sort of function as a pattern name
Children: names of identifiable children
Description: free text as I felt fit.
Importance is an integer where the lower the number the more important the item is. (Each child CD gets "1" added to its importance number when constructing the tree (from the RST layer above), so that items higher in the tree are always more important.)

Name	Children	Description
Title	text item	importance 1
	text item	style name/indication
	content	importance 2
	content	A media item, or any one of the following CD's: Title, Label, List.
	ordering	text item before content
Label	Content	importance 1
	Content	Could be any of the following CD's: media item, others?
	Text item	importance 2
	Text item	style name/indication
	ordering	content before label
List	list of children	importance 1
	list of children	All children have to be of the same type, whether media item or CD (this seems to be sensible). Permitted CD's are: Title, Label. NOTE: not a List. You lose the "titles" of the list items then.
	ordering	Unordered unless explicitly specified. Child 1 through child N is an obvious ordering.

Observations

I suspect that it is not coincidence that these three basic CD's correspond to Nuclear-Satellite combinations. Namely, N-S is Title/content, S-N is content/Label and N-N is List. What is missing is a list-type collection of non-similar siblings, e.g. {text item, image item, Title CD, Title CD, text item}. Either these don't normally occur in "good" communication, or I should have 2 CD's: an all-children-are-the-same CD (List) and a miscellaneous-anything-goes CD.

Note - Misc. CD to be added. What I have done is removed the possibility of having two satellites of a nucleus without stating what the relationship is between the two (or more) satellites, so I'm forcing an extra level of structure, even though it might seem unnecessary, we need it for knowing the relative importance and ordering among the children.

But then again, why bother with individual CDs? Why not use a single description, with importance and ordering values for the children. _If_ there is only a subset of combinations possible, then named CDs is a better idea.

Example CD trees

2-level menu

A menu is a Title plus a List. The CD tree for a nested menu is thus:

Title
   [text item1]
   List
      Title
	 [text item2]
	 List
	    [icon1]
	    [icon2]
      Title
	 [text item3]
	 List
	    [icon3]
	    [icon4]
	    [icon5]
      Title
	 [text item4]
	 List
	    [icon6]
	    [icon7]
	    [icon8]
	    [icon9]

So depending on the various circumstances, either the user sees this (we have reached the HFOs now), if there is enough screen space:

[text item1]
	 [text item2]
	    [icon1]
	    [icon2]
	 [text item3]
	    [icon3]
	    [icon4]
	    [icon5]
	 [text item4]
	    [icon6]
	    [icon7]
	    [icon8]
	    [icon9]

Or this if there is only room for the first level.

[text item1]
	 [text item2]
	 [text item3]
	 [text item4]

Or this if some form of horizontal layout is used.

[text item1]:    [text item2]    [text item3]    [text item4]
	         [icon1]  [icon2]

Chiaroscuro example

Title
   [text itemA - Rembrandt van Rijn]
   Misc (ordering text item before List; omit music if necessary
	    (hmm, I obviously need an importance factor here))
      [text item - descriptive text]
      [music item - classical music extract]
      List (ordered by feature from database - date)    
         Label
	    [image item1]
	    [text item1]
         Label
	    [image item2]
	    [text item2]
	 ...
	 Label
	    [image itemn]
	    [text itemn]

What about tables?

We should be able to generate tables from the underlying CD's. Is relative importance enough, would we need specialist table HFOs, and does alignment need to be explicitly specified somewhere, or should it just "come out in the wash" in the ways of visualizing grouping? (The CD->HFO processing.)