Blue Note 2: Automatic Layout 29/4/94 Lynda Hardman A brief summary of my chats with Thomas so far. (To be read at your leisure...) These are my notes to me, so if they're not clear come and ask me. You'll probably hate the terminology - I keep switching. The purpose of subjecting you to this you'll have an idea of what I've understood so far, and how it relates to what I want to do. 26/4/94 Talked to Thomas He is doing auto layout of (parts of) semantic nets, although I'm not sure he would agree with me here. I am doing layout of a bunch of data objects. At some stage earlier I have chosen them out of some selection and they are already ordered in importance and have different relations between them. I think his problem is harder because he has a *much* richer structure which they are trying to give the user access to. (In a sense it is a thesaurus of the information I have already selected. (Peter Bruza's work is also relevant somehow (though he didn't do auto layout).)) They are trying to represent a lot of the information in the structure by showing a subset of the complete structure and a selection of the relevant attributes. They are using graphical languages for expressing the connections/relations and colours for different attributes. His problem is also easier, since he is concerned only with spatial configuration and not with time. A good question (for me) is: what (minimum) semantics do you need in order to attempt to do auto-layout? Or, as Thomas puts it, if you have semantics then you can show them in a layout. Can I view time as a relation? (In Thomas's sense.) I could produce rules for layout, but I will always get conflicts in my rules, so something needs to be done about resolving these conflicts. 27/4/94 Read couple of papers Thomas's GMD report 825 and [Arens et al. 93] -- "Automatic Layout as an Organisation Process" Thomas Kamps and Klaus Reichenberger GMD Arbeitspapiere 825 27/4/94 Want to present information in a pleasing way, based on rules. Position nodes which are related to each other in different ways. Some relations are "stronger" or more relevant than others. They do graph analysis to find out what the similar relations are, then illustrate these with different graphical relations. For example, for example an artist has a date of birth and death. However, artists influence or have been influenced by other artists. The relation between artists is very different from that between an artist and his/her dates. The layout needs to take this into account. In their example the artists are shown in labelled boxes, influences are shown with arrows and birth/death dates are drawn in boxes next to the artist (in a different style). I need to know how the graphical layouts are chosen (e.g. what do you use for stronger association? arrows or proximity?) -- "Structure and Rules in Automated Presentation Planning" Yigal Arens, Eduard Hovy & Susanne van Mulken IJCAI '93, 1253 - 1259 27/4/94 They take the angle that you first need a discourse structure (semantics) before you start on the presentation side. (I agree.) But given you already have the semantic structure then how can you translate that into a presentation? They mention that you need rules such as "what information does the allocation process depend on?" and "what does it produce?". While this is discussed it doesn't leave me any the wiser (perhaps because they are deciding about which medium to use). Of interest is the "Presentation segment" (bottom p1255) comprising a * name (unique ID) * purpose (communicative goal) * medium * position (where the medium should appear in the presentation) * content (semantic content) To me the position is the thing I'm looking to define. I assume the purpose, medium and content are already given. I then want to somehow assign the "position" in my 3D (spatial and temporal) space. I'm interested in the "somehow". For instance, in their Fig 5 are charts and captions. *Why* is the caption below a chart, and where is that information stored? -- 28/4/94 Thomas explained lots more... Q1 Why do they use relations all the time and never attributes. A No big deal, but it makes things homogeneous to handle, and any attribute is, by being describable as a function, a simple relation. Q2 How do you choose between structure decompositions? A Sort of the wrong question! Long answer... To get from a subnet with many (different types of) objects and many (different) relations you need to do some simplifications before you can hope to display anything coherent. (And there are two problems - how much can a human absorb anyway, and how can you best display the amount of information you want to show. We ignore the first :-) There are a number of ways of doing this simpification. One is described in GMD 825 "Automatic Layout as an Organisation Process". The idea, in its essence, is to take a subgraph (in this case of 2 objects and 3 relations) and look at the structure of the relations to see how the most redundancy can be pulled out of the graph. (Thomas explained how you can first remove highly connected parts of the network then analyse the pieces you have left. These are, by definition, acyclic and their structures can be classified.) My question was how can you decide which "simplification" process is the most beneficial, and whether the order of application of each reduction step mattered. They are not yet looking at making this choice - the paper is more a statement of looking for simplification in the graph in this way. There are, however, other ways of making simplifications in the graph (or subnetwork). The second is to look at those relations which are functions and see if the domain values can be used as a means of grouping the objects (simple case - artists born in a certain century). A third simplification method is to take a subgraph related to a particular object then "flatten" the graph - i.e. the object is related directly to each node in the subgraph. You can then look through all these directly related nodes and see if there are groupings which can be used. (So here the structure is being ignored and the attributes of objects looked at, well except that objects don't have attributes... :-) ----- And now for lots of reading. Lynda ----- Automatic Layout - the Problem (and possible approaches to solutions) 11 May 1994 What do I take to be input, and what do I want as output? Whose life am I trying to make easier? Assumption 1: There is already a structured set of information the reader is interested in. Assumption 2: The output is the layout of one windowful of information. Assumption 3: I'm not sure if I'm making someone's life easier, but that what is done now (manual layout) will not be able to be done in the future. There will not be the time to spend on laying out transient information. So, what we're offering is better quality of layout for this transient information. (Maybe the info. itself isn't transient, but the way it is presented to others is transient.) Compare with Mosaic type info.? Also want to describe general styles for the layout which each individual layout will conform to. Assumption 4: The objects I am displaying on the screen are not mine to manipulate. I am displaying what I am given. (Well, this is true for sound, pictures, video. But what about text. I'm not about to alter the content, but may alter the fonts, size....) Assumption 5: I'm only concerned with a handful of objects at a time. This is because my objects are "large" - picture, sound fragment, video clip... - and each contains a fair amount of information. I don't want to present too many at once, or the reader will only get confused/pissed off/disoriented. I *do* want to give access to all the others that may be relevant to the reader - but they are part of the next, or subsequent, display. (Do I need to argue this more?) ----- (big) Question 1a: Do I need more information than hypermedia structure? In order to form a basis for my layout decisions. *What* is the information I need to use a a basis for generating the layout. Thomas and Klaus have a full knowledge representation - I don't want to go that far. Can I stop somewhere half way? What do I miss out on? These are the fundamental questions to answer. (big) Question 1b: *How* do I use this structural information? Given the structure, how can I use it to derive constraints that can be expressed in the layout? (For example, should there be rules such as: objects with the same parent should be placed close together.) (big) Question 2: What is the essence of a layout style? (And does Klaus know this already??? Or is this what graphics designers get paid for :-) ) I am going to have to come up with ways of changing the "look and feel" of a publication [[I also need to argue *why* I want this]]. Given the *same* newspaper articles I have to know enough to be able to generate the NRC Handelsblad, the Herald Tribune, the Sun or Le Monde. OK, a lot of the differences between them have to do with word, sentence or paragraph length, but there are also differences at the level above that - font styles, sizes, mixes, margins, columns, ruling. It is all going to have to be guided by some higher level style-guide. So what are the ingredients for this style guide? [[The implication of this is that I'm going to want a *lot* more control over the layout of my (especially text) channels in CMIFed. I want to specify the size of (empty) top, bottom and side margins (independently of each other, or perhaps dependently?). I want to put different types of ruling around the edges of channels (thickness of line, offset from edge).]] (little) Question 3: What are multiple columns good for anyway? Newspapers have columns. Why? Narrow lines of text are easier to read, but why have multiple columns? I presume because lots of long columns look odd on a page. However, why not make our columns ever so slightly wider and only use one per article? You can scroll/page (whatever) through the article. So - can we junk multiple columns? And if not, *why* not... ---- Requirement 1: [[I need to argue *why* this is a requirement.]] Once the reader has seen one piece of information once it should always be presented in the same context on a re-visit. In other words, the generated layouts become "static" once the reader has seen one. This requirements means that a complete history has to be kept of everything the reader has ever seen in the system. This should be minimal information c.f.. size of a video fragment. (How it should work, we'll see later. Mosaic manages to do something similar. One simplifying factor is that readers will only look at a tiny fraction of what is available.) ---- Argument 1: Fillers are not necessary (this is against the IEN philosophy) Fillers are the extra bits of info newspaper people put on the page to fill it up, 'cos the article they would have liked to put there doesn't fit, or because the articles that are there aren't large enough. In the electronic environment space is (almost) for free. There is absolutely no reason to crowd a window in the way a newspaper page is crowded. OK, maybe you want an electronic newspaper to sort of resemble a paper one so readers don't feel entirely alienated, but there are other (better) ways to cope with too little or too much information on a page. The last thing you want is a trade-off that has an influence through all the other layouts. The better way, in the case of too little info. (so there are no longer any scrolling objects), is to expand any pictures slightly, enlarge margins and choose a layout that "wastes" space. For too much information (assuming you don't have too many items) then shrink the pictures slightly, and put scroll bars on the longer text. If you have too many items you shouldn't be trying to cram them in the window in the first place.... [Maybe you want some sort of box filling algorithm that uses TeX "springs" in two dimensions (3, when we get to time, but that's the next chapter...). The objects between the springs can also expand and contract according to the pressure on them by the spring.] Argument 2: Once through generation is not enough. (this is pro the IEN philosophy) Once you have generated a layout that may not be the layout you will go with. You may have satisfied all the constraints, but it looks "bad", 'cos, for example, two pictures are offset in a strange way from each other (each belonged to its own sub-area and were laid out within those areas). So, either you need complex rules which do layout at different levels simultaneously, to you have to try one layout then reject it and try another. (Would Prolog be useful for this?) Another strategy would be to prescribe the layout, and then find objects which could fill it. (That's essentially what I was doing in the Amsterdam Demo, except I also had the ability to make my layouts.) ---***---