SEMANTIC INTERACTION ==================== by: michiel hildebrand at: $Date: 2007/02/23 20:18:13 $ ab: This blue note is intended to get more insight in data driven generation of interactive presentations guided by the perspective of the interface. 1. Search and navigation on the (semantic) web ---------------------------------------------- Search and navigation on the current web are very successful techniques that give access to human created documents or part of a document. It is the task of the user to determine wether a document contains useful information. The semantic web aims to provide uniform access to web content through machine processable annotations and ontologies. This infrastructure allows developers to create intelligent services that automate tasks users currently have to do themselves. 1.1. Semantic driven search and navigation I want to explore the process of -- searching for documents -- browse through them to determine relevance -- navigate to linked information -- determine relevance -- and so on. In particular I question what this process looks like on the semantic web, and how we can develop intelligent services that automate (parts of) this process. ============================================================================== Is it possible to (partially) automate the search for relevant documents? ============================================================================== 1.2. Meaningful presentation generation We have to take into account a crucial difference between the current web and semantic web. The first consists of human created documents while the latter contains collections of annotated heterogeneous resources, including human created documents. On the current web a result of a search or navigation action is a (hopefully coherent) document. On the semantic web a result could be a document, or any other resource, such as an abstract concept or property (URI), image, video, sound, string, number, etc. A set of resources, as a result from a query, is probably not very well suited for direct human consumption. A central question of our group's research is how we can automate the generation of meaningful presentations from a set of annotated resources. 1.3. Requirements Intelligent services that can find relevant documents on the semantic web are at least required to: - select relevant resources - structure the selected resources - generate the presentation from the selected resources This seems to correspond with Alia's model in her blue note Formalizing Design Rules. (I have to admit here until now I did not read the Standard Reference Model for Intelligent Multimedia Presentation Systems very thoroughly yet.) Alia is doing research on the presentation side. Given a structure how can we present it. I can't say I want to do research in selection and structuring. In the process of finding relevant documents, both selection, structuring and presentation plays a role. ============================================================================== How are the processes of selection, structuring and presentation related to each other? ============================================================================== 2. Interactive presentations ---------------------------- Search and navigation techniques are intended to select information that matches a request of a user. As on the current web it still remains the task of the user himself to determine if the resources are useful to him. I believe that semantic driven interaction can help the user make this decision. To accomplish this semantic interaction has to take (at least) the following issues into account: - What exactly does the user want to know? - What and how much does the user want to know about a resource? Can a system give a satisfying answer to these questions? My aim is not to create a systems that answers these difficult questions directly. Instead I want to explore how they can be approached step by step in an interactive presentation. Focus is on presenting resources in a way that allows further meaningful interaction. The user decides what she/he wants to know and how much. The system assists by indicating why a resource is relevant and how it is related to other resources. In this view, a meaningful presentation behaves as a guide in the users task to determine relevance of resources. In an interactive presentation, search, navigation and presentation are closely intertwined with each other. At each step both selection and presentation are necessary to determine the relevance of a resource and the relation to other resources. ============================================================================== Why is a resource relevant? How is a resource related to other resources? ============================================================================== 3. Presentation driven vs data driven ------------------------------------- The whole chain from selecting resources to presenting can be approached in two directions. 3.1 presentation driven The selection of a resource is determined by the presentation. The final structure of the presentation is known, the system has to fill in the elements. The selection of the relevant resource and which properties to show has to be defined in advance, by a template or a set of rules. + control over the final form - relevance determined in advance 3.2 data driven Explore all relations as given by the semantic graph and select all relevant resources. + domain independent + in theory possible to find all relations - many results possible (problematic for computation and visualization) Would like to have the domain independent character of data driven approach but some guidance from the presentation side, without restrictions to a predetermined form. ============================================================================== Can we combine data and presentation driven approaches? ============================================================================== 4. interface perspective ------------------------ Two directions I want to explore in this research: 1. The system guiding the user in the process of finding relevant results. 2. The presentation as a guide in data driven selection of relevant resources. I think both directions can be tackled by taking the perspective of the interface into account. The perspective of an interface describes what can be expressed in that interface. ============================================================================== How can we express the perspective of an interface? How does a perspective determine the context of a resource? ============================================================================== 4.1. hierarchy perspective -> Give example of the faceted browser. 4.2. Temporal and spatial perspective -> more to come