Transcription
Work on Model-based UI at HIIS / ISTI-CNRFabio Paternò, Carmen Santoro, Davide Spanohttp://giove.isti.cnr.it/ISTI-C.N.R.HIIS LaboratoryPisa, Italy
Model-basedUser Interface Design MotivationsFirst generation Second generation UIDE, HumanoidMastermind, Adept, Mobi-DThird Generation UIML, TERESA, PUCModel-Based UI at HIIS / ISTI-CNR2
Abstraction Levelsin Interactive Systems Task and object Abstract Interface – Single selection object with high cardinalityConcrete Interface – I want to select a work of art– List Interaction object with X elementsImplementation – List object in Java or XHTML or .Model-Based UI at HIIS / ISTI-CNR3
Possible TransformationsForwardReverseTask ModelAbstractInterfaceConcreteInterfaceModel-Based UI at HIIS / ISTI-CNR4
Multiple Levels of AbstractionAdvantages Focus on the main design choicesLinking semantic information andimplemementation elementsSemantic Web vs Interaction SemanticInteroperability through many possibleimplementation languagesModel-Based UI at HIIS / ISTI-CNR5
The ConcurTaskTreesNotation for Task ModelsHierarchicalstructureTemporal relationsTaskAllocationModel-Based UI at HIIS / ISTI-CNR6
Specifying Platformdependent TasksModel-Based UI at HIIS / ISTI-CNR7
The user interface of theCTTE tool - DEMOModel-Based UI at HIIS / ISTI-CNR8
Task model - based design1. Analysis oftemporal operatorsamongst tasks2. Analysis of eachtask (objects,attributes, .)Identify structureof dialoguesChoose suitableinteraction objectsModel-Based UI at HIIS / ISTI-CNR9
Design PracticeModel-Based UI at HIIS / ISTI-CNR10
Communication-orientedComposition operators Grouping: a set of elements logicallyrelated to each other Ordering: existing of an order amonginteractors (i.e. temporal) Hierarchy: a logical hierarchy among a set ofinteractorsRelation: 0ne interactor related to agroup of other interactors (i.e. disablingthem)Model-Based UI at HIIS / ISTI-CNR11
Structuring the UserInterface Grouping – Example: Task decomposition - grouping of correspoding interactiontechniquesOrdering – Sequential communicating tasks adjacent interaction techniquesRelation – Control tasks (one to manyrelations)Hierarchy – Frequent tasks – More spaceor larger attributesModel-Based UI at HIIS / ISTI-CNR12
TERESA XML Two platform-independent languages : task (CTT)and abstract interfaceOne level (concrete interface) representedthrough a number of platform dependentlanguagesDesigners aware of the potential platforms (notdevices) early on in the design processMethod allows developers to avoid dealing with aplethora of low-level details (transformation fromconcrete description to implementation isautomatic)Easy to add support for new implementationlanguages Model-Based UI at HIIS / ISTI-CNR13
The Structure of the Abstract User InterfaceUser lection .ConnectionModel-Based UI at HIIS / ISTI-CNR14
Concrete User Interface Defines some concrete aspects of the user interfacedepending on the current platformProvides indications for the implementation of abstractinteractorsEXAMPLEConcrete levelAbstract levelI-- interactor id "Go to section3" interaction - control navigator button label "Go to selected section"/ /navigator /control /interaction Model-Based UI at HIIS/ /interactor ISTI-CNR- interactor id "Go to section3" interaction category "interaction" control type "control" navigator object "navigator"/ /control /interaction -15
Example of platform-dependentconcrete interactor choiceEXAMPLE:Single choice abstract ow cardinalityRadio ButtonRadio ButtonMedium cardinalityList BoxDrop Down ListHigh cardinalityList with scrollbarsDrop Down ListModel-Based UI at HIIS / ISTI-CNR16
Example of platform-dependentcomposition operator implementationEXAMPLE: Grouping Operator Desktop Computers FieldsetBulletBackground Colour/Image nMobile Phones Unordered List in ColumnFieldsetModel-Based UI at HIIS / ISTI-CNR17
The Authoring Model-Based UI at HIIS / ISTI-CNR18
Example of TERESAgenerated User InterfaceDEMOModel-Based UI at HIIS / ISTI-CNR19
Vocal Interaction Characteristics: linear, not persistent, fasterand more natural for some operationsProvide feedback to check the status ofapplicationBrief prompts and short lists of options toreduce memory capabilityManagement of events (no-input , no-match,help)Model-Based UI at HIIS / ISTI-CNR20
Speech implementation ofcomposition operators Grouping: Insert a sound Insert a pause Use some keywords Use a specific volume ofsynthesizer voiceOrdering Alphabetical order Use some keywordsRelation Change context(change type of menu)Hierarchy Increase or decrease the volumeof synthesizer voiceModel-Based UI at HIIS / ISTI-CNR21
Specifying generalparameters for allpresentationsModel-Based UI at HIIS / ISTI-CNR22
VUI vs GUIFirstpresentationWelcome to the Marble Museum VoiceResponse System. This service recognisesSystem:FifthThehas tobeenachievedthroughyourBoatspeechprovideyou withthe gits ininformationyou request.(grouping sound) The artworks hesound(grouping) following: Boat, Totem,presentationthe sectionare thematerialiswhitemarble.you wouldlike informationon one ofIfHole.you Ifwouldlike somegeneral information,System:System: (Five second pause)theseplease sayif itsname(groupingsound )sayinformation;youwouldlikeinformationthatif youwouldliketo toreturnto toSystem: ficartworks,sayartworks;if youthemainmenu,sayhomeorifyouwouldlikethe mainmenu,say home.wouldliketobookaticket,sayticket.to (Timego backto Pleasethe previous menu, say back.out)System: (groupingsound ) say your ck) ok, you have chosen Boat.System: Ok, loading information about artworks.(The system goes to presentation about Boat)System: (The system goes to presentation aboutArtworks Section) Welcome messageManagement of no input event Provide feedback Description ObjectUI at HIIS / ISTI-CNRCompositionModel-Basedoperators 23
Adding support for a newmulti-modal platform Define new concrete description languages(as refinement of the common abstract one)Identify target implementation language(s)Identify how to support multi-modalityModel-Based UI at HIIS / ISTI-CNR24
Generation of Multi-ModalInterfaces X V – W3C standardSupported by OPERA and NetFront Browser,also for PDAsEMMA not supported by any public toolSMIL not interaction orientedX V application structured into three parts(document definition, head and body)Model-Based UI at HIIS / ISTI-CNR25
Design of MultiModal Support CARE properties (Complementarities,Assignment, Redundancy, Equivalence)Application to composition operators,interaction and output-only elementsInteraction structured into prompt,input, feedbackIdentify meaningful solutions, providesuggestionsModel-Based UI at HIIS / ISTI-CNR26
Design of MultimodalSupport (graphical voice) Identification of new platforms (multimodaldesktop, multimodal PDA, )Design how to support composition operatorsand interactorsmultimodal desktop: compositon operators - graphically supportedinteractors - graphical prompt, input either graphicalor vocal, feedback in both modalitiesmultimodal pda: compositon operators - supported both graphicallyand vocallyinteractors - redundant/complementary prompt,input eitherModel-Basedgraphicalor vocal, feedback in bothUI at HIIS / ISTI-CNRmodalities27
The Authoring EnvironmentMulti-Modal attributesdefinitionsModel-Based UI at HIIS / ISTI-CNR28
The Authoring EnvironmentModel-Based UI at HIIS / ISTI-CNR29
MultiModal TERESAhttp://giove.isti.cnr.it/teresa.htmlMobile XHTML MPDigital TVGraphical onXHTML/SVGForm-basedXHTMLGraphical VocalX V30
MARIA XML Requirements(MARIA – Model-based Authoring enviRonmentfor Interactive Applications) XML-based Languages with SchemasSupport for Abstract Data TypesMore engineered and powerful language(e.g. Pacman)Able to generate user interfacesincluding complex Javascripts and AjaxscriptsModel-Based UI at HIIS / ISTI-CNR31
MARIA Tool Requirements(MARIA – Model-based Authoring enviRonmentfor Interactive Applications) New Authoring Environment Integrated Support for Web Services Mappings WSDL/LUIGeneration/RefinementNot only traditional top-down approachesTransformations not hard-coded but definedexternally and performed with XSLTIntegration of BPMN/BPEL with Model-based UIs.Model-Based UI at HIIS / ISTI-CNR32
Agenda How to Manage JavaScritpsIntegration of wider set of interactionmodalities in multi-device environmentsMulti users applicationsIntegration of model-based user interfaceswith Semantic WebServFace and OPENModel-Based UI at HIIS / ISTI-CNR33
Model-Based UI at HIIS / ISTI-CNR 13 TERESA XML Two platform-independent languages : task (CTT) and abstract interface One level (concrete interface) represented through a number of platform dependent languages Designers aware of the potential platforms (not devices) early on in the design process
