Transcription
The 4C Framework:Principles of Interaction in Digital EcosystemsHenrik Sørensen, Dimitrios Raptis, Jesper Kjeldskov, and Mikael B. SkovDepartment of Computer Science / Research Centre for Socio Interactive DesignAalborg University, Selma Lagerlöfs Vej 300,DK-9220, Aalborg East, Denmark{hesor, raptis, jesper, dubois}@cs.aau.dkABSTRACTseveral and sometimes overlapping purposes, have becomepart of our everyday lives. Consequently, the ways we usedigital artifacts have changed. We have become users ofnetworks of artifacts rather than of individual ones [13] andwe share and connect to each other’s digital artifacts indifferent ways than we did just ten years ago [15]. Accessto digital services and content has become fragmented, andso a need for our artifacts to work together has grown.Today it is therefore important to be able to design ubicompsystems where the user’s interaction spans across multipleinterconnected digital artifacts.Recent years have seen an increased research interest inmulti-device interactions and digital ecosystems. Thisresearch addresses new opportunities and challenges whenusers are not simply interacting with one system or deviceat a time, but orchestrate ensembles of them as a largerwhole. One of these challenges is to understand whatprinciples of interaction work well for what, and to createsuch knowledge in a form that can inform design. Ourcontribution to this research is a framework of interactionprinciples for digital ecosystems, which can be used toanalyze and understand existing systems and design newones. The 4C framework provides new insights overexisting frameworks and theory by focusing specifically onexplaining the interactions taking place within digitalecosystems. We demonstrate this value through twoexamples of the framework in use, firstly for understandingan existing digital ecosystem, and secondly for generatingideas and discussion when designing a new one.Understanding and designing multi-artifact interactions ishowever not only a question of accommodating changes inhow we use our existing interactive artifacts. There is also aneed to address the challenge of unlocking the full potentialof the artifacts and infrastructure available to us. The ideaof using digital artifacts together has existed for a long time(for example Rekimoto’s Pick and Drop [24]) but questionsstill remain open in terms of designing interactionsmeaningful to ubicomp environments. It is, for example,still not a trivial task to take a picture using a smartphone,and then show it on a large screen nearby. Neither is it clearhow to provide feedback to a group of people using asystem together. How to pursue these issues appropriatelyis also a challenge in itself.Author KeywordsDigital ecosystems; multi-artifact; multi-user; framework;structures; relationships.ACM Classification KeywordsH.5.m. Information interfaces and presentation (e.g., HCI):Miscellaneous.We believe that approaching the challenge of multi-artifactinteractions in a holistic way, as an ecosystem, can help usdiscover new interaction design opportunities. Recently,more theoretically founded ecosystem approaches havebeen published in the HCI literature (e.g. [6, 12, 13]). As acontribution to this research direction, we present the 4Cframework (Communality, Collaboration, Continuity, andComplementarity), which can help understand and designinteractions for digital ecosystems by dividing these into 4themes and 8 principles. The framework emerged byanalyzing existing, real-world digital ecosystems driven bythe question: What makes a digital ecosystem more thansimply a collection of artifacts?INTRODUCTIONUbiquitous computing (ubicomp) has not yet reached alevel corresponding to Weiser’s [28] vision of invisiblecomputers seamlessly integrated into our environment.However, digital interactive artifacts remarkably similar tohis notions of pads, tabs and boards constantly surround us.Smartphones, tablets, laptops, TVs, and smart watches,each capable of accessing various services and servingPermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. Copyrights forcomponents of this work owned by others than the author(s) must behonored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires priorspecific permission and/or a fee. Request permissions fromPermissions@acm.org.Ubicomp '14, September 13 - 17 2014, Seattle, WA, USACopyright is held by the owner/author(s). Publication rights licensed toACM. ACM 978-1-4503-2968-2/14/09 15.00.http://dx.doi.org/10.1145/2632048.2636089The next section presents related work and theoreticalbackground. The 4C framework is then presented. Thevalue of the framework is then demonstrated through anexample case of analyzing an existing digital ecosystem andan example case of informing the design of a new one.Finally, we discuss implications for design and research.1
RELATED WORKdigital technology that a person owns, has access to, anduses”, and based on empirical studies they present acategorization of artifact properties into physical,functional, informational and interactive aspects. Adding tothis, Bødker and Klokmose [6] elaborate on the notion ofpersonal artifact ecologies through the Activity Theorybased Human-Artifact model as a framework forunderstanding the way people perceive and appropriateartifacts within an ecosystem. They furthermore argue thatthere might be a need to change the conception of anartifact as a physical device, and that a more inclusivenotion will help describe the role of a device in specificrelationships better [7].In the following we present related work in the areas ofmulti-artifact interactions and ecosystem thinking in HCI.The idea of using several artifacts together has always beenimportant within ubiquitous computing. Rekimoto [24], forexample, envisioned nearly two decades ago what he calledmultiple-computer user interfaces, and argued thatinteraction techniques must overcome the boundariesamong devices in multiple-computer environments. Otherresearch following the same line of thinking has sinceemerged where various aspects of multi-artifact interactionsand “digital ecosystems” have been investigated. Wäljas etal. [31], for example, presented a framework for crossplatform service user experiences and found that users maynot be as sensitive to consistency issues across platforms asis often assumed. Trimeche et al. [27] have introduced aphone-centric approach to what they call a multi-deviceecosystem, while Kawsar and Brush [15] have studied usepatterns of connected devices in private homes. Looking atcommercially available “ecosystems of connected devices”,Levin [18] demonstrates a variety of ways that devices canrelate to each other to form a powerful whole encompassingconsistent, continuous, and complementary experiences ofusers. Because a large part of our use of digital artifactsinvolves displays the visual aspects of digital ecosystemsplay an important role. Previous research on this matterinvolves a taxonomy for multi-person-display ecosystems[25] and studies on how to use mobile devices together withother devices [1, 11]. Related to this, previous research hasalso investigated how collaboration between users can befacilitated in multi-device environments [16, 23].Also grounded in Activity Theory, Bardram [3] presents theActivity-Based Computing (ABC) framework for designingubicomp systems for collaborative work spanning acrossseveral interactive artifacts. In this framework the activity isplaced in the center, and focus is on representing those in acomputerized system in a way that supports them beingpersistent, stateful, and distributed across networkedcomputers so that users can move work activities with themwhile roaming between devices. This is done throughdesign principles of activity-centered resource aggregation,activity suspension and resumption, activity roaming,activity sharing, and activity awareness. Similarly, Nardiand O’Day [19] also put human activity in the center intheir notion of “information ecologies”, which they describeas systems of “people, practices, values, and technologiesin a particular local environment”. In this work, particularattention is furthermore brought to the relationshipsbetween people, not just artifacts, and the habitation oftechnology in a particular location, focusing attentionbeyond a single person interacting with technology, andproposing a sensitivity to the notion of locality. The latterhas also been explored by Bell [4] who uses the term“cultural ecology” to describe the cultural properties ofcollections of interrelated artifacts in particular localities,such as in a museum.Complementing this technical and application-orientedwork, theoretical work has used holistic and ecologicalthinking as a way of understanding multi-artifact systemsand interaction from the point of view that we cannot fullyunderstand an individual artifact if we do not investigate itas a part of a larger whole. Notably, Forlizzi [12], Jung etal. [13], Bødker and Klokmose [6, 7], Bardram [3], Nardiand O’Day [19], and Hutchins [13] all provide importantinsights into the relations and dynamics of people’sorchestration of artifact ecologies. They also emphasize theimportance of inter-artifact relationships to our perceptionof both individual artifacts and artifacts in concert.From the related theoretical perspective of DistributedCognition, which like Activity Theory has its roots inVygotsky’s cultural-historical psychology (cf. [29]),Hutchins [13] takes an ecological approach to how peopleprocess and interact with information and artifacts in theworld, emphasizing the social as well as situated aspects ofcognition, and putting groups of people in the center. Fromthe perspective of Distributed Cognition, people andartifacts are all ecological elements of a cognitiveecosystem, and human knowledge and cognition are notlimited to the individual, but distributed by placing, facts,knowledge and memories onto artifacts, people, and tools inour environment. This perspective is particularly usefulwhen analyzing and designing complex collaborativesystems involving multiple people and artifacts/toolsoperated in concert as a larger whole, such as the controlsystems on a ship [13].Placing the artifact in the center, Forlizzi [12] introduces a“product ecology” framework to describe interrelatedsystems of a product, other surrounding products, people,activities, place, and social and cultural context of use. Thisframework facilitates analyzing and describing the dynamicsocial relationships surrounding interrelated products, andexamining how users adapt to it. With a focus on theindividual user in the center, Jung et al. [13] introduces“personal ecologies” of artifacts as a way to understand aset of digital artifacts and how they interconnect with auser. They define the personal ecologies as “a set of allphysical artifacts with some level of interactivity enabled by2
Putting the interaction between users and digital artifacts inthe center, four basic structures of relationship emerge: 1)many users interacting with many artifacts, 2) one userinteracting with many artifacts, 3) many users interactingwith one artifact, and 4) one user interacting with oneartifact. These are illustrated in Figure 2.These frameworks and theoretical perspectives are allvaluable for understanding and designing multi-user andmulti-device digital ecosystems in that they promoteexploring a holistic view including artifacts, people,activities, groups, etc. This allows us to describe andunderstand, theoretically, different holistic aspects ofubicomp systems involving multiple users, devices andactivities, such as the complementing properties of differentartifacts for a group of friends finding their way usingseveral map artifacts [6], the mediating role of shared workactivities amongst collaborating roaming nurses in ahospital [3], or the joint sense-making and interactiontaking place in the operation of a complex control system[13]. However, we feel that these existing frameworks andtheoretical perspectives are limited in their ability to informspecific understanding and design on the level of specificuser interface and interaction techniques. While we are ableto understand the dynamics and relationships betweenartifacts, people, activities, groups on a high level ofabstraction using, for example, Activity Theory orDistributed Cognition, such understanding rarely explainswhat specifically makes a particular interface design orinteraction technique perform well, in a way that is easilyapplicable to a different design case. It is often also verydifficult to translate such abstract understanding intoconcrete interaction design for digital ecosystems.Figure 2. Four basic structures of relationshipsbetween users and digital artifactsMany Artifacts and Many UsersOut of these four structures, it is mostly the three structuresinvolving many artifacts or many users that are of interestfor digital ecosystem interaction design as somethingdistinct from traditional personal computing. Single-userinteraction with one dedicated artifact, for example a userand a PC application, is a well-researched case within HCI,and present well-known challenges and possible solutionsfor researchers and designers. Hence, while all fourrelationships appear in the interactions within a digitalecosystem, we will narrow our focus to interactionsinvolving many users or many artifacts, or both.FOCUSING ON INTERACTIONContributing to the work on digital ecosystems, we suggesta complimentary way of understanding digital ecosystemswhere we focus explicitly on the interaction between usersand digital artifacts instead of looking at ecosystems aspeople, products, activities, or groups. From such aperspective, digital ecosystems can be described as anetwork of nodes that interact with each other throughrelationships [22]. For a digital ecosystem, this networkconsists of users and digital artifacts, dynamically boundedby the users’ activities (Figure 1).Sequential and Simultaneous InteractionIn looking at users’ interaction with several digital artifacts,this can be further divided into two, depending on whetherthe interaction is sequential or simultaneous. In sequentialinteraction, users will start doing something with one digitalartifact, and then continue it with another. This could, forexample, be checking email on one’s phone, and thenmoving on to a laptop computer to read a particular one indetail. In simultaneous interaction, users do somethingusing several digital artifacts at the same time. This could,for example, be getting an overview of one’s calendar onthe large screen of a desktop computer while sending ameeting invitation from one’s phone.THE 4C FRAMEWORKFigure 1. Digital ecosystems of users and digital artifactsbounded by activities and related through interactionsBased on our analysis of ecological thinking and existingcases of digital ecosystems that are either commerciallyavailable or reported in the literature, we have developedthe 4C framework for describing, explaining, and informinginteraction design in digital ecosystems. The framework isdepicted in Figure 3, and described and exemplified in thefollowing sections.A user activity could, for example, be working at the officeincluding the artifacts being used in this location, or it couldbe watching Netflix at home including other members of thehousehold and their relevant digital artifacts. Both activitiescould involve several users and several artifacts.3
Figure 3. The 4C framework of principles for interaction design in digital ecosystemsor the projector in a meeting room, which does notnecessarily need to know who you are to provide its serviceor functionality. While this might seem trivial, we considerthis principle as important as personalization because webelieve that a decision to implement a generalization ratherthan a personalization principle can have great impact onhow a digital ecosystem is perceived and used, and becauseforcing a user to login might sometimes be irrelevant.The 4C framework combines the different structuralrelationships of many users and many artifacts with thedifferentiation between sequential or simultaneousinteraction in a 2x2 matrix of four themes of communality,collaboration, continuity and complementarity. Under eachtheme we have identified and listed two specific principlesof interaction design for digital ecosystems. While the fourthemes are meant to be comprehensive within the scope ofthe framework, the listed interaction design principles arenot complete, but open for additions and refinements.CommunalityThe first theme covers situations of sequential interactioninvolving several users. We refer to this as cases ofcommunality in reference to communal computing whereartifacts are shared between users, but with an emphasis oneach user interacting with the artifact at a time. This couldbe in a public setting where communal computing has, forexample, been used to describe computer resources madeavailable in libraries [10]. The concept can however beapplied widely to, for example, public displays or tabletsshared among family members.Figure 4. Personalization and generalization: Kodak’sFacebook photo-printing kiosk, and a typical ticket machineWe have identified two distinct principles that facilitatecommunality in different ways. The first is personalization,meaning that the relationship between users and artifacts isindividual and tailored to each person. A common exampleof this is the use of accounts or profiles on network serviceslike Facebook, where each user has access to somethingparticular to their person. The second principle isgeneralization, meaning that the relationship between theartifact and the users is not a personalized one but the sameas for everybody else. We use the principle to describecases where an artifact can be used immediately by anyonewithout “knowing” who the user is. This could, forexample, be in the case of a ticket machine at a train station,CollaborationThe second theme covers situations of simultaneousinteraction by many users, which we refer to broadly ascases of collaboration. Collaborative use of digital artifacts[16, 23] has long been a topic within the area of CSCW,and like this research field we use the term collaboration ina broader sense than describing just ways of workingtogether and coordinating activities, to include all kinds ofsocial computing situations for recreational and socialactivities. Broadly speaking, simultaneous collaborativeinteraction in a digital ecosystem is for the purpose ofdoing, or engaging in, a shared activity or task involvingjoint interaction with one or more shared digital artifacts.4
well-researched ubiquitous computing concept of allowingusers to switch between artifacts by transferring the state oftheir activity or interaction from one artifact to another,either partially or completely [3]. This is, for example, seenin Amazon’s Kindle, where people can continue reading abook on one device from where they left off on another one.Another example of migration is Apple’s AirPlay whereone might browse media on an iPhone, and pass on itsplayback to a large display or sound system.Figure 5. Division in a multi-player car racing game (left), andmerging through shared controls on a mixing desk (right)Within the theme of collaboration we have identified twodistinct principles of division and merging. Division meansthat the interaction with an artifact is split between usersand provides them with individual parallel points ofattention. The most common example of this principle isspatial partition of graphical user interfaces, such as splitscreen views in multi-player video games, or large displayswith separate workspaces, where people can interactindependently through different views. Division can also bedone by other means, such as separated audio channels,where different users hear different sound. The principle ofmerging means that several users’ simultaneous interactionwith an artifact is done “over the top of each other” throughone shared representation. This could, for example, be inthe case of a multi-user board game on a shared tabletwhere several users’ interactions are merged visually intoone, or the case of a large mixing desk where differentpeople can jointly and simultaneously manipulate the soundof different instruments through merged physical controls.Figure 6. Synchronization of media files using iCloud (left),and migration of media file playback using Airplay (right)ComplementarityThe fourth theme is about simultaneous interaction withmultiple artifacts. We refer to this as complementarity,where interaction with one artifact adds to the interactionwith another artifact, and these jointly make up a largerwhole. This is similar to the concept of “composition” incross-platform design described by Wäljas et al. [31], butwith focus on simultaneous interaction, and the effectcreated when using several digital artifacts together as one.We have identified two principles of complementarity indigital ecosystem interaction design. The first one isextension, which describes the case where one digitalartifact directly adds to another one. This could simply bethe use of several smartphones and tablets to create a largerdisplay area [18, 20], or the use of what has been called a“companion app” to provide supplementary functionalityfor another device. As an example of this, Adobe’s NavApp moves selected tools in Photoshop onto an iPad,making it work in a similar way to a painter’s palette inconcert with the canvas on an easel. The other one is remotecontrol, where complementarity is achieved by one digitalartifact simply controlling another, as is well known fromtraditional TV or sound system remotes. While perhapsseemingly trivial, this principle of interaction in a digitalecosystem is in fact very common, and many companionapps provide exactly this functionality for, for example,media players or home automation systems.ContinuityThe third theme addresses situations of sequentialinteraction involving several artifacts. We refer to this ascases of continuity, where an interaction starts on oneartifact and then continues on another. This enables peopleto use several artifacts [31] and re-access content on adifferent device [2]. Such continuous interactions can befacilitated by keeping data consistent across artifacts, or byallowing activities started on one artifact to be continued onanother one, exactly where it was left off.Our framework describes two principles of continuity insequential interaction within a digital ecosystem, namelysynchronization and migration. The principle ofsynchronization simply means that data and data structuresin a digital ecosystem is kept consistent across all devices.When an artifact synchronizes with an ecosystem, contentand its organization is replicated to this artifact, and whenchanges are made on one artifact this is applied to all otherartifacts as well [17]. Well-known commercial examples ofthe synchronization principle in digital ecosystems are“cloud-based” storage services like Dropbox, Google Driveand iCloud where one’s files are automatically replicated,or made accessible, across devices. Other examples includeemail and calendar services that facilitate continuity in theinteraction from one artifact to another by synchronizinginformation content. The principle of migration refers to theFigure 7. Adobe’s Nav App extending Photoshop onto a tablet(left), and a remote control App for a media center (right)5
USING THE 4C FRAMEWORKIn this section we will demonstrate the value of the 4Cframework for understanding an existing digital ecosystem,and for designing a new one. As an example of an existingdigital ecosystem we have chosen the Netflix online videoon demand service. Exemplifying the design of a new onewe describe how the framework was used to design a multidevice shared music player system for parties or othersocial gatherings.Figure 9. When accessing Netflix, the user can specify whothey are, which influences what content is presented to themUnderstanding a Digital Ecosystem: NetflixNetflix does not make use of the generalization principle, asthere is no mode where content is not tailored to a profile.Even before the profile-switching functionality wasintroduced, the service revolved around making suggestionsthrough the recommendation engine based on preferencesand viewing history. If one was to implement the principleof generalization in Netflix, this could be done by disablingthe recommender system, or by basing recommendations onsomething other than personal preferences and history ofinteraction, for example, current topics or events.Netflix is an online streaming service that allows users towatch various video content on different devices includingPCs, smartphones, tablets, gaming consoles, and compatibleTVs. Through subscription, Netflix provides a collection ofmovies and TV series that can be browsed online, removingthe need for physical media and the need for users todownload and store media content. The home screen andinterface for browsing content is shown in Figure 8.CollaborationNetflix supports collaborative aspects of use throughmerging but not through division. When several people arewatching Netflix together, their combined interaction, forexample through multiple remote controls, is merged ontoshared artifact, typically a large display viewable by all.Hence they will have to coordinate and negotiate their jointinteractions towards this shared point of focus. The mergingof interactions, however, does not take into account thepersonalized profiles of the different people watching ittogether. Because only one profile can be active on thesame device at the time, a group of viewers have to chooseone of their profiles to use (Figure 10) and the contentpresented is therefore not tailored to the merged preferencesof the group. Likewise, when watching Netflix as part of agroup while using someone else’s profile, this activity is notregistered in ones own viewing history, and therefore nottaken into account in later recommendations. As a commonresult of that, Netflix often recommend a movie or TVshow that one have in fact already watched on Netflix.Figure 8. Netflix provides an interface designed for a largescreen where the user can browse for video content.Netflix is an interesting case for digital ecosystem research.It is sufficiently complex as the service can be distributedacross many artifacts and users, but it is also appropriatelysimple as it focuses on a delimitated activity, and revolvesaround one primary type of content. As a case that involvesmultiple users and multiple artifacts, it can be used toillustrate the insights enabled by the 4C framework. Wewill do this by going through each of the four themes of theframework in relation to the interaction with Netflix.CommunalityNetflix supports communality through personalization butnot through generalization. In order to access the service,users must purchase a subscription plan, which involvescreating an account and using this to log in to the service.The content presented will then be tailored to the usersprofile, reflecting, for example, their geographical locationand what has previously been watched. Extending on itsability to personalize the service, Netflix additionallyallows up to five people in the same household to use theservice individually within one subscription. This is doneby creating additional user profiles within the same account(Figure 9), after which the recommendation engine is ableto provide personalized suggestions to a particular user,facilitating communal use of the shared artifact.Figure 10. Who’s watching? When watching Netflix togetherin a group, people’s individual profiles cannot be mergedNetflix does not use the division principle in its userinteraction. If one was to imagine division in Netflix, thiscould, for example, be done through a picture-in-picture or6
Herein lies another opportunity for Netflix to modify theway the interaction with their digital ecosystem is designed.If a Netflix player knows the composition of the group ofpeople watching it, then the continuous interaction betweendifferent Netflix players through synchronization andmigration can be improveda split-screen view. While this would perhaps not be anobvious thing to offer in a service like Netflix, combining itwith divided audio could enable different people to watchdifferent content on the same large screen display. If thestreaming service was extended to include live TV, divisioncould also be used to facilitate the viewing of multiplesources by multiple people, for example during sportsevents with parallel games, or big news events. Using asplit screen design could also enable divided interactionwhere one user could be watching a video on half of a largeshared display, while another is browsing content, lookingup information online, etc. on the other half.ComplementarityThe final features of Netflix as a digital ecosystem can bedescribed as support for complementarity through extensionand remote control. When interacting with Netflix throughseveral devices simultaneously, these digital artifacts can beused to perform complementary functions extending theinteraction space onto those devices (Figure 12).ContinuityNetflix supports continuity in the interaction across severaldevices in sequence through both synchronization andmigration. Continuous use from one artifact to another issupported primarily because Netflix synchronizes data forthe individual user across different devices. However,because Netflix streams video rather than downloadingwhole media files to the use
Complementarity), which can help understand and design interactions for digital ecosystems by dividing these into 4 themes and 8 principles. The framework emerged by analyzing existing, real-world digital ecosystems driven by the question: What makes a digital ecosystem more than simply a collection of artifacts?
