Transcription
Copyright 2003 IEEE. Published in the Proceedings of the Hawai'i International Conference On SystemSciences, January 6-9, 2003, Hilton Waikoloa Village Big Island, Hawaii.The DISCIPLE System for Collaboration over the Heterogeneous WebAllan Meng Krebs, Mihail Ionescu, Bogdan Dorohonceanu, and Ivan MarsicCenter for Advanced Information Processing (CAIP)Rutgers — The State University of New JerseyPiscataway, NJ 08854-8058 USA 1 732 445 6347{krebs, mihaii, dbogdan, marsic}@caip.rutgers.eduAbstractWith the proliferation of mobile devices we witness anincreasing demand for supporting collaboration amongusers working in the field and in the office. A keycomponent for collaboration in this domain is sharing andmanipulation of information using very different displaycapabilities on the diverse devices. We present a systembased on a distributed repository of shared data objectsand a client-server based infrastructure. The system isrobust to intermittent connections, and a mixture of slowand fast links. To preserve bandwidth, applicationspecific data distribution agents decide what data to sendto the clients. We also present a framework for buildingcollaborative applications for clients with differentdisplay and processing capabilities. We describe exampleapplications implemented both as Java applets to run inWeb browsers and as Java spotlets to run on Palm OSbased handheld computers. Using these applications weevaluated the framework and the results show that theframework is scaleable, offers good performance and hasa high degree of code reusability.Keywords: Collaboration, heterogeneity, mobile devices,wireless communication, agents.1. IntroductionA key component for synchronous collaboration isreal-time sharing and manipulation of information. Mostcollaborative applications provide synchronizationsupport through the WYSIWIS technique [10].WYSIWIS allows one user to point or discuss an objecton the screen that they are confident is visible to theircollaborator. Providing support for WYSIWIS with pastsystems was a difficult but not impossible task becausethe groupware used common platforms. Developinggroupware applications that are interoperable acrossdiverse environments is significantly more difficult andcostly. However, with the recent anytime-anywhereproliferation of computing technology, support forheterogeneity is inevitable. Mobile applications involvingsynchronous collaboration are emerging in many fields,e.g., business, healthcare, military services andtransportation. The classic example is the provision ofjust-in-time assistance between a deskbound expert and amobile fieldworker using a portable device. The mobileworker may work with blueprints while the expert isusing a 3D CAD model to repair a vehicle or theplumbing in a building. Kraut et al. [8], for example,show that fieldworkers make quicker and more accuraterepairs when a remote expert is providing assistance.The heterogeneity of computing platforms hasdimensions of processing speed, storage capacity, inputdevices, display capabilities, response latencies, networkreliability and bandwidth, with network and displayaccounting for the most prominent differences. This paperfocuses on different display capabilities and on networkreliability and bandwidth. We propose a solution forenvironments where clients have: Slow and/or unreliable connections, e.g., wirelessconnection to the Internet. Different display and processing capabilities, e.g.,PDA versus desktop workstations. Slow and/or unreliable connections in the caseswhere data can be distributed over multiple servers.The problems above can be found in areas such asmilitary or civilian rescue or disaster relief applicationsfor distributed operations planning, which often involvemobile devices connected using low bandwidth links.Another area is business applications with mobile users,where the users need to collaborate (perhaps in real-time)on shared resources, such as a corporate calendar and aresource management system.The solution we propose consists of a distributedrepository containing the shared data, a client/serverbased communication infrastructure, and a framework forbuilding collaborative applications for a heterogeneousenvironment. We use agents to synchronize and distributethe data objects, and dynamically load code fortransformation of data between different representations.The clients can run in web browsers as applets, or asspotlets on PDAs and other small devices (spotlet is thebasic user interface class for Java on Palm OS). The ideaof the proposed system is illustrated in Figure 1.
The paper is organized as follows. We first reviewrelated work. Next we describe the architecture of ourDISCIPLE system and the framework we built forheterogeneous collaboration. Then we present details ofthe solutions for the core problems in designing thesystem. Finally, we conclude the paper.makes easier implementation on small devices likePDA’s. [11] does not address the different issues specificfor synchronous groupware or heterogeneity of rability between the existing groupware systemsthat use different group coordination policies or differentsoftware architectures. Dewan and Sharma [2] addressthis issue. Conversely, we address the case where theparticipants’ platform capabilities are significantlydifferent.While the above-mentioned related work addressesmany issues in collaborative systems and groupware,most of them is not concerned with heterogeneity andnone with the special cases for wireless connected smallclients. Our essential principles for heterogeneouscollaboration are that every user’s action is interactivelyand continuously reflected in other users’ workspaces,with a varying degree of accuracy or realism or through aqualitatively different visualization, and that users shouldbe able to work off-line and get their work seamlesslysynchronized with that of the groups, upon reconnect.1.1. Related Work2. System ArchitectureThe need to allow conferees to collaborate ondissimilar terminals was recognized early on by ative work [3]. An early design for heterogeneousgroupware is presented in [7], but it does not deal withplatforms of significantly different computing andcommunication capabilities. Rendezvous [6], GroupKit[12], and several groupware toolkits thereafter use modelview separation so that developers can create models anddrive different views. However, no such implementationis reported.The Visage system from Maya Design [5] is apowerful (single-user) visualization system for creatingcustom visualizations and direct manipulation of largeand diverse datasets. While Visage addresses diverse datavisualization with polymorphic views, it is not explicitlyintended for collaboration in heterogeneous computingenvironments. Recent work on Visage Link [10], anextension of Visage, adds multiuser collaborationcapabilities, but Visage Link does not considerheterogeneous models.A system for data management support is presented in[11]. The main focus is on asynchronous groupware andintegrating awareness support, but it is related with ourwork, as it uses a replicated object store and local cachingto support mobile clients. It differs mainly by having bject), which manages the data associated with aspecific application. Our approach has a clearer separationbetween data and application, which simplifiesdevelopment of different applications sharing the samedata objects. Also, having more lightweight data objectsThe system architecture of the DISCIPLEcollaborative framework comprises four main layers (seeFigure 2): the distributed repositories; the basiccommunication infrastructure; the Manifold heterogeneous environments; and the task-specificapplications. The architecture for DISCIPLE presentedhere is completely different from the earlier versionpresented in [15].Client 1subscribefor eventsClient NSynchroAgentsnotifyabout eventsServer(s)RepositoriesFigure 1: Main components of DISCIPLE system.2.1. Distributed RepositoriesThe role of a repository is to store persistentlyinformation. Repositories are one of the main componentsof the system, and are distributed and partially replicatedover both servers and clients. The main idea of therepository is that each client has a copy (and in somecases the only copy) of the objects that the client isinterested in. In the example of a disaster reliefapplication, the objects could be all objects that arephysically within a 10 miles range. Almost the same ruleapplies to the servers: the servers hold a copy of theobjects that their clients are interested in. Updates to theobjects are propagated between all the entities that hold acopy of the same object. The approach for dealing withthis propagation and with the possible conflicts isdescribed below.The main goal of the server and repository design is tomake the server and repository completely independent ofthe application semantics. This is achieved by using anabstract data type, called UForm (short for “universalform”), which encapsulates the data. The UForm
2.2. DISCIPLE Communication InfrastructureFigure 2: Overview of the DISCIPLE system architecture.essentially consists of a unique identifier and a keyed listof properties1. The interface for the UForm is shown inListing 1. Thus, the repository separates data frompresentation, as in Model-View-Controller, but theseparation is mediated via pure data entities (UForms),rather than via active objects. The content of therepository defines the application state.In addition to the unique ID and the property list, aUForm contains three fields used in merging the stateswhen the client becomes temporarily disconnected fromthe server: dirty, deleted and lastType. The dirtyfield specifies whether or not the UForm was modifiedwhile the client was offline. When an object is deleted, itis not removed from the Repository or from the clientstate, but instead is marked as deleted using the deletedfield.The operations on a repository are defined in a simpleJava interface, which is a common interface for bothservers and clients. The implementation is not bound toany specific data structure. It could be anything from ahash table to an object-oriented database and theimplementations can be different across the clients andthe servers. In the present reference implementation wehave used the hash table, both on servers and clients.The UForms can be linked in complex data structuresby using UForm identifiers as properties in other UForms.Listing 1: The UForm interface.public interface UForm {public void setId(long id);public long getId();public boolean isDirty();public void setDirty(boolean dirty);public boolean isDeleted();public void setDeleted(boolean deleted);public void setType(String type);public String getType();public boolean hasProperty(String key);public Object getProperty(String key);public void setProperty(String key, Object value);public Object removeProperty(String key);}1The concept of UForms is borrowed from [10], which in turn derviesfrom e-forms [1].The communication infrastructure is based on anintelligent reflection server, which reflects messages (usercommands) between the clients. A key part of is thecollaboration bus (cBus) which transmits the commandmessages.The built-in intelligence on the server assures that thecommands are only reflected to the clients that areactually interested in the object the command acts upon,thereby minimizing the network traffic. This is a veryimportant issue as the system is targeted for mobile userswith low-bandwidth connections. In addition to reflectingthe commands to the clients and others servers, theserver’s local repository is also updated by thecommands. The way the server decides whether or not toreflect a command to a certain client is described in asection below.The class hierarchy of the commands used for thecommunication is shown in Figure 3. All the commandsderive from the Command class whose attribute beanIdidentifies the application that originated the command.The commands can be partitioned into two sets. The firstis a set of management commands, for managing theclient’s connection to the server and setting up andmanaging the client’s agent(s) on the server. The secondset of commands deals with changes to UForms. All thecommands for UForm manipulation derive fromUFormCommand.The cBus allows applications that use differentdatasets to share the same server instance (session) andthe same repository on the server. This is particularlyimportant for systems with limited resources that want towork with more than one collaborative application at atime (e.g., a chat and a situation-map application).2.3. Manifold Application FrameworkTo facilitate the development of collaborativeapplications for heterogeneous computing environments,we defined the Manifold framework for buildingcollaborative applications [9]. The framework consists ofa base package with five Java interfaces for the ntrollerdesignpattern.Defaultimplementations for some of these interfaces have alsobeen provided to help developing applications using theframework.The five basic interfaces are intended for use in allimplementations of Manifold regardless of whether thetarget platform is a desktop workstation, a PDA, or amobile phone. An application built with the frameworkcan run as standalone or as an applet in a Web browser.The interfaces are simple enough to be used under thepresent version of J2ME CLDC [14] for Palm Pilot
CommandbeanId : longManipulatorThe Manipulator does the actualmanipulation in response to the useractions. It creates Commands inresponse to the manipulation andpasses them to the Controller CmdUFormCommandErrorCommand(Figure 2) in order to enable off-lineuserName : Stringcondition : ConditionuForms : UForm[]targetId : longerrorCode : longpassword : S tringerrorMessage : Stringwork. The resulting model can bedeviceType : longsetCondition()setUForms()setTargetId()compared to the Model-Viewcondition : ErrorCode()uForms : UForm[]getErrorCode()Controller design pattern. UFormssetErrorMessage()correspond to models and GlyphssetUs erName()getErrorMessage()getUs erName()correspond to views/controllers.setCondit ion()GetUFormsCmdThe local repository is a hash tablegetCondit ion()setUForms()with the entire client’s UFormsgetUForms()rather than being another scenegraph. Figure 4 shows an examplecollaboration diagram, in this casefor creating a UForm:1. The user clicks on theSetUFCmdRemoveUFCmdRemoveUFPropertyCmdSet UFPropertyCmdapplication window to create anuForm : UFormkey : Stringkey : Stringobject. The mouse event is sentproperty : ObjectsetUForm()setKey()to the Viewer.getUForm()getKey()setKey()2. The Viewer invokes the methodgetKey()setProperty()createManipulator() on thegetProperty()current Tool (which in thisexample is an object creationFigure 3: The command class hierarchy.tool). The Tool returns aManipulator that encapsulateshandheld computers. The five basic interfaces are asthe manipulation sequence for creating a specific kindfollows:of object, e.g., a geometric figure.Controller3. The Viewer next invokes grasp() on the returnedThe Controller is the command dispatcher andManipulator. The Manipulator creates the UForm andprovides the interface for communicating with theencapsulates it in a Command, which is returned to theapplication. It accepts the locally generated CommandsViewer.and the remote Commands from peer clients, and4. The Viewer forwards the Command to the Controller.executes the Commands on the local Repository.5. The Controller sends the Command to the serverthrough the DISCIPLE Client module. Then,Glyphdependingon the concurrency strategy, it either waitsA Glyph is the graphical representation of a UForm.fortheCommandto return back from the server orThe scene graph of an application may also containprocessitimmediately(step 6).PolyGlyphs, which correspond to branch nodes and are6. The Controller invokes execute() on the Command.containers for collections of Glyphs.7. This causes the Command to invoke setUForm() onViewerthe local Repository, which adds the UForm object toThe Viewer is the canvas or window, where the glyphsthe Repository.are displayed, e.g., icons on a map. The Viewer also8. The Repository then informs the Viewer that an objecthandles user events.has been added to the Repository. How this is done isToolnot defined in the basic framework, and it is left to theThe Tool encapsulates the current manipulation modeapplication developer to define the actual mechanism.and defines the kinds of actions can be performed on the9. The Viewer adds a view of the UForm (a Glyph) to itsUForms. Typical tools include create tool, select tool, andscene graph and links it together with the UForm.property change tool.Again, this is not defined in the basic framework, but areference implementation is )convert()isReversible()
A similar collaboration diagram applies for manipulatingthe properties of the UForms.3. Distributed Repository SynchronizationThe idea behind the distributed repositories is that dataresides on the servers with clients being interested in thespecific data objects. A client retrieves a copy of dataobjects into its own repository while working. Therefore,data is moved and/or copied around between the serversand clients.Each user gets only a subset of shared data specifiedby the data distribution conditions. When a user firststarts the collaborative application, a default condition isset for the user where the user receives all the data in thesession. At run-time the user can change the conditionsvia a user interface offered by the framework. When theconditions are set, the user is notified only about themodifications of the UForms that satisfy the conditions.We call these UForms interesting UForms. If other usersmodify the global application state, only the data relatedto the interesting UForms is forwarded to this user. Theconditions are declared as a set of rules for each clientconnection to decide when to move or copy data. Agentsfor event distribution (described below) use theseconditions to acquire the objects for the application.3.1. Event DistributionOnce the collaborative session is set up, the useractions are distributed as commands to the remote clients.The simplest way to transmit the commands between theclients and the servers is by using Java Serialization. Weemploy this method in case of the applet implementations.However, the J2ME CLDC does not support serializationand another method must be used for the Palm Pilotimplementation. Our present version converts eachcommand into a text string that is sent to the peer where itis converted back into a command.3.2. Adapting Data to Device via TransformersAs the objects are to be visualized on very differentdevices, with very different computing andcommunication capabilities, some transformation ofcontent must be performed. If some clients are visualizingin 3D and others in 2D, the 3D coordinates have to betranslated into 2D and vice versa. The simple solution ofjust discarding or adding a z coordinate would not worksince, for example, the rotations around the x or y-axiswould be very complicated to handle.A solution could be to keep data the same on alldevices and transform locally if needed. This may soundlike a feasible solution, but is not the best idea. Naturally,a 3D workstation with all support from Java 3D fortransformation can fairly easy transform coordinates, but9: addGlyph(Glyph)1: Mous eEvent2: createManipulator(Object)User: Viewer: Tool3: gras p(Object)4: tor8: uFormSet(UForm)5: Send Command to server6: execute(Repository):Command7: setUForm(UForm):RepositoryFigure 4: Collaboration diagram for creating a newUForm.if the z coordinate is not present in the first place, onecannot do much about it. The original data must be in a3D form. Transforming coordinates on the 2D clientsmight not be a good idea either, because the clientsrunning 2D most likely do it for performance reasons andcannot handle additional burden of the transformation.Another problem occurs if Palm Pilots are involved. Evenif visualization is only done in 2D, the J2ME CLDC doesnot support floating-point numbers, so the coordinatesmust be converted to integer numbers, thus making itimpossible to keep the same data format on all the clients.The result is that the distributed repositories are matchedto the platform capabilities and user interests.The command events must be transformed to aplatform-specific format before being delivered remotely.The command transformation must be performed in sucha way to maintain the repositories in synchrony. Theprocess is illustrated in Figure 5. Let us assume that theuser interaction at client α results in a command cα. Acommand consists of a sequence of primitive operationson the repository graph G (see UFormCommands inFigure 3). These are: (Op1) create a vertex ui, (Op2) deletea vertex uj, and (Op3) modify vertex property. Using atransformation fαβ, the command cα gets transformed to cβfor delivery to site β. The transformation takes place intwo stages. In the first stage the transformer decides onwhether a particular primitive operation in the commandapplies at the remote site. The result of the decision isbinary: true or false. For example, if cα is as follows:cα Op1 Op1 Op3 Op1 Op2 Op3then cβ may be as follows:c1β f 1αβ(cα) Op1 Op3 Op1 Op3The agents for event distribution (see below) performthis task. In the second stage, the properties associated
Client βcβ fαβ(cα)Client αGαGβcαServerG:u1 u2 u3rGΩ cβunFigure 5: Command transformations. The structure ofthe repository graph G with UForms ui is shown in thelower left corner.with the operations of the type Op3 must be converted tomatch the remote domain. The final result will be:cβ f 2αβ(c1β) Op1 Opβ3 Op1 Opβ3The property converters are defined by a simpleinterface with methods to convert an entire UForm oronly a specific property.As the server does not know anything aboutapplication-specific details, the application itself has toprovide the necessary transformers in the form of agentsand property converters. This is done when a clientconnects to the server for the first time. As part of thelogin command sent to the server, the client can specify aURL where the server can load the necessary agents andconverters if any. The server loads and instantiates theagents and converters to use them when communicatingwith the client. The server calls a method on thecommands called convert with the appropriate converteras argument. The convert() method of the commandsthen calls the necessary methods of the converter,depending on the kind of command.3.3. Agents for Event DistributionAs our system is targeted for wireless environmentswhere network resources are sparse, we try to minimizethe network traffic by using agents for event distribution.The agents filter the events sent to the user according tothe users requirements, thereby saving network resources.Each user decides on what subset of the data he or she isinterested in receiving notifications about, and set up anagent at the server to decide which updates are sent backto the user. In order to respond to the very dynamic natureof wireless networks our agents support priorities forevent distribution. When configuring the agent, the usercan specify priority of different subsets of the data he orshe is interested in, so updates about the least importantdata is only sent if network conditions permit it, and themost important updates are sent even at very bad networkconditions, using three levels of priority. This allows theevent distribution to degrade gracefully when networkconditions are getting worse.An example of an application with users interested indifferent subsets of the full data set is a firefighterapplication for large operations, like forest fires, in whichwe have clients at Fire Company (one fire engine), FireBattalion, and Fire Division level. Each Fire Captain(leader of a Fire Company) will want to see location ofhis fire fighters, neighbor fire companies, and known firefront close to the position of his company. The BattalionChief will be interested in seeing only each of thecompanies as a unit and not the individual fire fighterswithin. Also, the chief might want to see his neighborbattalions, the entire fire front within the range of 10miles and all known evacuation vehicles within the rangeof 30 miles.These hierarchies in the real world impose also ahierarchy in the collaborative applications. Although thedata is the same, the participants should be able to specifythe subset of the data the respective participant isinterested in. Moreover, this subset is dynamic, since theconditions can change. For example, the companycommander can become interested in specific squads iftheir mission is critical for the company’s mission.The agents are specified upon login of the client,where the initial condition and the URL of the agent touse, is sent to the server. The server loads the agent andinitializes it with the initial condition, where after theagent will filter the events sent to the client. The user canchange the conditions dynamically in response tochanging requirements. Each agent is application-specificand knows the structure of the UForms that are exchangedbetween the clients and the server. In this way, the serverremains completely independent on the applicationsemantics. The server keeps an instance of the agent foreach client. The agents are simply Java classes that haveto implement a specific interface in order to collaboratewith our system.An alternative solution would be to use expert systemsthat are able to evaluate at run-time the conditions sent bythe clients. One of such systems is JESS (Java ExpertSystem Shell) [4]. Integrating such complex expertsystems with Manifold is part of our current work.3.4. Operation under Intermittent ConnectivityA major problem with handheld computers, and ingeneral with the devices based on wireless networks isthat the connections can frequently be temporarilyinterrupted or broken and resumed later. For acollaborative application, it is very important that theusers are able to continue working even when theirdevices are disconnected from the network and then, atreconnecting, their work should be merged with theglobal state.We implemented a heartbeat protocol where clientssend periodically heartbeat messages to the server. If theserver does not reply for a constant number of time
intervals, the client assumes that the connection is down.The user can work offline as normal, but we mark all themodified objects (including the newly created ones) asdirty. On the server side, if it does not receive theheartbeat message for a constant period of time it can alsoassume that the connection is down and does not sendnotification messages to that particular client until itreceives a login request.The client then starts sending ping-like packets todetect when the connection to the server is restored. Oncethe server responds, meaning that the connection isreestablished, the client sends to the server its actual state,as recorded in the local repository. The server resumes theconnection, receives the state and executes the algorithmfor merging the states. We designed and implemented anew algorithm for merging the states of the server and theclient. A sketch of the algorithm is shown in Listing 2.An interesting situation can arise. Suppose two users,A and B working on the same whiteboard. Initially, bothusers are online and the common state contains arectangle, R. Suppose now that A goes offline and, whileit is offline, it deletes the rectangle R. Meanwhile, user Bmoved R to a new position. What should happen when theA goes back online? The algorithm has to decide whatstate of the object R to keep. This problem resembles toconflict resolution in concurrency control and cannot havea general solution since it depends on context. A possiblesolution is adopted in our algorithm as follows. We assignto each user a priority and we keep in the lastType fieldof each UForm the priority of the last user that accessedthe object. In our case, if the priority of the user A wasgreater than that of user B, the object will be deleted atmerging, and will be moved in the opposite case. Otherconflict resolution policies can be applied that has notbeen the main issue of our research.4. EvaluationTo evaluate the DISCIPLE system and Manifoldframework we developed two applications forheterogeneous environments. The development of theapplications was an evaluation in itself, assessing thefeasibility of using the Manifold framework to developapplications for heterogeneous environments. The twoapplications were also used to measure performance ofthe DISCIPLE system.4.1. Example ApplicationsThe two applications are an application forcollaboration on a map and a simple chat application.They are implemented both in applet and in spotletversions for running in a Web browser and on a PalmPilot, respectively.Listing 2: Pseudo-code for state merging algorithm.UFormArray MergeStates(UFormArray clientState, UFormArray serverState) {// resultState is initially emptyUFormArray resultState new UFormArray() ;for (int i 0; i clientState.size(); i ) {UForm clientForm clientState.getAt(i);if (clientForm.isDeleted) {if (serverState.contains(clientForm)) {UForm serverForm serverState.get(clientForm.getId());if ( ! serverForm.isDeleted) {serverForm.isDeleted true;}} else { // serverState does n
application, the objects could be all objects that are physically within a 10 miles range. Almost the same rule applies to the servers: the servers hold a copy of the objects that their clients are interested in. Updates to the objects are propagated between all the entities that hold a copy of the same object. The approach for dealing with
