Transcription
RainMan: A Workflow System for the InternetSantanu Paul, Edwin Park, and Jarir ChaarIBM T. J. Watson Research CenterP.O. Box 704Yorktown Heights, NY 10598AbstractAs individuals and enterprises get interconnected viaglobal networks, workflows that scale beyondtraditional organizational boundaries and executeseamlessly across these networks will become relevant.We address the problem of designing a scalableworkflow infrastructure for the Internet that supportsboth flexibility in workflow participation andinteroperability between heterogeneous workflowsystem components.RainMan is a distributed workflow system developed inJava that lives naturally on the Internet. RainMan is aloosely-coupled collection of independent services thatcooperate with each other rather than a monolithicsystem. Some of the useful features of RainMan arebrowser-based workflow specification, participation,and management, and dynamic workflow modification.The RainMan system is based on RainMaker, ourgeneric workflow framework that defines a core set ofwell-defined interfaces for workflow components.1. IntroductionWorkflow systems are essential to organizations thatneed to automate their business processes [Silver95].The attraction of these systems is that they helporganizations specify, execute, and monitor theirbusiness processes in an efficient manner overenterprise-level networks [FlMa, VisFlo, InConc,ActWork]. Workflow systems provide improvedthroughput and tracking of processes, and betterutilization of organizational connected via global networks such as the Internet,they will attempt to team up in various ways to sharework and processes across traditional organizationalboundaries. An Internet infrastructure that enables suchinteractions as workflows would be of great value;unfortunately, traditional workflow systems designedfor centralized workflow execution do not lendthemselves well to these new workflow applications.Internet-wide workflows require an infrastructure thatsupports decentralized workflow execution, fication, and low-cost workflow participation. Theproprietary and monolithic design of current workflowsystems makes it very difficult to address theserequirements.The Workflow Management Coalition (WfMC) hasproposed a reference architecture and defined interfacesfor vendors of traditional workflow systems tointeroperate [WfMC]. The WfMC standard is a usefulstep in the direction of interoperability, however, itsmain shortcoming is that it promotes a monolithicworkflow system architecture that is not flexible orscalable enough to address the needs of Internet-wideworkflow applications which must necessarily beloosely-coupled.We designed RainMan as a distributed workflowsystem for the Internet. The system comprises acollection of independent and lightweight services onthe network. Our Java implementation uses openstandards and Web browser-based user interfaces. Weare using RainMan to experiment with a range ofinteresting features such as dynamic workflowmodification which allows a workflow graph to bechanged during execution, disconnected participationwhich allows participants, designers and administratorson the Internet to be infrequently connected to thenetwork, and downloadable workflow execution whichallows workflow subprocesses to be downloaded acrossthe Internet on demand.RainMan is based on RainMaker, our generic workflowframework that defines a core set of abstract interfacesfor workflow components. The main purpose ofRainMaker is to facilitate the design andimplementation of flexible, interoperable, and scalableworkflow system ionEvalPaymentPlanFigure 1: A Business Loan Approval Workflow
ProcessDefinitionTool2. Workflow Systems BackgroundThe traditional use of workflow systems has been in theautomation of highly structured, process-basedapplications that are common to banks and insurancecompanies. Figure 1 shows a typical (simplified)BusinessLoanApproval workflow in a bank. Workflowsystems provide extensive support for the design andautomation of such applications in an efficient manner.Some of the important features they provide are:1. Specification Tools: These are used to specifyprocesses using high-level specification languages.There is no commonly-accepted basis for specificationlanguages, each workflow system imposes its ownlanguage which can be based on directed acyclicgraphs, rules, state machines, Petri-nets, or other anyformalism. In general, it is common practice to programworkflows visually as a directed graph where the nodesrepresent activities and the arcs represent control anddata flows between activities.2. Execution environments: A workflow systemprovides an execution environment for workflowinstances; this involves interpreting the process bystepping through the activities and assigning them toappropriate organizational resources such as humans,applications, and other workflow systems.3. Audit Logs and Tools: These are necessary tomonitor and track the progress of workflow instancesthrough the workflow system.Admin/MonitoringToolsinterface 1interface 5WorkflowServerinterface 2WorkflowServerinterface 4interface e 2: WfMC Reference ModelThe architecture defined by the WfMC standard is notwell suited for workflow execution on the Internet. Themain problem is that the workflow server is monolithic.It is responsible for process execution, auditing,management of the organizational directory, anddistribution of activities to appropriate participants(performing role resolution as necessary).Most importantly, the server is responsible for hostingand managing worklists on behalf of all participants.This is problematic; since worklists are hidden withinthe workflow server (not externally addressable),activities can be sent only to worklists that reside in thesame workflow server. Clients participating in multipleworkflows on heterogeneous servers must have aworklist on each server and must manage each of theworklists (Figure 3).WorkflowServerprocess wServerWorkflowServerprocess interpreterworklists4. Worklist Management: Management of worklists;worklists are persistent storage locations where humanparticipants receive work assigned to them.In effect, traditional workflow systems are sophisticatedprocess specification and execution environments thatallow organizations to run process-based applicationsefficiently, within the context of a closed environmentover which the workflow system has significantauthority.In response to a growing number of proprietaryworkflow systems, the Workflow ManagementCoalition has defined a set of interfaces to enableworkflow system interoperability. These interfaces aretied to a workflow system architecture known as theWfMC Reference Model (Figure 2) which definesinterfaces between workflow servers and othercomponents such as process definition or specificationtools,workflowclient applications,invokedapplications, administrative tools, and other workflowservers. The specific details of these interfaces aredescribed in [WfMC].DirectoryInternetWorkflowServerinterface 2WorkflowClientFigure 3: The Pull ModelIn addition, the workflow client application mustmanage multiple connections to each of these workflowservers since the architecture assumes a continuouslyconnected workflow client. This is not a scalable orflexible solution, especially if one is to participate in alarge number of workflows on the Internet, or use thinclients or PDAs, or operate in a disconnected mode. Ananalysis of the WfMC architecture reveals additionalproblems for decentralized workflow execution;interested readers may examine them elsewhere[Paula97, Schu96].
3. New Workflows on the InternetIn RainMan, we explore the potential of the Internet toenable decentralized workflow execution viainteroperable workflow components that reside acrossthis global infrastructure. We hope to enable new kindsof workflows involving dispersed individuals, multipleorganizations, scattered network resources, andheterogeneous workflow systems. A few motivatingexamples of workflows that should be possible over theInternet are presented here.between heterogeneous servers are possible. In fact,Interface 4 of the WfMC standard is designed to enablesuch workflow interactions. However, the drawback ofthe Interface 4 approach is the high cost of setting upsuch interactions, and significant pre-agreementrequired between participating servers.Bank: Loan Approval valPaymentPlanInternetConsider a virtual team of IT consultants from differentorganizations, scattered across the globe, working on aproject that is coordinated via workflow. Theconsultants may be mobile and intermittently connectedto the network. Irrespective of location, the projectleader must monitor the workflow and work assigned toconsultants must appear on their worklists. Eachconsultant may participate in many other workflows atthe same time.In such decentralized workflows, participants mustreceive work from multiple workflows usingheterogeneous clients ranging from desktops to laptopsto PDAs (Figure 4). The work distribution model needsto be different from that proposed by WfMC. Instead ofparticipants connecting to workflow servers explicitly topull their work, the workflow infrastructure mustautomatically route work to them over a global network.Process Complete (Results)Start Process (Data)CheckAssetsResultsEvalApplCheckDebtsEval Firm: Check Credit Ratings ProcessFigure 5: Peer-to-Peer Workflow ExecutionFinally, as electronic commerce applications becomeavailable on the Internet, it is conceivable thatworkflows may be downloaded for just-in-timeexecution. Downloadable workflows make senseespecially in cases where pre-installing and maintainingworkflows is not cost effective. For example, consideran education brokerage service on the Internet[Hama96] that specializes in locating custom educationservices (Figure 6).DownloadEducation BrokerageWorkflow Client (disconnected)DownloadNetwork servicesCourse materialCompleteStartRequest for Proposal ementsCompleteRequest for Proposal BStartContent Provider AWorkflow ServerBudgetInternetDownloadWorkflow ServerInternetWorkflow ClientWorkflow ServerWorkflow ClientFigure 4: Decentralized Workflow ExecutionNext, consider the BusinessLoanApproval workflow inFigure 1 running on a bank’s workflow server. TheCreditCheck step may itself be a nested subprocess thatis delegated to a CreditEvaluation firm with a workflowserver supplied by a different vendor. During execution,the bank’s server would notify the firm’s server to startan instance of its local CheckCreditRatings workflow.On completion of the subprocess, the bank’s serverwould resume its suspended workflow (Figure 5). Withglobal connectivity, these peer-to-peer workflowsContent Provider BFigure 6: Downloadable Workflow ExecutionIn a plausible scenario, the Brokerage workflowdownloads a RequirementGathering subworkflow to theclient organization and requests its execution. Next, theBrokerage workflow downloads the requirementsgathered to content providers along with aRequestForProposal subworkflow that the latter mustexecute to create the proposal. At the end, the brokeragecompares all the proposals received and notifies theclient.To enable workflows such as these on the Internet,significant issues related to security and organizationalprivacy must be addressed. Such services, based onstate-of-art distributed systems security considerations,
will have to coexist with the workflow infrastructure.4. RainMaker Workflow FrameworkRainMaker is a generic workflow framework we havedeveloped to build interoperable workflow components.The details of the framework are available elsewhere[Paulb97]; only a brief outline is presented in thissection.RainMaker identifies four important abstractions withinthe workflow domain. Workflow instances areconsidered Sources, or service requesters. Sourcesgenerate Activities, or service requests that aredelegated out. Instances of humans, applications,organizations, and other entities that handle thesedelegated requests are considered Performers, orservice providers. Performers manage units of workcalled Tasks, which implement the delegated requestsissued by Sources. It is an important characteristic ofthe workflow domain that Tasks can be long-running. Acentral feature of RainMaker is that a Task may be aworkflow instance that recursively acts as a Source; thisprovides natural support for delegation of subprocesses.The core RainMaker interfaces that help support thisexecution model are:PerformerAgent: An abstract interface that isimplemented by Performers on the network. Theinterface provides mechanisms for delegating,controlling, and querying Tasks on the Performer. SourceAgent: An abstract interface implemented bySources on the network. It provides a callbackmechanism for Performers to return the results ofTasks to Sources.issues Task requests, and handles responses fromPerformers. The interfaces also describe the controlmechanisms by which Tasks can be suspended,resumed, and aborted; and query mechanisms by whichtheir status can be tracked. The interaction betweenSources and Performers is shown in Figure 7.PerformerAgent ()TaskID::createTask(SourceAgent source,ActivityID activityid,TaskRequest taskreq)AbortID::abortTask(TaskID taskid)SuspendID::suspendTask(TaskID taskid)ResumeID::resumeTask(TaskID taskid)TaskStatus::queryTask(TaskID taskid)List(TaskID)::listTasks()Table 1: PerformerAgent interfaceSourceAgent interfaceCompletedID::completedTask(PerformerAgent performer,ActivityID activityid,TaskResponse taskresp)RefusedID::refusedTask(PerformerAgent performer,ActivityID nt performer,ActivityID activityid,PerformerAgent newperformer,TaskID rAgent performer,ActivityID activityid)Table 2: SourceAgent interfaceTaskrequestSourcecreate TaskPerformerTask ask completeTaskresponseFor the remainder of the paper, the italicizedSourceAgent and PerformerAgent refer to theRainMaker abstract interfaces. The terms Source andPerformer are used generically to refer to entities (orobjects) that implement the RainMaker SourceAgentand PerformerAgent interfaces respectively.The PerformerAgent and SourceAgent interfacesdescribe how proprietary Sources and proprietaryPerformers can interact with each other (Tables 1 and2). In essence, the PerformerAgent interface concealsthe internals of how a Performer or service provideractually performs Tasks in response to the Taskrequests. Symmetrically, the SourceAgent interface is acallback interface implemented to conceal the internaldetails of how the Source actually generates Activities,Figure 7: Source and Performer InteractionIn addition to the core interfaces, RainMaker alsodefines the Worklist interface (Table 3). Worklists arean important workflow metaphor similar to electronicmail boxes; they provide a mechanism for humanparticipants to access work assigned to them byworkflow systems. In the case of RainMaker, Performerentities that represent human participants on thenetwork can implement the Worklist interface and allowparticipants to view and access the Task Requests sentto them by various Sources.
Worklist x()WorkItemRequest::getWorkItem(WorkItemID wid)void::completedWorkItem(WorkItemID wid,WorkItemResponse wresp)void::refusedWorkItem(WorkItemID wid)void::forwardWorkItem(PerformerAgent newperformer,TaskID taskid)Table 3: Worklist interfaceThe strength of the RainMaker framework lies in itsgeneralized push model of Task distribution that allowsTasks to be delegated to Performers independent oftheir implementation. This aids the interoperability ofheterogeneous workflow systems and components.Implementations of the PerformerAgent interface canrepresent arbitrary service providers: humans,applications, roles, organizational units, and workflowsystems. Implementations of the SourceAgent interfacecan embody heterogeneous workflow applicationsdescribed as rules, as control/data flow graphs, and evennon-workflow applications such as collaborations orhuman Sources.Builder &Monitor AppletstartSourcedesigned to be replaceable; it can be reimplemented ontop of any messaging system. RainMan runs on aTCP/IP token ring network of Wintel machines.5.1 RainMan User Interface ComponentsThe user interface components of RainMan allowworkflow users to interact with the runtimeenvironment. The currently available ones are theBuilder, the Worklist Client, and the Administrator, allimplemented as Java applets. This makes anyJava-compliant Web browser a usable client for allRainMan user interfaces, and has the additional benefitof making RainMan user interface components portableand hardware-independent. The Builder and WorklistClient are described in this section.5.1.1 RainMan Builder AppletThe RainMan Builder (see Figure 9) in its currentincarnation is a single Java applet that combines threefunctions. It is an interactive graphical environment forspecifying workflows as directed, acyclic graphs.Performers are assigned from the RainMan directoryservice view available within the Builder. Theservice-specific aspects of an Activity are handled usingthe JavaBeans component object model. Workflowgraphs can be saved to and loaded from a workflowspecification stClient AppletWorklistClient AppletFigure 8: RainMan System Design5. The RainMan SystemThe RainMan system (see Figure 8) is a distributedworkflow system prototype written in Java usingRainMaker interfaces. It consists of a loosely- coupledcollection of distributed lightweight services required todeliver workflow functionality to Internet users. Itcurrently consists of approximately 60 Java classesdeveloped with Java JDK 1.1.4 and it uses Java’sRemote Method Invocation (RMI) for transportbetween distributed components. The transport has beenThe Builder also acts as a workflow graph interpreter(Source) and implements the SourceAgent interface. Foran executing workflow, the Builder posts Task requeststo specified Performers. On receiving notification ofresults, it inspects the workflow graph and posts thenext set of Task requests. Since the Builder dynamicallyinterprets the workflow graph, it is possible to changeor refine the ‘downstream’ specification of theworkflow graph even after the workflow has beenstarted. This is a useful feature that enables thedefinition and execution of ad hoc processes, anddistinguishes RainMan from traditional workflowsystems. It allows for dynamic updates to the workflowgraph to deal with emerging scenarios.This is particularly relevant to decentralized workflowexecution on the Internet, because we expectPerformers to have significant autonomy over theirinternal domains and hence be capable of raisingexceptions in response to Task requests from Sources.In addition, Performers can change their capabilities, aswell as join or leave the network at will. Workflowsystems that cannot be dynamically modified can bevery limiting in such situations.
Figure 9: Builder AppletSupport for groups and roles is an important part ofworkflow execution. Workflow systems are used toimprove process throughput; one of the ways to achievethat objective is to assign an Activity to a role at buildtime. A role represents a group of ‘fungible’ Performers(i.e. ‘designer’, ‘tester’, ‘manager’, etc.); the binding ofan Activity to a specific Performer is postponed untilexecution. In RainMan, we are experimenting with theuse of special-purpose Performers to manage roles;section 5.2.3 covers this topic in some more detail.The choice of a good workflow specification languageremains a matter of considerable debate within theworkflow community. RainMan uses a vanilla workflowspecification language based on directed, acyclicgraphs. It is important to realize that RainMan offers anexcellent infrastructure for deploying a wide range ofspecification tools based on heterogeneous languages,since the details (and potential idiosyncrasies) of aspecification language are completely encapsulatedwithin a Source implementation and not exposed eitherto the Performers or to the RainMan infrastructure. Theclean separation of responsibilities of workflow routingand Task execution between Sources and Performersrespectively makes RainMan a suitable infrastructurefor plug-and-play operation with heterogeneous Sourcesand Performers.Finally, the Builder also helps users monitor the state ofa workflow execution. It provides visual cues to theowner or administrator about the global state of aworkflow in execution at a coarse level of granularity.Even though the three functions of Builder, Source, andMonitor are currently physically part of the same Javaapplet, they remain distinct logical entities in ourworkflow architecture. The next version of ourprototype will separate some of these functions,allowing for features such as disconnected and remotebuilding and monitoring.5.1.2 RainMan Worklist Client AppletThe RainMan Worklist Client is a Java applet thatallows users to access their Worklists from within aWeb browser (see Figure 10). A Worklist is a remoteJava object on the network that implements thePerformerAgent and Worklist interfaces and acts as thePerformer for a human participant. The participant canuse the Worklist Client to view the contents of theremote Worklist, and selectively download specificTask requests and perform them.
Appropriate Task Handlers are automatically launchedto allow the human to interact with Task requests. Eachparticipant or Performer has a set of capabilities inRainMan. For example, for humans capable ofdocument processing, the Worklist Client currentlyhandles incoming Document Processing Task requestsvia a specialized Task Handler that can locate areferenced document from a network data store andlaunch the necessary local application (word processor)needed by the human to interact with the document. Oncompletion, the Task Handler returns the response to itsWorklist, which in turn returns it to the requestingSource.Since each Task request received from the Worklist ishandled locally on the client machine, disconnectedoperation is handled easily. The client applet needs toconnect to its remote Worklist only for the purposes ofreceiving and returning Activities. However, aconnection to the network may still be needed if theTask needs to reach certain data elements on thenetwork in the course of its execution, or if theapplications needed to perform it are not locallyavailable.The RainMan Worklist Client Applet is significantbecause it offers a valuable proof-of-concept thatdemonstrates how human performers can receive workfrom heterogeneous workflow sources using a single,unified user interface. Current workflow systemsusually come with proprietary client applications thatprovide worklist access by pulling work from theirproprietary servers. In contrast, RainMan requires thatthe Worklist Client pull work only from its designatedWorklist on the network, to which Task requests arepushed from various workflow backends. The RainManapproach imposes a much lower burden on the WorklistClient since it no longer has to handle multipleconnections, one with each workflow server it isreceiving work from. In an interoperable workflowworld, an equivalent of the RainMan Worklist ClientApplet could replace proprietary worklist clients.Figure 10: Worklist Client Applet
Task priorities and deadlines, while not yetimplemented in RainMan, can be supported. Suchconstraints, based on convention and agreementsbetween Sources and Performers, can be specified aspart of the Task request message sent to a Performer. APerformer unable to meet these constraints may refuseto service the request. Similarly, a Source may use thedeadline information associated with an Activity to timeout on a Performer that does not respond by thedeadline. However, attempts to generalize the behaviorof arbitrary Sources and Performers with respect topriorities and deadlines leads to the broader question ofnegotiation protocols between Sources and Performers.Such protocols, while immensely useful, have not beenstudied in the context of the RainMaker framework.5.2 RainMan Runtime EnvironmentThe RainMan runtime environment provides acollection of distributed services that run on the Internetinfrastructure. The user components described in theprevious section allow workflow users - workflowdesigners, participants, and administrators - to interactwith these distributed services on the network. Theservices currently available are the Directory Service,the Worklist Service, and a variety of Performers.collection of smaller domains, etc.) can behierarchically arranged so that Sources can findPerformers across domains. Performers within a domainwould always be registered with their local RainMandirectory. Sources would always lookup their localRainMan directories for Performers; the localdirectories could in turn access other RainMandirectories by traversing the directory hierarchy tolocate matching Performers. A related example is thatof the Corba Trading Service [OMG97]; multiple CorbaTraders can be explicitly linked into a network fortransparent navigation.5.2.2 RainMan Worklist ServiceIn workflow systems, worklists are inboxes associatedwith humans. In RainMan, a Worklist is a Java objectthat implements the RainMaker Worklist andPerformerAgent interfaces. Therefore, a RainManWorklist is a Performer that is owned by and representsa human on the network (this is not a limitingassumption - Worklists can easily represent applicationsas well). Worklists offer persistent storage of Taskrequests posted to humans from Sources.Source5.2.1 RainMan Directory ServiceSourceWANThe RainMan directory service plays the key role of atrading service in the RainMan system. It containsinformation about Performers on the network and theircapabilities. When a Source needs to issue a Taskrequest to a Performer, it first performs a lookupoperation on the directory service to locate theappropriate Performer on the network. The directoryservice interface also provides methods to register andunregister Performers. Using the Administrator applet, aRainMan administrator can manage the directoryservice and its contents. Mobile Performers use thedirectory service to locate their Worklists as well, ipation. For expediency, the RainMan directoryservice is currently prototyped as a custom Javaapplication. We are planning an LDAP-based[Yeong95] directory service for the next version of theRainMan prototype.For widespread workflow deployment on the Internet, itis necessary for the RainMan directory service to bedistributed. The design of distributed directory servicessuch as DNS [Mock87] and X.500 [CCITT] can beused as a guideline for RainMan directory design.RainMan directories in individual domains (a domaincould be an organization, a geographical location, gure 11: Reusable WorklistsIn RainMan, Worklists are treated as addressablenetwork objects, and provide a level of separationbetween Sources and actual human performers, whichmakes asynchronous exchange of requests andresponses between them possible. In other words,requests can be posted to a Worklist on the networkeven when the human performer is not connected, andthe human can access and perform them withoutconnecting to the Source. Most importantly, in contrastto traditional workflow systems, a RainMan Worklist isa first class entity that can be reused by multipleSources, thus eliminating the need for explicit,dedicated connections to workflow servers on the partof the performer (see Figure 11). This is in sharpcontrast to the architecture shown in Figure 4.
Worklists on the network are managed by anindependent RainMan Worklist Service. At the presenttime, the Worklist Service consists of a single Javaapplication on the network, called a Worklist Server,that manages a large pool of Worklists. The WorklistServer is analogous to a POP [Myers96] or an IMAP[Cris93] server that stores e-mail boxes for multipleusers. The clear separation of the Worklist Service fromthe Workflow Server is a novel design point inRainMan. This is useful because the Worklist Service isnow streamlined to perform a dedicated function in anautonomous manner, and a Worklist Server can run ondedicated powerful computational resources thatguarantee performance, availability, security, andreliability. Furthermore, since Worklists are practicallyindependent of each other, we expect it to be relativelyeasy in this architecture to address scalability in termsof distributed workflow participants by implementingthe Worklist Service as a distributed service; newWorklist Servers can be added to the network to hostWorklists as the number of participants increases. Weplan to experiment with these issues in the near future.5.2.3 RainMan PerformersThe Performer abstraction is useful in modelinghumans, software applications, groups and roles,workflow servers, and entire organizations that performTasks on behalf of a workflow (see Figure 12). As wehave seen, Worklists act as Performers for humans. Inaddition, we are building a host of other Performers thatcan be used by RainMan workflows. An interestingPerformer class is the SMTPGatewayPerformerthat allows RainMan workflows to send out e-mail overthe Internet. This Performer is an SMTP client writtenin Java that imple
RainMan is based on RainMaker, our generic workflow framework that defines a core set of abstract interfaces for workflow components. The main purpose of RainMaker is to facilitate the design and implementation of flexible, interoperable, and scalable workflow system components. E
