Transcription
Integrating SharePoint withDocumentumRami A Al Ghanim and Sara A Al MansourEMC Proven Professional Knowledge Sharing 2010Rami A Al GhanimSystems AnalystSaudi Aramcorami.ghanim.1@aramco.comSara A Al MansourSystems AnalystSaudi Aramcosara.mansour@aramco.com
Table of ContentsIntroduction . 3Company Overview . 4History of Enterprise Content Management in Saudi Aramco IT, Dhahran . 5Our Journey with the Integration . 8How it started . 8EMC Documentum Repository Services for Microsoft SharePoint (EDRSMS) . 10Challenges . 14Future Goals and Integration Road Map . 15Disclaimer: The views, processes, or methodologies published in this compilation are those of the authors. They do notnecessarily reflect EMC Corporation’s views, processes, or methodologies.2010 EMC Proven Professional Knowledge Sharing2
IntroductionWhen users browse any Enterprise Content Management (ECM) system, they have certainexpectations of where, what, when, and how to find the right information in the right time. Usersexpect that a number of employees can work together to manage, version, edit, and storedocuments. Information access is controlled, based on user roles. Users expect to be able toeasily store and retrieve their information. Users do not expect duplication or redundancy oftheir documents or efforts. The feeling is, if the information exists and the users have access toit, they, as users of the ECM system, have the right to simply find it. Users anticipate improvedcommunication and collaboration. And last but not least, users expect all of this to be donethrough a user-friendly interface.Saudi Aramco employees have the same expectations, and would like to use one system thatwould make all their document-related issues go away. We, as service providers, could give thisto the Saudi Aramco users but we have one simple dilemma; we have two systems for us andthe user to choose from, with pros and cons to each. A way to combine the two systems wouldbe wonderful.This article starts with a brief overview of Saudi Aramco and the history of ECM in SaudiAramco. Next, we will walk you through the journey of integration; why and how it started andhow we used EMC Documentum Repository Services for Microsoft SharePoint (EDRSMS) toresolve this situation. Finally, we will discuss our challenges, future goals, and the integrationroad map.2010 EMC Proven Professional Knowledge Sharing3
Company OverviewOwned by the Saudi Arabian Government, Saudi Aramco — with over 54,000 employees — is afully-integrated, global petroleum enterprise and a world leader in exploration and producing,refining, distribution, shipping, and marketing. The Company manages the largest provenreserves of conventional crude oil, 260 billion barrels, and manages the fourth-largest gasreserves in the world, 263 trillion cubic feet. Saudi Aramco, through its affiliate, Vela MarineInternational Ltd, owns and operates the world's second largest tanker fleet to transport itscrude oil production, which amounted to 1.7 million barrels per day in 2008. In addition to itsheadquarters in Dhahran, Saudi Arabia, Saudi Aramco, through its affiliates, has joint venturesand subsidiary offices in China, Japan, India, the Netherlands, the Republic of Korea, Malaysia,Singapore, the United Arab Emirates, the United Kingdom, and the United States. SaudiAramco also refines and distributes oil products throughout the Kingdom of Saudi Arabia tomeet domestic daily energy demands. For more information, visit www.saudiaramco.com.2010 EMC Proven Professional Knowledge Sharing4
History of Enterprise Content Management in Saudi Aramco IT,DhahranTo better appreciate our current situation, and what we have been through regarding ECM, weneed to take you back to the year 2000! It was around that time when web applications werebecoming very popular in the region and Saudi Aramco was searching for a user friendly, webbased document management system. At that time, the only one that fit Saudi Aramco’srequirements was Livelink provided by Opentext. Livelink was acquired and rolled outcompanywide as a document management and sharing application. It was used for uploadingand sharing product documentation, submitting status reports, project collaboration, simplebusiness process automation, and various other purposes. It was also around that time whenDocumentum was acquired but used in a much smaller scale for specific applications.As time passed, users became used to document management solutions and started to havemore challenging requirements. Documentum started to be used more widely and became morepopular within Saudi Aramco. Still, Documentum was used for specific applications that weredesigned and used at a corporate level. Of course, Livelink was still there and very mature,stable, and reliable. Livelink was very user friendly and provided most of what our usersneeded. For a few years, we continued to provide Livelink as is and made a customizedapplication on top of Documentum.Saudi Aramco uses SAP for business process automation. Some of these processes needed acomplex, secured document management system to store and manage the documentsassociated with these business processes. So a project was started to look into the best productin the market that can provide the needed requirements. We had two document managementsolutions, Documentum and Livelink; so a technical decision had to be made. At that time,Documentum was the only document management product certified by SAP among the two; sothe decision was to integrate SAP with Documentum. After that project was completed,Documentum became the repository for all SAP documents. Of course, Livelink becamecertified shortly after.We also started getting requests for specific needs and requirements from the Livelink usersbut, since we were only providing Livelink out of the box, we couldn’t fulfill those requests.2010 EMC Proven Professional Knowledge Sharing5
However, we were able to satisfy those needs with Documentum. This introduced a newdocument management and sharing application along with a few questions; “Why do you havethe two applications?” “Why do they both provide almost the same features?” “Which oneshould I use?” All valid questions from our customers and management. The answer was to gowith the application on which we invested the greatest amount of time and effort. So we nowhad two systems; neither of them cheap in any way. We had to think right and fast. SaudiAramco conducted an application road map to decide on the future of many of its majorapplications. Documentum was selected to be the solution for Saudi Aramco documentmanagement since it was used in many corporate customized applications and was therepository for Saudi Aramco SAP.Once this decision was made, another set of questions were raised, “What to do with theLivelink Content?” and “How do we manage the heavy users of Livelink?” We faced thechallenge of moving the Livelink Content to Documentum and converting all the Livelink users toDocumentum users. We looked at many solutions and nothing met our specific needs. Wedeveloped our own migration tool and migrated all of our Livelink content to the customizedDocumentum application, called e-Cabinet. A major awareness and change managementcampaign was conducted. The result was that in 2006, Documentum became Saudi Aramco’sdocument management system.At around the same time, SharePoint 2007 Beta Release was distributed by Microsoft. We got arequest to establish a Knowledge Sharing System on top of SharePoint. Of course we all sawthe amazing features and user friendliness of SharePoint, but from a system perspective, wedidn’t want to add another application. Our users had a number of issues that a knowledgesharing system would resolve. One of the issues that Saudi Aramco faced was a hugeretirement rush. All the extremely knowledgeable employees who have been working for thecompany were scheduled to leave within the next five years. Their knowledge needed to becaptured and saved for current and future employees. Another issue was the duplication ofefforts when it came to analyzing and solving similar problems. As an example, when anengineer working in Plant A notices a major problem with one of the machines in the plant, he orshe needs to look into the problem and analyze what could be causing it. This engineer foundthe solution and proceeded to fix the problem. Solving this problem could take hours, days, orweeks, depending on the situation. Now, if another engineer in Plant B faces the same problem.what does he or she do? They go through the same process of analyzing and solving the issue.2010 EMC Proven Professional Knowledge Sharing6
This could be done more efficiently if the engineer had a place to go where he or she couldsearch for similar types of issues or even connect with the engineer who faced this similarproblem. We also had an issue with the huge number of information or organizational web sitesthroughout the company and the need to consolidate those in one area. It was very clear thatwe could resolve those problems using SharePoint collaboration and Content Managementfeatures. So we spent some time looking into the Beta and eventually went ahead withSharePoint 2007.During that time, Livelink was fully migrated to e-Cabinet, the customized application on top ofDocumentum. So, after all of that we are back to square one. We have two very similarapplications that provide very similar features. “Where do we put our stuff?” “Where can I findthis?” “What’s the difference between Knowledge and Content?” All were valid questions thatstarted to be repeated. The answers were a bit more difficult this time around. Why did weprovide the two systems? Looking into the situation and analyzing it further we discovered thatwe did need the two systems. We discovered that each had its own areas that were extremelyefficient. Having the two ECM systems made it very difficult for us and our users. So we knewwhat we had to do, but had no idea how to do it. The two systems–Documentum andSharePoint–had to remain but only one would be apparent to users. Since we had invested a lotin Documentum’s Enterprise Content Management features and modules, and after studying thesituation and looking into the market trends, we decided to keep SharePoint 2007 as the userfriendly interface to Documentum; it would be our Knowledge and Content ManagementSystem. That starts our story with integrating SharePoint with Documentum.2010 EMC Proven Professional Knowledge Sharing7
Our Journey with the IntegrationHow it startedNow that we knew exactly what we wanted to do, we thought the rest would be easy WRONG!!! Due to the growing popularity of SharePoint, everyone provided some sort ofintegration. We started looking at all of them!! And I do mean ALL of them. We had specificrequirements that needed to be met for us to take advantage of both of our systems; otherwise,what was the point? We wanted to have the user continue working on SharePoint as is, with thesame look and feel of SharePoint. At the same time, we wanted all the documents to reside inDocumentum. Due to Microsoft’s recommended 100 GB limitation of the SQL database, weknew we would face an issue there. Additionally, because we had invested in other EMCDocumentum ECM features such as Records Management, we wanted to be able to continueusing them after the integration took place.None of the products we looked at fit our specific requirements. Most of them focused on theidea of web-parts, which would provide a window view to Documentum from within SharePoint.Even though that was very easy to implement and provided a very similar document view andupdate interface to the one provided by SharePoint, that wasn’t acceptable to us or our users.One of the products provided by EMC – Archive Services for SharePoint – seemed to have thepotential, so we decided to invest more time and effort on that product. We felt that ArchiveServices would mature into a product that would fit our needs based on our testing and its roadmap. Two of Saudi Aramco’s employees (who happen to be the two writing this article) weresent on a year and a half business assignment in EMC. We were sent to gain knowledge aboutEMC’s SharePoint integration products and actually helped build one of them. We weren’t spiesbut we did learn a lot. We had the chance to work with the designers of Archive Services as itmatured and – of course – changed its name quite a few times.Regardless of the name changes, we felt that the product we were following had reached apoint where it provided exactly what we were looking for; EMC called it EMC DocumentumRepository Services for Microsoft SharePoint (EDRSMS). To confirm our belief, a project teamwas formed consisting of one project manager, two Documentum Administrators, two2010 EMC Proven Professional Knowledge Sharing8
SharePoint Administrators, and two Documentum Analysts. This project team was tasked withevaluating the controlled release of the EDRSMS through the Controlled Release Program(CRP). The CRP was very useful to Saudi Aramco, as it gave us early access to the productand allowed us to collaborate with other companies and the product design team during theentire process of planning, installing, and testing EDRSMS. After the completion of the CRP, itwas clear to us that it did provide exactly what we were looking for. Then the real fun started!!2010 EMC Proven Professional Knowledge Sharing9
EMC Documentum Repository Services for Microsoft SharePoint(EDRSMS)After evaluating the product, a quote provided by Saudi Aramco for use in the product rolloutbest expresses our feeling about the product and why we decided to go with it. "EDRSMScreates a true partnership, like Yin & Yang, SharePoint having the friendly face for everyone tosee and Documentum providing the muscle for everyone to use. EDRSMS combines best-ofbreed solutions allowing users to access their documents from the more user friendly interface(SharePoint) while storing and managing them in a robust content management platform(Documentum)."EDRSMS journals SharePoint content into Documentum and provides seamless integrationbetween SharePoint and Documentum. Before going into how EDRSMS works, let’s go over thepoints that led to choosing EDRSMS. First of all, the integration was transparent to the users. Allthey had to do was log in to SharePoint to upload, update, or view their documents. Although itappeared as if they were in SharePoint, they were all actually in Documentum. Second, itrelieved SQL Server bloat by having the content now stored in Documentum instead of the SQLServer; this led to easier manageability of the storage. Third, it improved productivity since theusers would not need to access the two systems. Finally, it would allow us to capitalize on ourinvestment in the ECM features of Documentum and its add-on modules such as RecordsManagement (as promised in the future release of EDRSMS.)Before installing EDRSMS, both existing SharePoint and Documentum environments needed tobe studied to properly design how the integration would fit within both environments. Taking acloser look at our Documentum environment (Figure 1) illustrates how our web clients connectto Documentum through the J2EE or IIS Servers which go through the connection brokers toour Documentum Content Servers. We have five content servers hosting a number ofRepositories. The Content Server we are targeting for the integration is the one we call PDW. Ithosts the e-Cabinet Repository.2010 EMC Proven Professional Knowledge Sharing10
On the other side of it, we have our Microsoft SharePoint environment. Figure 2 shows how ourweb clients would connect through the web interface to one of our Web Front Ends. In this case,we have three Web Front End Servers; one of them hosts the Central Administration. All thecontent and metadata is stored in the SQL Server databases.2010 EMC Proven Professional Knowledge Sharing11
To integrate SharePoint with Documentum, we needed to install and configure the neededcomponents of the product where they fit, as illustrated in Figure 3. We first needed to create anew Repository (Rep EDRSMS) that would host the SharePoint Content. Optionally, you couldalso install an EDRSMS DAR file that would take care of Advanced Mapping. This means itwould bring a set of optional SharePoint properties back to Documentum such as SharePointLocation, SharePoint Created By, SharePoint Modified By, SharePoint Deleted, etc. This wouldbe useful for reuse of the content or if you want to query Documentum for some specificinformation based on this mapping. We decided to go with Advanced Mapping. We would thenneed to install the Admin Component on the server that hosts Central Administration. Thiswould be used to configure and monitor EDRSMS through the EDRSMS Configuration page.You would then need to install the EDRSMS Component on EACH Web Front End Server. ThisComponent consists of the Journaling Service (JME) and the Cleanup Service (CCS). The CCSidentifies the items that have been deleted from SharePoint and also need to be deleted fromDocumentum. The Journaling Service does the actual content journaling and deletion in2010 EMC Proven Professional Knowledge Sharing12
Documentum. Once the Journaling Service is installed, you need to create the EDRSMSdatabase on the SQL Server (preferably) in its own instance. This database will host theconfiguration information and contain the blob ID information for the content that is stored inDocumentum. After you have done all of this, your content will now be stored in theDocumentum Repository and only the blob information will reside in the EDRSMS database.2010 EMC Proven Professional Knowledge Sharing13
ChallengesOur requirements didn’t end there. To be successful, we needed to consider existing content inSharePoint and Documentum. Unfortunately, EDRSMS does not consider existing SharePointcontent, and would only integrate new content. After collaboration with the EDRSMS designersand project team, we decided to take two approaches to this.1- Forward Approach: We would do nothing. Once the user updated their content inSharePoint, it would go though the journaling process. We sometimes called this the“Lazy Approach.”2- Backward Approach: We built a program that would force the content through thejournaling process. We would use this approach for SharePoint sites with large-sizedcontent that should move ASAP.We also needed to consider the Documentum Content that already existed in e-Cabinet. If wetruly wanted to benefit from the integration and give the user one point of access, we needed tomigrate that content to SharePoint. Due to the integration, that content would go back toDocumentum. Technically, this change could be done in one step but we have to considerchange management and awareness of the e-Cabinet users. A phased approach was adoptedto minimize the impact of the change on the users.We also had some backup/restore requirements that EDRSMS didn’t address. For example, weneeded to be able to copy or move our SharePoint environment to a new one for upgrades or incase of disaster. Although possible, it had to be performed without the out-of-the-box solutio
At around the same time, SharePoint 2007 Beta Release was distributed by Microsoft. We got a request to establish a Knowledge Sharing System on top of SharePoint. Of course we all saw the amazing features and user friendliness of SharePoint, but from a syste
