Architectural Design And Best Practices Project Final .

Transcription

Architectural Design and Best Practices ProjectFinal Report and Design Recommendations (A006.1)Prepared for the Virginia Department of EducationFebruary 28, 2011Technical Point of Contact:Louis McDonald CIO/CTOlouis.mcdonald@cit.org 703.689.3037Administrative Point of Contact:Pat Inman Contract Managerpat.inman@cit.org 703.689.3037

A006.1 Delivverable – Final Report and Desiggn RecommendaationsConteentsExecutivve Summary . 4 1 Intrroduction . 6 1.1 Study Goal . 6 1.2 Project Deliiverables . 6 2 Research Processs . 6 3 Keyy Messages . 8 4 5 3.1 Stakeholderr Managemeent . 8 3.2 Federated Systems Perfform Poorly . 9 3.3 Data Governnance . 9 3.4 Leveraging Existing Sysstems . 10 3.5 Use of Commmercial Systtems . 10 3.6 Multiple Haash Keys. 11 3.7 Requiremennts and System Architeccture . 11 3.8 Clearly Defiined Securitty Policies . 12 Archhitecture Beest Practice Case Studiess . 13 4.1 Indiana Deppartment of Education . 13 4.2 Iowa Deparrtment of Edducation .18 4.3 Army Suicidde Mitigatioon Project. 20 4.4 Texas Educcation Agenccy . 23 4.5 DLA Data Convergencee and Qualityy Project . 26 4.6 NORC at thhe Universitty of Chicagoo Data Enclaave . 29 Subjbject Matter Expert Interrviews . 34 5.1 Dr. Bhavanii Thuraisinghham . 35 5.2 Paul Carneyy. 36 5.3 James Camppbell . 37 5.4 Susan Carteer . 38 5.5 Raj Rameshh .40 5.6 Ron Kleinmman . 41 5.7 Peter Dobleer .42 5.8 Dr. Laura Haas . 43 5.9 Dr. Thilini Ariyachandrra . 44 5.10 Dr. Cynthiaa Dwork . 45 6 SLDDS Architecture Overvieww . 47 Archhitectural Designn and Best Practiices Project P a g e 2

A006.1 Delivverable – Final Report and Desiggn Recommendaations6.1 SLDS Sevenn Functionall Componentts . 47 6.1.11 Portal . 47 7 6.1.22 Securitty .49 6.1.33 Workflflow . 50 6.1.44 Reportting . 53 6.1.55 Lexiconn . 56 6.1.66 Shakerr . 57 6.1.77 Data . 58 Phyysical Infrastructure . 60 7.1 Developmennt Environmment . 60 7.2 Test Enviroonment. 61 7.3 Production Environmennt. 62 Appendixx A: Secondary Architeccture Best Prractice Case Studies . 63 A.1 Illiinois State Board of Educcation . 63 A.2 Noorth Dakota Departmentt of Public Innstruction. 65 A.3 Washington Education Reesearch and Data Centerr. 68 Appendixx B: Best Praactice Case Studies Interrviewee Listt . 70 B.1 Inddiana Departtment of Eduucation . 70 B.2 Iowwa Departmment of Educaation . 70 B.3 Daata Strategiess – Army Suiicide Mitigaation Projectt . 70 B.4 Teexas Education Agency . 71 B.5 Daata Strategiess – DLA Datta Convergennce and Quaality Project . 71 B.6 NOORC Data Ennclave . 71 B.7 Illiinois State Board of Educcation . 72 B.8 Noorth Dakota Departmentt of Public Innstruction . 72 B.9 Staate of Washiington Educcation Reseaarch & Data Center . 73 Appendixx C: Materiaals Sent to Best Practicess Intervieweees . 74 C.1 Besst Practices Interview Template . 74 C.2 Arrchitectural Best Practice, Design & Planning Suupport Projeect Overvieww . 76 Appendixx D: Materiaals Sent to Suubject Matter Experts . 77 D.1 Subbject Matterr Expert – Innterview Temmplate . 77 D.2 Virginia Statewwide Longittudinal Data System - Exxecutive Summmary . 79 D.3 Virrginia Statewwide Longituudinal Data System - Ussage . 82 Archhitectural Designn and Best Practiices Project P a g e 3

A006.1 Delivverable – Final Report and Desiggn RecommendaationsExecuutive SummaryThe educcational landdscape has chhanged drammatically sinnce the establishment of StatewideLongituddinal Data Syystems (SLDDS) throughoout the Unitted States. A grant progrram1 funded bythe U.S. Departmentt of Educatioon, as authorrized by the Educationall Technical Assistance Act of2002, hass helped to change manyy states’ K-122 data systemms significanntly and mayy, in fact,revolutioonize the future managemment and utility of educational dataa. States thatt haveimplemennted these data systemss now have morem accuratte and robust data and ann enhancedability too access, anallyze and utillize the dataa in a mannerr previously unavailablee in the past.These successes, howwever, have been achieveed only throuugh rigorouss planning, meticulous design,and assidduous implemmentations.d a federal multi-year grrant toIn 2010, the Virginia Departmentt of Educatioon (VDOE) was awardedenhance its statewidde data systemm, and launcched the Virrginia Longittudinal Dataa System (VLLDS)project. The project team was chharged with creating a syystem that will address the needs off allders; additioonally, the teeam faced chhallenging Feederal and Virginia security and privvacystakeholdrequiremments. To meeet these chaallenges, VDOOE commisssioned the Center for InnnovativeTechnoloogy (CIT) too identify keyy success facctors that coould provide guidance inn the developpmentof a fully secure and private, as well as more efficient, SLLDS in Virginnia.In order to achieve thhis complex objective, thhe CIT projeect team inveestigated othher large dattaintegration projects in state educcation agenccies, other goovernmentall agencies annd in otherindustriees. The team examined published infformation from a varietyy of sources, and a gapanalysis was performmed on these reports andd articles to identify misssing informaation and criiticalareas whhere further research musst be conduccted. In addiition, the teaam interviewwed nine SLDDSleaders and a numberr of industryy leaders whho managed large integraation projectts. Theseinterviewws provided the foundatiion for the compilationcof best practtices and keyy takeawayss thatwere esseential to thee success of each of thesee projects.Informattion collectedd during thee research annd analysis phase was annalyzed to iddentify commmonthemes and these theemes were orrganized intto a set of beest practices that were inncluded in a finalreport. This process revealed sevveral key elemments2 that were cruciall to ensuringg successfulimplemenntations. Thhe primary thhemes that emergedeinclluded, amonng others, thee necessity foordetailed project plannning and maanagement (the(importaance of data governance and stakehooldermanagemment) as welll as the needd to conductt comprehennsive researchh and planniing beforeimplemennting the tecchnology and creating thhe system arrchitecture (the use of coommercialsolutionss and leveragging existingg systems).12 The Stateewide Longituddinal Data Systtems (SLDS) Grrant Program The final report includeed challenges aand obstacles tto be avoided and provided rrecommendatiions for a nummber of ms. In addition,, the report coontained a suppplement that inncluded the coomplete reportts with preliminary action itemdetailed findings for eaach interview aand data project researched. Archhitectural Designn and Best Practiices Project P a g e 4

A006.1 Delivverable – Final Report and Desiggn RecommendaationsThe CIT project teamm, along withh VLDS subjject matter experts, tookk the originaal conceptuaalSLDS arcchitecture annd incorporaated results fromfthe besst practices research andd subject matterexpert innterviews to develop an implementattion architeccture. The refined architecture (Secction6) expannded the detaails for functtional compoonents, secuurity, reportinng, and howw data requesstswould bee managed given knownn constraintss. The team developed ann understandding of the fllow ofinformation, and the necessary workflows too support the different scenarios to support the SLDSdeployment.All this innformation allowed the team to devvelop a physiical infrastruucture archittecture (Secttion7). This included thee physical haardware, the location, annd the functionality for thhe hardwaree.Followinng standard lifecycle devvelopment prractices, threee versions of the infrasttructure aredescribedd; Developmment, Test, annd Productioon.Archhitectural Designn and Best Practiices Project P a g e 5

A006.1 Delivverable – Final Report and Desiggn Recommendaations1 Intrroductioon1.1 Sttudy GoallThe goal of the Archiitectural Dessign and Besst Practices Project was to provide thhe VirginiaDepartmment of Educaation (VDOEE) with an up-to-date annd relevant assessment of the bestpracticess related to the design, development, deploymennt, and operaation of a StaatewideLongituddinal Data Syystem (SLDSS).1.2 Prroject criptionMonthly Status RepoortsList of alll SMES interrested in parrticipating innintervieww programIntervieww Schedule aand Intervieww TemplatePreparatiion and pre reading matterial as requuiredfor Intervview SMEsConsoliddated outputt from intervviewsTemplatee for Final DeliverableMonthly Status RepoortsPMO Suppport, to incclude programmntation i.e., WWork plan, SScope,documenRequiremments, Scheddule, Risk, annd ChangeManagemment plans as requested by the ProgrramOfficeArchitecttural Best Prractices ReportWorkshoop Agenda and Presentation materiaalSummaryy Workshopp Findings RReportport & Desiggn RecommeendationsFinal RepDelivery DatetMonthlyOctober 20110OOOctober20110October 20110ODDecember20010November 20N010MonthlyOngoingDDecember20010December 20D010JJanuary2011February 20111F2 Research ProcesssThe CIT project teamm applied itss establishedd CIT Conneect research and analysiss process toexecute this effort. This rigorouss “best practiice” methoddology includdes the identtification of andanalysis of informatioon providedd by Subject Matter Expeerts and by comparable SLDS projeccts orlarge data integrationns from the public and private sectoor provides high confidennce results.Additionnally, this meethod also inncludes analyysis and connsolidation of feedback received fromm theSLDS staakeholders.Figure 1: CIT Connect ProcessProject and Requirements DDefinitionTechnoloogy Identificatioon and ResearcchTechnology Assessment and Anterview ProcessInRecommenndation Developmment and ConsoliddationFeedback Integration and Report FinalizationCIT Connnect projectts are performmed under the control of a well-defiined project managemenntapproachh. This approoach providees visibility into project status at all times via reegular reviewws,status repports, and innterim deliveerables. The CIT Connecct Process, shown in Figgure 1, providdes adiverse, five-step appproach to souurcing innovvation, rigorous analysiss of alternativves, and aArchhitectural Designn and Best Practiices Project P a g e 6

A006.1 Delivverable – Final Report and Desiggn Recommendaationsstructureed engineerinng methodology for creaating final reecommendattions. The usse of thisstructureed multi-stepp approach maximizes thet likelihoood of successs, while reduucing risk. Prrojectexecutionn is guided by a project plan developped and mainntained by the CIT projeect managemmentteam.For the VDOE Archittectural Dessign and Bestt Practices Project, the process begaan with ananalysis of the informmation and requirementss provided by VDOE andd the identiffication of thheinformation and techhnology areaas to be targeeted for studdy. The seconnd process sttep focused oninitial ressource sourccing by developing a list of candidatee SLDS and large data inntegrationperts in the innformation and technoloogy areas defined for theeprojects and Subject Matter Expproject. This second process prodduced severaal deliverablles focused on Subject Matter Experrtsinterviewws and best practices casse studies thhat were pressented to the Virginia Department ofEducatioon between October 20100 and Decemmber 2010.The Archhitectural Beest Practices Report (A0004) focusedd on initial daata sourcingg, both to devvelopa list of candidate SLLDS and large data integrration projeccts to be anaalyzed and too collectpublished analyses, reports, and case studiess on existingg projects. Thhe set of canndidate projeects,which was documennted in Deliveerable A002-2, was commpiled by acccessing CIT’ss businessnetwork and by searching our daata resourcess to identify candidate companies annd organizattions.This list was delivereed to the Virrginia Department of Eduucation on October 31, 2010.The Consolidated Ouutput from Subject Mattter Expert Innterviews (AA002.4) focuused on initiaalresource sourcing byy developing a list of canddidate Subjeect Matter Experts in thhe informatioon andtechnology areas defiined for the project. CITT researcherss reached ouut to CIT’s buusiness netwworkin additioon to contaccting these iddentified expperts, who were discoveered throughh literaturesearches. The set of candidate Suubject Matteer Experts, which was doocumented in DeliverablleA002.1, was submitteed to the Department off Education on October 31, 2010.For bothh reports, CITT reviewed the sourced materialsmannd developedd a gap analyysis of informmationrequiremments. The gaap analysis was the basiss for the creaation of survvey tools andd an analysisframewoork which thhen were used to guide thhe intervieww process andd the assessmment steps thatfollowedd. The gap annalysis also guided the seelection of leeaders of proojects targeteed for directinterviewws.3 The infoormation prioorities that emergedefromm the gap annalysis were, therefore, usedto classiffy and to priooritize intervview candiddates via a prrocess basedd upon criterria that incluudeddomain relevance to VDOE, cost, complexityy, technical and businesss maturity, and stakehollderconsideraations. The survey questtions and tarrgeted list off candidates for best praactice interviiewswere proovided to thee Departmennt of Educatiion as Deliveerable A002--2 on Octobeer 31, 2010. Thetargeted list of Subjeect Matter Exxperts (Deliverable A0002.1), intervieew schedulee, questiontemplatee (Deliverablle A002.2), and pre-readding materiall (Deliverablle A002.3) were providedd tothe Depaartment of Edducation on October 31, 2010.The subssequent phasses of the proocess focuseed on consullting leaders of similar sttate andcommerccial efforts, synthesizingg and analyziing and orgaanizing theirr feedback annd presentinngthese besst practices to stakeholdders. Step thrree, the longgest phase off the project,, involved3 All of thee projects had potential relevvance to the efffort; however, it would havee been both inffeasible and duplicative to intervieww the leaders oof all of the canndidate projeccts, given this pproject’s short timeline. Archhitectural Designn and Best Practiices Project P a g e 7

A006.1 Delivverable – Final Report and Desiggn Recommendaationsperforming interviewws and synthhesizing andd analyzing thhe informatiion from botth the SubjecctMatter Experts and Best Practicees candidatees. During thhis phase, CIIT researcherrs conductedd nineinterviewws with leadders of large-scale projectts representiing both thee public and private sectors,4and ten Subject Mattter Experts.In step foour, the CIT Connect teaam organizeed and categoorized key leessons learneed from eachh ofthe intervviews and highlighted common themmes and uniqque insightss. The CIT Connect teammpresentedd the best prractice and Subject Matter Expert innterview finndings to VDDOE stakeholderson Decemmber 13, 20100 and integraated feedbacck and guidaance from thhe stakeholdeers to generaatethe reporrts A004 andd A002-4.Step five involved thee presentatioon of the CITT project teaam’s implemmentation arcchitecture anndassociateed physical innfrastructurre to the SLDDS stakeholdders on January 27, 2011. Presentatioonswere madde by the CIIT project team and VLDDS subject matter expertts. The purpose of thisworkshoop was to forrmalize an aggreed upon architecture for the SLDDS.3 Keyy MessaagesThe threee componennts of the VDDOE Architecctural Designn and Best Practices prooject covereddthree diffferent themees and focus. The Best Prractices inteerviews and analysis centered on theeimplemenntation and logistical prrocesses invoolved in largge scale data integration projects. ThheSubject Matter Expeerts interviewws focused oon technical best practicces for an LDDS architectuuraldesign.The folloowing topicss are the key messages annd best practices borne out of the Beest Practicess andSubject Matter Interrviews.Best Practtices InterviiewsStakehollder ManageementData GovvernanceUse of Coommercial SolutionsSLeveraginng Existing SystemsRequiremments Drive System ArchhitectureSubject Maatter Experts InterviewwFederaated Systems Perform PooorlyData GovernanceGUse off Commerciaal SolutionsUse off Multiple HashHKeysClearly Defined Seecurity Policcies3.1 Sttakeholdeer ManaggementWhen emmbarking uppon a systemms integrationn project, nuumerous stakkeholders pllay a part in theplanningg, developmeent, implemeentation, andd maintenannce of the sysstem. Knowiing stakeholders’requiremments, expecttations, and resources arre essential to a project’ss success.During thhe Army Suiicide Mitigattion Project,, Data Strateegies discoveered that maanaging thestakeholdders becamee an overwheelming task when it camme to obtaining memoranndums ofunderstaanding (MOUU) and data sharing agreeements neeeded prior too the integrattion of a datanto the system. Data Straategies also found that clear communnication bettween the prrojectsource inimplemenntation teamm and the staakeholders as well as commmunicatioon among thee stakeholdeerswas best facilitated by the projecct managers. The DLA Data Converggence and QuQuality Projecct4 A list of tthe organizatioons and compaanies with whoom we spoke mmay be found inn Appendix B. Archhitectural Designn and Best Practiices Project P a g e 8

A006.1 Delivverable – Final Report and Desiggn Recommendaationsmanagerss needed to ensure accurrate and timmely communnication of project statuss, feedback, andnext stepps; this creatted a foundation for a poositive collabborative enviironment. Thhis positivecollaboraative environnment amonngst the teamms contributeed to the oveerall successs of the projeect.The Indiaana Departmment of Educcation attemmpted to gathher stakeholdder requiremments througghlarge, moonthly meetings before discovering tthat meetinggs with indivvidual stakehholder grouppsproved too be the morre effective and led to inccreased buy--in.Lastly, ass Illinois is currently in the design sttage of its SLLDS, the Illinnois State Booard of Educcationhas hiredd a consultinng firm to perrform some stakeholderr managemennt. The consuulting groupp is inthe midst of gatherinng the techniical and proggram informmation for eacch of the 13 data systemss thatwill be inntegrated intto the Illinoiis SLDS and this informaation will haave a direct impact on thheSLDS’ finnal architectuure.(Army Suiciide Mitigationn, DLA, Illinoiss State Board of Education, Indiana Department of Education)3.2 Feederated Systemss Performm PoorlyFederated systems suuffer in perfoormance more than a cenntralize dataabase. Requiirements anddqueries should be plaanned prior to building the system to maximize performancce. Usually with adistributted databasee model, dataa converges into a warehhouse for simmpler analysiis. A globalschema is defined thaat allows forr an easier coonvergence of the data. The federateed model for theSLDS cannnot allow foor permanennt convergennce, nor is it likely that a global scheema will bedevelopeed that encommpasses all data sourcess. The disparrate data andd network immpacts on thhedistributtion of the daata can impaact the overaall performannce of the fedderated archhitecture.(Ariyachaandra, Doblerr, Haas)3.3 Daata GoverrnanceThe impoortance of daata governannce was a common message throughhout the courrse of this efffort.Data govvernance ofteen is viewed as a large innitial effort foor many data integrationn projects. Forsystems that continuue to expandd and to add data sourcess, however, it will be an ongoing effoort,one that,, our intervieewees notedd, is often undderestimatedd. In most caases, an SLDDS effort musstaccommoodate a nummber of disparrate stakehoolders and soources and, thus, requirees a higher thhannormal leevel of effortt to identify data ownersship and oveersight to enssure its accuuracy andsecurity. The fact thaat each data source will have its ownn data governnance createes an additioonallayer of complexity when attemppting to creaate and manaage data governance. Prior toimplemennting a stateewide systemm, it is critical for the staakeholders to agree on who owns thhedata in thhe system, who will overrsee and maiintain the syystem and who will apprrove output andrequests for access.The Nortth Dakota Department of Public Instruction andd the Washinngton Reseaarch and DattaCenter arre two projeects that are in the early stages of building their data integraation systems.Both of thhese projectts have had difficulties mmoving forwaard with thee implementaation of thesystem due because of stakeholdders’ inabilityy to agree uppon the ruless of the LDS’’ data governnance.The Armmy Suicide Mitigation andd the DLA Data Converggence and Quality Projeccts were proojectsthat stresssed the impportance of data governaance during the early plaanning stages prior toimplemenntation and continue to emphasize these elemennts as data sources are added. With theArchhitectural Designn and Best Practiices Project P a g e 9

A006.1 Delivverable – Final Report and Desiggn Recommendaationsaddition of a new datta source, thhe project maanagers musst understand the new soource’sgovernannce and howw its integratiion will affecct the LDS’ overarching governance.Appropriiate data govvernance, paarticularly inn a federatedd model, is keey to ensurinng that the data,the linkaages of data, and performmance of the system are optimized. Once the SLDDS architectuure isimplemennted and operational, it is important to monitorr the types of queries exeecuting in thhesystem. This monitorring allows for tuning off the system to improve its performaance, and tounderstaand how the security moodel is enforccing the rulees. It is not unncommon thhat rules willlneed to be tweaked as data goverrnance contiinues to evollve.Althoughh the Virginiia SLDS’ stakkeholders haave a strong understandiing of the syystem’s basellinedata goveernance, a nuumber of facctors will inffluence the need for upfrront and onggoing efforts. TheVirginia SLDS will addd future sources and thhis will requiire both a riggorous upfroont effort tominimizee rework andd redesign annd will neceessitate ongooing efforts when changees are made to theexisting data sourcess or when neew data sourrces are addeed.(Army Suuicide Mitigattion, DLA, North Dakota SLLDS, Texas Edducation Agency, Washingtoon Research annd DataCenter)3.4 Leeveragingg Existingg SystemmsSome SLDDS projects were able too leverage exxisting systemms to becomme the founddation of the SLDSsystem; this saved timme and resouurces duringg the design and implemmentation staages of the prroject.Indiana was able to leverage an existing systtem, Learning Connectioon, and expaand on itscapabilitties. Initially, Indiana hadd not planneed to expandd its Learninng Connectioon portal, as itwas builtt primarily ffor teacher netw

The Architectural Best Practices Report (A0 a list of candidate SLDS and large data integr published analyses, reports, and case studies ber 2010. 004) focused on initial data sourcing, both to develop ration projects to be analyzed and to colle