WIXLIB

James E. Siegler et al.other admission information (the sum of which comprises nearly half the total number of CDEs in ourregistry) are collected prospectively by the stroke program coordinator onto a standardized paper version ofour CRF (see Figure 1). In the days following admission,a board-certified vascular neurologist will documentonto this CRF key imaging and management data.Key CDEs are selected for initial collection based onthe ability to use responses as a filter for future studies.If an investigator establishes an ancillary project ideabased on subpopulations of the registry, the key CDEscan aide in guiding the investigator to establish whatadditional information needs to be collected as well ashow it should be collected. This is followed by applyingfor expedited Institutional Review Board (IRB) approvalfor the ancillary study, and additional needed variablescan be collected from the electronic medical record andchart using a study-specific CRF (see further discussionin Supplementary Data Abstraction for details). Theremaining data regarding a patient’s hospitalization,complications during stay, and outcome at the time ofdischarge and at 3 months are collected retrospectivelyonto the CRF by other research team members (medicalstudent volunteers, residents, nurse practitioners, faculty) trained in data collection. The only 90-dayoutcome measure collected is the modified RankinScale (mRS) score, a seven-point scale serving as themost commonly used functional outcome measure inneurological studies.24 Because The Joint Commissionrequires collection of the 90-day mRS and follow-upphone calls for disease-specific certification, our strokeprogram coordinator obtains the 90-day mRS by astructured and validated telephone interview, exceptwhen a patient was seen in the stroke clinic within the97-day range and the mRS is documented.Reconciliation of CDEsOnce the CDEs have been gathered onto the paperversion of our CRF, potentially inaccurate data pointsare validated manually by a more experienced researchteam member. In the event that inaccurate data aresuspected, the medical record would be reviewed by amore experienced member of the research team andthe variable of interest would be corrected on the CRFwith a time/date stamp indicating when the reconciliation was made as well as the initials of the reviewingteam member. After all data on a paper CRF have beenreviewed in this manner, the CRF data are then transcribed via single-entry into a secure, passwordprotected electronic master spreadsheet Figure 1 where a second reconciliation process occurs after theMSRJ # 2013 VOL: 02. Issue: Springepub May 2013; www.msrj.orgLessons from a stroke program registrydata are electronically transferred. Prior to analysis, eachCDE used in a given research study is then sorted fromsmallest to largest (for continuous variables) or A toZ (for text variables) in order to identify any grosstranscription errors (a letter or word in the place of anumber). This process is followed by identification ofany continuous numerical data that lie beyond twostandard deviations for that particular CDE (classified as‘potentially erroneous data’). These data are validated orcorrected using source data verification (SDV) once asecond review of that patient’s medical record hasoccurred. After all data have been accurately collectedand entered into this master electronic spreadsheet, it isthen transferred to a statistical software package foranalysis where the statistical files become recognizedby the research team as the updated primary registry.Each of these phases in primary registry creation hasbeen approved by the Tulane Medical Center IRB.Supplementary data abstractionOnce the primary registry is established, a researchercan posit a study question that he/she would like toinvestigate. The study question is discussed with allinvestigators who would be involved in the data abstraction, analysis, and drafting of the manuscript, and thenformed into a testable hypothesis by methodologists.The research team is then able to anticipate all quantifiable CDEs necessary to answer this question, whichincludes data collected in the primary registry as well asdata necessitating re-review of patient medical records.The CDEs that are needed for the study question are usedto create a supplemental CRF to collect the additionaldata. The new variables of interest are strictly defined andadded to the master codebook by the project PI. Once IRBapproval has been granted for the proposed study, datacollection with the supplemental CRF begins where itgoes through the same series of SDV as described aboveto ensure data validity. Once these additional data havebeen gathered and validated in a supplemental electronic spreadsheet, they can be added to the secure masterelectronic spreadsheet. A summary of our data collectionand interpretation methods can be found in Figure 2.ADVANTAGESIn an attempt to minimize some of the errorsinherent to registry production and maintenance, thefollowing three objectives were applied to the medicaldata registry:(1) The same CDEs are collected accurately andcompletely;Medical Student Research Journal023

Lessons from a stroke program registryJames E. Siegler et al.Figure 1. View of the paper and digital versions of our case report form (CRF).A. Representative view of the paper case report form on which data are collected.B. Screen view of the digital data collection tool (Microsoft Access 2007). Shown is a representative page in the collection tool thatcorresponds to the common data elements (CDEs) collected in the case report form (part A).(2) Each CDE has a standardized definition; and(3) Data which can be queried for future investigationsare provided.Objective 1 ensures the abstraction of accurate, verifiable, complete, and relevant information. However, lesscontrollable sources of data error still exist, such aserrors in laboratory results and other medical data documentation from electronic medical records. The completeness of information is valuable for two reasons:(1) All of the important facts for a given patient duringtheir hospitalization are collected; and(2) Each of these facts is collected across all patients inthe registry, reducing bias in data abstraction.024Medical Student Research JournalObjective 2 provides the framework for reliable andsimple information. Simple but concrete definitions,standardized within the literature, are required to studyspecific associations between variables and to permitcollaboration with other investigators when combiningvariables with the same definition.Objective 3 facilitates economical and timely information abstraction. It is important to consider thetimeliness of information abstraction as this is commonly a rate-limiting step in any methodology. It maytake an experienced data abstractor up to 90 min tocomplete one CRF and an additional 30 min to validateand transcribe these data into an electronic masterspreadsheet. Not all data from a given patient can becollected in a timely manner; therefore, fundamentalMSRJ # 2013 VOL: 02. Issue: Springepub May 2013; www.msrj.org

James E. Siegler et al.Consecutive patients with confirmed diagnosed ofstroke at our center are de-identified and assigned aregistry code number for future referenceCommon Data Elements are abstracted from eachpatient's medical record and documented on a casereport form with the corresponding registry codenumberData identified as missing or erroneous(transpositional or transcriptional errors) arereconciled on the case report form before entry intoa master electronic spreadsheetData from each case report form is transcribed into amaster electronic spreadsheet that is re-named withthe most recent date as a suffix to the filenamePotentially erroneous data (numerical data that liebeyond 2 standard deviations of the mean for thatcommon data element) or gross transcription errorsare reconciled in the master databaseReconciled spreadsheet data is transferred tostatistical software package, where it becomes theprimary registry, for analysisAnalysis & abstract/manuscript generationFigure 2. Summary of methods.CDEs must be collected quickly for screening purposesand then reviewed retrospectively if any more specificquestions regarding that CDE should arise. All of thesekey points within Objective 3 provide for flexible datathat can be utilized in many different forms fromreporting to ‘Get with the Guidelines’, creating reportsfor internal quality assurance, tracking changes withinour institution, and contributing to scientific research.These objectives are compliant with the MDR-OKcategorization protocol (for mergeable data, datasetstandardized, rules for data collection, observationsassociated over time, and knowledge of Outcomes)from a previous review that outlines effective medicaldata registry protocol25 and is consistent with therecommendations of the American Heart Association.26MSRJ # 2013 VOL: 02. Issue: Springepub May 2013; www.msrj.orgLessons from a stroke program registryThe stroke registry serves a key function, as itprovides a foundation upon which other studies canoriginate, as well as generate new hypotheses. Becausethe registry also provides a foundation for ideas tocultivate, data abstractors may notice anecdotal trendsor grow curious about certain functions pertaining tostrokes. This encourages a team approach to discussingnovel study ideas, providing students with the opportunity to design and implement a scientific investigation, and allowing faculty members to cultivate theirmentoring skills.The largest advantage of having a registry is theability to prospectively add patients to the registry,while allowing investigators to collect information retrospectively if needed. The continuous addition of newpatients increases the sample size of our studies fromyear to year. Furthermore, the combination of prospective and retrospective data collection methods has beensuggested as the most efficacious means for gatheringdata in terms of completeness and accuracy.13Impact on quality improvementFurthermore, the use of this registry has allowedinvestigations into this center’s practices in order toimplement internal quality improvement measures.Whenever a question regarding complications or outcomes is raised by hospital staff, the registry is queriedto obtain the needed data. For example, an emergencydepartment (ED) nurse expressed concern for treatinga patient who woke up with stroke symptoms witha thrombolytic. The registry was queried after IRBapproval, and we were able to report complicationrates for this group of patients and compare themto complication rates of patients treated within theAmerican Heart Association guidelines; the results weresimilar. While neither research objectives nor qualityimprovement can be identified as the primary purposeof this registry, the registry has certainly afforded ourinstitution both types of information. In an additionalexample, we examined whether outcomes were compromised by prolonged length of stay in the ED.27 Wefound that it was not the amount of time spent by apatient in the ED, but rather the presence in the EDduring the nursing shift change that was associatedwith increased frequency of pneumonia.27 This is oneof the best examples of a research query at this centerthat led to a change in hospital management; however,many small changes have been implemented followingresearch queries of the registry. While significant, thesehave not always resulted in publications through peerreviewed journals.Medical Student Research Journal025

Lessons from a stroke program registryLIMITATIONSAs in all investigations and clinical data registries,there are drawbacks to our registry. One primary pitfallis that there is no specific study in mind while collectingthe information for the registry. This leaves the teamat the liberty of the treating physician as to whetherspecific laboratory values are collected, imaging studiesare ordered, and so on. Much of the information withinthe registry is retrospective, which can create problematic issues if aspects of patient care needed forresearch purposes are not included within the medicalrecord.While there are advantages to the checks andbalances of multiple points of data entry, there is alimitation to this feature as well. The multiple points ofdata transfer increase the likelihood that human errorcan affect the data transfer and also increase the totaltime spent on the process, thereby decreasing efficacy.15 Because screening of data for irregularities isconfined to outliers and gross typographical errors, it ispossible that minor errors may go undetected if they fallwithin a normal distribution for a specific data point.Over half of the errors in clinical data gathering are dueto data entry technique according to a recent study, butthere is still a substantial portion of errors that aregenerated during the reconciliation process that appears to be dependent on the knowledge of researchpersonnel.28 One unique feature of the registry is thesimilarity of the paper and digital versions of our datacollection tool (see Figure 1). Because the two forms arenearly identical with regard to the data copied fromthe paper version to the electronic version, we havefound that this reduces the risk of human error duringtranscription.Furthermore, the use of multiple team members inthe abstraction of similar data points may risk interabstractor reliability (meaning lack of consensus indefinitions of data elements between abstractorsmay lead to inaccurate gathering of these data)29 andpotentially lead to abstractor drift (meaning smallchanges in understanding CDEs by a given abstractormay result in unforeseen discrepancies in data collection). We strive to minimize this with the implementation of a very specific codebook of CDE definitions.Because the majority of our CDEs are collected prospectively by the Stroke Program Coordinator and atrained vascular neurologist, this leaves little room forpotential error with our remaining data abstractors.These errors may be reduced with the implementationof a double-entry approach,30 but such a methodology026Medical Student Research JournalJames E. Siegler et al.may not be efficient in large patient populations withlarge quantities of data.31We also implement a mandatory training period of allnew research personnel whereby a more experiencedsupervisor (usually a senior medical student with twoor more years of experience with our team) is requiredto monitor any new data abstractors and data entrypersonnel until such a time when the junior studentcan carry out these tasks accurately, effectively, andwithout further assistance. During this time, the seniorteam member also allocates a sufficient amount oftime educating junior team members regarding generalaspects of stroke pathophysiology, clinical diagnosis,laboratory and imaging studies, and management.Bi-monthly meetings with research personnel on ourteam also afford us the opportunity to review anddiscuss clinical data and their definitions in an opensetting as well as an opportunity to assess the status ofour new and ongoing investigations.Another disadvantage is that this is a single centerthat can only offer insight into a specific population ofpatients who present to our institution. This limitationprohibits our ability to generalize our results to othercenters and other studies. Our center is very unique inthat it serves patients in the New Orleans arearegardless of insurance status, and the source population of New Orleans (being in the ‘Stroke Belt’) is not arepresentative sample of the United States.32 Thisis why we have made clear, specific variable definitionsso that we can combine our registry with otherregistries to increase sample size and improve ourgeneralizability.LESSONS LEARNEDIn establishing a stroke registry, we have learnedmany lessons regarding initiation of the registry, developing CDE definitions, and commencing projects fromthe registry. One factor pertains to the responsibility ofthe research project leader, which may be a doubleedged sword. While the leadership experience gainedby medical students and residents in piloting anindependent study, working with a team from start tofinish, and presenting results in peer-reviewed journalsand at conferences is invaluable, follow-through andkeeping deadlines can be challenging due to conflictingobligations. We have learned that communication ofgoals and interests is paramount, which fosters a trueteamwork approach where students, residents, andfaculty work closely together to complete projects in atimely manner. Bi-monthly meetings to communicatethe status of the registry and related projects, and theMSRJ # 2013 VOL: 02. Issue: Springepub May 2013; www.msrj.org

James E. Siegler et al.dissemination of meeting minutes and a running list ofprojects, papers, and abstract deadlines have helped inestablishing and re-establishing expectations and resource utilization.We have also learned that investing the time tocarefully train research personnel with regard to datacollection techniques, variable definition classification,and data entry greatly reduces the errors in datacollection. At this center, all members of the researchteam are required to be certified in the NIH StrokeScale examination33 as well as undergo IRB training andcertification. New members also go through a period ofproper training and supervision from a more experienced team member as explained above. In an attemptto maintain data accuracy, we also limit the reconciliation of data errors to trained and experienced clinicalpersonnel, such as upper level medical students andresidents who understand the biological and statisticalmeaning of these data elements and can more easilyrecognize outliers, errors, and inconsistencies (e.g., theerroneous coding of a patient who expired when he orshe was discharged to home).We have learned that it is important to inform facultyand residents at your center about your registry. Theyshould know which data elements are included so thatthey can assist in the collection of information frompatients and effectively dictate these pertinent elements in their patient notes. At our center, we keepother faculty and residents informed about our registryby inviting them to our bi-monthly research meetingsand actively discussing the results of our research atregularly scheduled vascular conferences, grand rounds,and other meetings. We have also created templates foradmission and discharge notes, which include the mostimportant CDEs.The main lesson learned in this process is that data aremore effectively and accurately collected when a strokecoordinator or other trained clinical personnel collectthe majority of patient information prospectively, ratherthan retrospectively via chart review. Because of theactive, prospective collection of data by this teammember, with many elements collected for reportingto The Joint Commission for maintaining Primary StrokeCenter certification, any uncommon data elementsneeded for the registry that are not intuitively gatheredby residents or medical students (such as a specifichistory of liver disease) can be collected by the coordinator before the patient might be lost to follow-up.It is worth disclosing that in the generation of thisregistry, methods and protocols have been activelyevolving. The lessons learned during the early phases ofMSRJ # 2013 VOL: 02. Issue: Springepub May 2013; www.msrj.orgLessons from a stroke program registryregistry production have already been applied to thecurrent phase. For instance, we began the data abstraction process in 2008 with an almost entirely retrospective approach using a limited version of a CRF(approximately four pages in length with just over 350CDEs). In January 2011, the CRF was significantly revisedfor a number of reasons in order to improve the efficacyand completeness of our data collection. The revisedCRF now includes more data points that can be used forresearch queries (approximately 18 pages in length withover 1,000 CDEs) and is better organized with respect tothe order of information collection. From our experience, while there are more data to be abstracted, theimprovement in organization has dramatically shortened the time necessary for data collection andimproves the accuracy and completeness in each ofthe CDEs of the registry.FUTURE DIRECTIONSStroke is a leading cause of disability and death in theUS population,34 and research with the use of registrieshas grown to be an effective way to improve the careand quality of life of individuals who suffer from thisdisease.6, 35 37Now that the Tulane Medical Center’s primary registryhas been ongoing for several years (since 2008), futuredirections of the registry are under discussion. Currently, in the Electronic Health Records (EHR) beingused, data cannot be captured by electronic means.Instead, all information must be abstracted throughmanual searches with individual data point abstractio

Stroke Center certification and to address a specific study question related to the safety and efficacy of com-bined anti-platelet therapy during the acute phase of ischemic stroke.19 The larger registry includes all but a handful of data points requested by 'Get with the Guidelines-Stroke'12 and all of the data points needed