Lesson 8: Strategies For Data Quality Management And Improvement - AHIMA

Transcription

Slide 1Lesson 8: Strategies for Data QualityManagement and ImprovementThis presentation will address another mechanism for providing quality data .EDITS.An equally important role of the registrar is data management. Abstracting and getting the data into thedatabase correctly is very important. But, maintaining the data - through evaluating the completenessand quality of data - are equally important.EDITS is a very technical subject. The purpose is not to get into the technical programming of edits but tomake you aware of what edits are, resources to help you manage edits and what to do if you get an edit.

Slide 2WebEdits (NAACR10B) - NAACCR Call for Data Edit Report02/15/12GENEDITS Output - Feb 15, 2012 8:26a.m.EDITS Runtime Metafile EDITS\META\NAACR10B.RMFInput Data File tmp\9558.INError Message Report File tmp\9558.OUTPatient Id Last NameFirst Name Site Morph 3 Dx Date (Rec #)Edit NameError Message (E: Error W: Warning M: Message)Field (Column)Current Value New ---------------------------01113196 GOAHWAYTOC778 9698 12062011 (14)Morphology--Type/Behavior (SEER MORPH)E:Grade not valid – ICDO3Histology Type ICD-O-3 (301) 8000Behavior ICD-O-3 (300) 3Grade (306) 6Over-ride Histology (1129) BLANK Date of Diagnosis (283) 1206201101058879 CHECKAGNBETTERC482 8461 02222011 (37)Primary Site, Morphology-Type ICDO3 (SEER IF25)E:Site & Morphology conflict - ICDO3Over-ride Site/Type (1128) BLANK Primary Site (291) C482Histologic Type ICD-O-3 (301) 846100869338 THISWORKWILLC341 8000 02262011 (46)Summary Stage, Date of Diagnosis (NAACCR)E:If year of Date of Diagnosis 2001, then SEER Summary Stage 2000E:cannot be blankDate of Diagnosis (283) 02262011SEER Summary Stage 2000 (529) BLANK This is an example of a typical Edits report. It lists the errors that were found when the data was ranthrough the Edits program. Some of the elements of this report will be discussed in detail throughout thispresentation.

Slide 3What is an edit?Edits “check” and“enforce” the datastandards reflected inthe data standards anddefinitions manuals.In this context, edits are computer programs that contain the necessary logic and algorithms to check thevalidity of cancer registry data.The data standards that your data is checked against are those specified in the FORDS, the ICD-O,Summary Staging Manual, AJCC staging manual, Collaborative Stage manual, etc.It takes as many rules as it can from your manuals like the surgery code must be 98 for leukemia, or thesite code must be C42.1 for leukemia and turns the rule into programming language that your softwarecan understand. The program can then compare the items included in the edit to see if the data item (orcombination of data items) is in agreement with the rule.Edits are built into the cancer registry database software. The database will have an edits feature that willallow you to run the edits against the data.

Slide 4Why are EDITS used? STANDARDIZATION and ACCURACY Improve the quality, comparability, andusefulness of cancer registry dataThink about the Big Picture:Once the data are collected from the source document, they are increasingly shared electronically withmultiple data bases.

Slide 5Flow and use of cancer data provided by cancer registrars.Public Health InformationResearch ResultsTrends and StatisticsNCDB (CoC)NAACCRNPCR (CDC)65 State and TerritoryCentral Registriesin North AmericaCoC Approved Programs(@1450 programs)Patients (1,067,280)SEER (NIH)* See note belowAll other Programs(@2400 programs)Patients (266,820)* A portion of these (11 states and 4 geographical areas) report to SEER.For example:Facilities (CoC accredited and non-CoC accredited) send data to the state central registry which in turn(depending on the state) is required to meet NAACCR, SEER, and/or NPCR guidelines.If the facility is a CoC accredited cancer program, they are also required to send the data to the NCDBand participate in studies using registry data.You may also use software provided by a vendor with clients in many other states that also have localand national needs for their data.All of these national and state-wide data collection efforts have increased the importance ofstandardization and accuracy of data. When data from different facilities are shared, it can then becompared because it was collected using the same set of rules.As discussed in the Abstracting Methods course, one of the reasons for providing text is that no one isperfect. Everyone makes mistakes. Edits provide another safety net for helping catch some of thoseerrors and allowing you an opportunity to find and correct mistakes as well as, hopefully, learn from thosemistakes. Yes, everyone knows that a male cannot have ovarian cancer. But, mistakes that obvious arevery easily made.

Slide 6Who uses EDITS? NAACCRSEERNPCRCOC (NCDB) State cancer registries Hospital cancer registries Cancer registry software providersEveryone collecting cancer registry data uses Edits. Edits are based on a consensus of all of thestandard setting agencies. They not only decide what should be checked, but also what can be checkedusing the edits format – and how the edits would go about doing all of this. Edits usually contain thosedata items that are collected nearly universally and used for analysis. Then, all of these checks areincorporated into computer program language.Because each of these agencies have slightly different reporting requirements and standards, therequired edits for each standard setting agency are grouped together in an edit set. The majority of editsin the edit set are the same because they are looking at the same data items. But, there are differencesas well to address those requirements specific for that agency. For example, one agency may allow afield to be blank while another may require it to be filled with a special code. If you report to more thanone agency, it is possible that you will have to run more than one set of edits on your data. Edits and thevarious edit sets specific to each agency the registry report to provide the registrar with the assurancethat the cancer registry data are being edited the way that particular standard setter requires.

Slide 7The Metafile ns/VolumeIV.aspxThe edits metafile contains the compilation of the various edits that can be ran on the data. There aregenerally several hundred edits available. Each edit will have its own programming code and logic. All ofthis information is combined into one document, called the edits metafile, that is in a format that can beimported into the cancer registry database. The metafile contains the data item descriptions, recordlayouts, edit logic and edit documentation.NAACCR is the organization that publishes and distributes the edits metafile. The NCDB, state registries,software vendors, and others who are responsible for providing edits will take these metafiles andcustomize them based on their agency’s specific requirements. For example, the NCDB will modify themetafile to meet their requirements and will make them available to CoC accredited facilities.This is a print screen of the various metafiles that have been made available over the past few years.Each year, as the data collection requirements change, so must the edit metafile. The edits that are ranon the data must be in accordance with the coding rules and instructions that were in effect. Therefore,the metafiles are updated frequently. Each metafile is given a version number, for example, for 2008data, the version in effect was version 11.2, for 2009 data, it was version 11.3. For 2010 data, the versionin effect is version 12.0, for 2011 data it was 12.1 and for 2012 it is version 12.2. In most facilities, it isthe responsibility of the standard setting agency to provide the appropriate metafile to the software vendorwho will then ensure that the proper edits are available in the cancer registry database.You may read in your CoC or state central registry submission requirement documentation that a certainversion of edits must be ran on the data. This is what that is referring to. Each metafile contains editsapplicable to data with certain years of diagnosis. Remember that your software vendor plays animportant role in helping you through this process. The take home message when you hear reference toa specific version of the edit metafile is that the version is tied to a specific set of guidelines and the editmetafile and your data have to match.

Slide 8Types of EDIT Checks Single field“not a valid value for ” Interfield“conflict among ” InterrecordSex 0Sex 2Primary Site C61.9Prim 1: Sequence 00Prim 2: Sequence 02As mentioned earlier, the edits are built into the cancer registry database software. There are a fewdifferent ways that the software uses edits to check data. Some edits identify definite errors thatrequire correction. Others may identify possible errors that require review. The three main waysedits are used to check the data are:1.Single field edits:These edits look at a single data item and verifies that only the acceptable values are used.Usually the error message begins with wording to the effect of “not a valid value for ”Example: For the Sex data item, only the codes listed in the FORDS can be used. It is arequired data item and therefore must be collected according to the requirements specified inthe FORDS. In this abstract, a code of 0 was entered in the Sex data item. Because 0 is not avalid code, an edit is created. To correct the edit, code 0 must be changed to a valid code. 2. Interfield comparison: These edits compare codes in one data item to other related data items. These error messages usually begin with wording to the effect of “Conflict among ”. Example: The metafile includes an edit that will compare the information in the Primary Site andSex data items. In this example, the primary site is prostate cancer and sex was recorded asfemale. According to the edit logic, a female with prostate cancer is a conflict. An edit error isgenerated and allows the registrar to review the case to make a correction. Because more thanone data item is involved, the error can be in either data item. Both data items need to bereviewed to determine exactly what the error is and what the correction needs to be. Weassume that the sex is incorrect, but it is possible that the site is wrong (should be C16 insteadof C61). The text should supply sufficient information to determine which code was incorrectwithout having to repull the medical record. If not, then the quality of the text should beexamined.3. Interrecord comparison: These edits compare data in one tumor record to related data items in another tumor record forthe same patient. This would be for situations in which the patient has more than one primarycancer (multiple primaries). Example: This edit is checking to see if the sequence code is entered in a “logical” order. Theedit can’t possibly know that a sequence of 02 and 03 should have been 01 and 03, becauselogically these are in an acceptable order. But, it could catch an instance where the sequencefor the first primary needs to be updated from 00 to 01 due to the diagnosis of a second primary.

Slide 9When Do EDITS Occur? Real-time: Ran atthe time ofabstracting May be on a caseby case or fieldby field basis Corrected at thetime ofabstracting Batch: Ran at the timeof (before) datasubmission Edit report generated Corrections made toall cases listed on thereportEdits are generally ran against the data at two different times in the abstracting process. Whenabstracting a case and an incorrect code is entered into a data item, an error message pops upimmediately. The software will not allow the abstractor to move to the next data item until the error isresolved. Or, after the case is abstracted and the abstractor wishes to mark the case as complete, editsare ran and any errors will be displayed. The abstractor must go back into the case and correct the errorsand clear all edits before the status of the case is marked as complete and ready for transmission. Theseare examples of real-time edits. The edits are ran and corrected at the time of abstracting.The other type is often called “batch” edits. Here, the registrar would instruct the software to create a filethat contains a batch of cases. Usually this is done when the data needs to be submitted to the NCDB orthe state central registry, but the file could be created for any number of reasons. The file will contain allof the cases that need to be reported. But, before the file is actually sent, those cases need to be ranthrough the edits again (even if there were edits ran at the time of abstracting). Many things can occurthat would cause new edit errors to appear. For example, an updated edit metafile may have beeninstalled and contains new edits that need to be ran on the data. The file will be ran through the editsprogram and will produce an edit report similar to the one displayed at the beginning of the presentation.The registrar must then go through the report, bring up each case on the report in the database, andmake any corrections. Once all of the corrections are made, it is important to recreate the file with thesecases so that it contains all of the corrections and no errors. When submitting data to the NCDB or thestate registry, the file containing all of the data to be submitted should be ran through the required editsand all corrections made BEFORE the submission. As you learned earlier in the module, cases that failedits will be returned by the NCDB and may be returned by the state registry (depending on theirrequirements). Any corrections must be made and the case resubmitted in order to be in compliance withthe standards.

Slide 10EDITS are REQUIRED EDITS must be ran on each caseAny errors must be correctedEach case must pass all EDITSRe-run edits as many times as necessary until yourdata file contains no errorsRegardless of how your software handles edits, it is important that each case pass all of the edits. And,you must make these corrections before transmitting the data to the NCDB or the central registry. Do notassume because that is the way you coded it, that it must be correct. You may need to review the text oreven the original source document to confirm the correct code.Also, you may have to run edits several times before all edits are passed. Correcting one data item maycause a new conflict with a different data item. For example, if one edit identifies that the primary sitewas miscoded, changing the site code will have an effect on many different data items such as laterality,stage, surgery codes, etc. Changing the information in site code will most likely cause new conflicts withthese other data items. Running the edits again will identify these new conflicts.After you correct any errors, be sure to RE-CREATE a new data file that contains the corrections, run itthrough edits again, and make any additional corrections. Continue this cycle until there are no errors.Once the file has no edit errors, then it should be submitted.

Slide 11Guidelines and Procedures Software specific procedures NCDB Call for Data Guidelines– l– http://www.facs.org/cancer/ncdb/edits.html State Registry GuidelinesHow the process of running edits are accomplished using the cancer registry software may varydepending on the vendor. There is no standard way of accomplishing this task. Usually there areselections on a drop down menu that provides the options for running edits. The NCDB and the stateregistry will provide specific guidelines for when and how edits are to be ran and corrected. The NCDBprovides instructions to CoC accredited facilities on the NCDB website. There are several documentsavailable regarding the call for data including a few that pertain to edits. This information is updatedevery year to correspond with the current year’s call for data. The registrar should review this informationcarefully before submitting data to the NCDB.Slide 12Great Resource! RegistryPlus Online Help: Integrated Edit Helpand Easy-Click Cancer Manuals CDC NPCR product Download and install on your local hard disk Link to this application:– rpohtech info.htmThis is a great resource, not only for edits but for abstracting as well. This application was built anddistributed by NPCR and is called Registry Plus Online Help. It is free and can be downloaded from theCDC website directly to your desktop.Registry Plus Online Help is updated with the most recent manuals in publication; however, there may bea delay between the release of the manual and the updated version of the application. You should checkthis site periodically for updates, especially after a new manual version has been released.

Slide 13Provided on this slide is a print screen of this application that shows a list of all of the resources includedin the application.Registry Plus Online Help has full copies of the FORDS, Collaborative Staging, and Multiple Primary andHistology Coding manuals as well as the NAACCR Data Dictionary, the SEER Coding Manual, and theintroduction and histology sections of ICD-O-3 (Note: the ICD-O-3 is not easy to use here. It isrecommended that the book be used.)This presentation is related to EDITS. The information pertinent to edits is the last one listed.

Slide 14The edits Help tab lists every edit that could be ran on cancer data. There are several hundred editsavailable. If your state or software requires a specific edit, then it won’t be included here. But, the largemajority of the edits are listed.This link includes a description of the edit and often helpful hints in correcting the edit.

Slide 15It provides instructions for how to read the edit report (like the one earlier in the presentation).

Slide 16You can click on the edit name and it will display all of the details related to the edit including the editdescription and the programming logic.

Slide 17The edit description lists:1) The data items that are checked. These are hyperlinked to the data dictionary (or definition of the dataitem) like the definition for the Primary Site data item in the FORDS so that you can read the rules forthe data item.2) The error message that will appear on your edit report so that you can make sure that you are lookingat the same edit that is on your report edit.3) Some edits also include helpful hints for resolving the edit. For example, in this edit, it lists site andhistology combinations that are impossible. If you have this edit error, you can check this document tosee if the site and histology coded in your abstract is considered an impossible combination.

Slide 18This particular edit goes even further and provides additional information and instructions for helping tocorrect the error if the site and histology are an impossible combination. This is a valuable resourcecontaining information that is not easily found in the coding and staging manuals.This resource was briefly introduced in the Cancer Disease Coding and Staging course in the lessons onusing the ICD-O-3. This is the full document and reference for the partial information that was provided inthat course.

Slide 19Which field is the problem?Error Message:Behavior Code ICD-O-3 cannot 2 for this site and/or histologyPrimary SiteHistologyBehavior CodeC42198002So, what do you do if you get an EDIT?Remember, you may get an edit when you try to save the case at the end of abstracting the case, or youmay get a report of edit errors when you try to create a file to submit to the NCDB or the state registry.The first thing you need to determine is which data item is causing the edit. The edit error will list the dataitems that the edit is programmed to check and also the values that were entered in the abstract.Next, you need to think about the coding rules for that data item and determine how this case isconflicting with the rules.Let’s look at this example.The three data items that were checked in this edit were Primary Site, Histology and Behavior Code. Thevalues recorded in these fields tell us this is a leukemia with a site code of bone marrow (we would alsodetermine that when we looked up the case in the database).This combination of site (bone marrow) and histology (leukemia) follows the rules and is not a conflict, sothat part is ok.The other data item listed was behavior code. We should look to see if the behavior code is in conflictwith the site or histology or both. The manual that is used to code the data item in question is a goodplace to start to find out what the rules are and what the error may be. In this case, we would have usedthe ICD-O-3. Remember there may be rules in the other coding and staging manuals as well. By beingin the habit of using the manuals, you are more familiar with their contents and where additionalinformation may be provided!In the ICD-O-3, 9800 is listed with a behavior code of /3 (invasive). Based on the rules for codingleukemia, we know that leukemia is always coded with a behavior code of /3 (invasive). The behaviorcode of /2 (in situ) is in conflict with this rule. Therefore, the error is in the behavior code. To correct thecase and clear the edit, the behavior code should be changed from code 2 to code 3 in the abstract.When the edits are ran again, this edit should no longer appear.This is a fairly easy and obvious example. But, in the data entry process, typographical errors this simpleare easily made.

Slide 20Which field is the problem?Error Message:Conflict Among Primary Site, Histologic Type ICD-O-3,CS Site-Specific Factor 9, CS Site-Specific Factor 10, GradePrimary SiteHistologyCS SSF 9 Gleasons patternCS SSF 10 Gleasons scoreGradeC6198140/30440089Some edit errors are not so straight-forward. This error message says there is a conflict between theinformation in the combination of data items listed but doesn’t tell us exactly what the conflict is. All ofthese values are valid codes for these data items. And, none of them were left blank. So, where is theconflict?What manuals were used to code the listed data items? ICD-O-3, Collaborative Stage and FORDS.When coding the grade for a prostate primary and a Gleason score is stated, what flag does that raise?The Grade Conversion tables in Section One of the FORDS! What does a Gleason pattern of 044 andscore of 008 tell you? That you know the grade.008 is a Gleason score that is equal to poorly differentiated on the conversion tables in the FORDS. Ifyou know the Gleason pattern and score, then by default you must know the grade. It can’t be 9(unknown).What do we need to do to correct this edit? The grade of 9 should be corrected to 3 (poorlydifferentiated).Many edits are similar to this one. Abstracting is more than understanding the coding rules for one dataitem at a time. You have to understand the logic and relationship between data items. And, whenabstracting, you may not necessarily be thinking about the relationship between data items as you maybe more focused on trying to determine the code for that particular data item. This is great example ofhow edits can provide a valuable role in identifying errors and conflicts in the data.

Slide 21Which field is theproblem?#1111111 PIN: 007233222 Nam: COLONSNC: 00 Pri: C184 HTI: 8140 BCI: 31SSN:Dat: 05302011Edit: Surgery, Rad, Surg/Rad Seq (COC)E: Conflict among surgery items, Rad--Regional RX Modality, RX Summ--Surg/Rad Seq2RX Summ--Surg Prim Site (859)RX Summ--Scope Reg LN Sur (861)RX Summ--Surg Oth Reg/Dis (862)Rad--Regional RX Modality (909)RX Summ--Surg/Rad Seq (875)[30][5][0][25][0]34This is a copy and paste from a typical edit report and looks more similar to the format that you will see.The points below are numbered and correspond to the numbers on the slide.1. There is the edit name and then the error message (this is what you can look up in Registry PlusOnline Help to see if there are hints for resolving the edit)2. The number in parenthesis after each data item name is a unique id number for the field and can beignored.3. The number in the bracket is the value that was entered in the abstract. This abstract has a surgerycode 30 and radiation modality code 25.What’s wrong with this case?4. The code entered in the Rx Summ--Surg/Rad Seq field was 0 (not done). Because radiation wasgiven, this cannot be 0, not done.How should this case be corrected? The code 0 should be correct to 3 (RT after surgery) or theappropriate code that correctly identifies when RT was given in relation to the treatment.

Slide 22Site and Morphology ConflictsCheck versus Impossible Check– Histologies that rarely occur in that site– Source documents should be reviewed foraccuracy– Option to set an Override Flag ONLY After Confirmation Impossible– Biologically impossible– The particular cancer cannot arise in thespecified siteAnother edit that we see quite a few of are Site & Morphology Conflicts. It can either state that it is a“check” or an “impossible” combination. These can sometimes be difficult to resolve.A “check” error does not imply that this is absolutely an error but rather that this combination is sounusual that it should be checked to ensure that it correctly reflects what is in the medical record. If, afterappropriate review, the combination is determined to be correct, then you can set the override.When there is absolutely no way the specified combination could have occurred, you will get an“impossible” error. Either the site or the morphology (or both) need to be corrected. Sometimes theseare easy to fix. You just had a typo in your code. Sometimes they are not so easy, especially if that ishow it stated in the chart (melanoma of the peritoneum, for example). The slides introducing the RegistryPlus Online Help software showed an example of an “impossible” error and the helpful hints that wereprovided to help resolve the edit.

Slide 23Site and Morphology ConflictsCheck versus ImpossibleNPCR Patient Id Seq Central Last NameFirst Name(Rec #)Edit NameError Message (E: ErrorW: WarningM: Message)Field (Column)Current ValueNew ----------------------5678901200LERNINIMAPrimary Site, Morphology-Type Check (SEER IF25)E:Site & Morphology conflictOver-ride Site/Type (917) BLANK Primary Site (227) C559Histologic Type (232) 8933(41)This is an example of a CHECK error. It edit names includes the word “check”. This particular edit issaying that there is a conflict between the site code (uterus, NOS) and the histology (adenosarcoma).Resolution of this may require inspection of the abstract’s text or review of the original source document.1) Is the histology correctly coded? Did you look it up correctly in the ICD-O-3 and enter it correctly?2) Is the primary site coded correctly? Make sure it is the primary site of origin instead of the site ofregional or metastatic spread - perhaps you coded the site of biopsy instead of the primary site.Often the problem is due to a typo or when you look it up in the ICD-O-3 you find the correct code for thehistology term stated in the text.If the unusual or rare combination is determined to be correct, be sure to include a note in the text todocument such discussions and decisions. Then, set the override. The edit will specify which overridefield to set (Over-ride Site/Type).

Slide 24Another resource!ICD-O-3SEER Site/Histology Validation t one is dated: 12/04/2009Provided in both PDF and Excel formatsAnother resource for coding site and histology and for resolving edits is the SEER Site/HistologyValidation List.The link to download this to your computer is listed on the slide. The PDF format is recommended as it iseasier to read.The edits that produce the site/histology “check” errors stem largely from this document. This documentlists the usual histologies for each primary site. If the site and histology combination is not listed in thisdocument, then the edit error is produced.

Slide 25Example of Site/Histology Validation ListNote: Histologies that occur rarely or never may not be included.Here is the section from this manual for C55.9 (the site code from our edit error).Notice code 8933 is not listed under C55.9. Our edit error was just a check - meaning this is a very rarecombination and requires further investigation (BUT, it is not an impossible combination). You may evenneed to discuss with your physician advisor before making a final decision.Because this file is a PDF document, you can also do a “find” on 8933 to see if this histology IS listed forany other GYN site.If you search for 8933 in the site and histology validation list, you will find that it IS listed for C54 (CorpusUteri). Does that mean the site code should be changed to C54? NO! Does this mean that 8933 andC55 is absolutely wrong? NO! Remember our edit was only a “check” error. Also, keep in mind thathistologies that only occur rarely may not be included.What this information does give us is a clue of what to look deeper for in the chart or consult with thephysician to see if the more specific site code of C54 was perhaps the primary site instead of uterus, NOSBEFORE just accepting it as the best you can do and set the override.Throughout this program, it has been emphasized to investigate further before defaulting to a less specificcode and to strive to assign the most specific code possible. This is a perfect example of this. Utilize theresources that have been made available to help you do this!

Slide 26When Can I Set the Over-ride Flag? Designed to work with the EDITS process Identifies rare, but possible, combinations If, and only if, upon review, the unusualcombination is verified as being correct Indicates the

data, the version in effect was version 11.2, for 2009 data, it was version 11.3. For 2010 data, the version in effect is version 12.0, for 2011 data it was 12.1 and for 2012 it is version 12.2. In most facilities, it is the responsibility of the standard setting agency to provide the appropriate metafile to the software vendor