Transcription
Science and Engineering Ethics (2006) 12, 23-39Managing Data for Integrity: Policies andProcedures for Ensuring the Accuracy andQuality of the Data in the LaboratoryChris B. Pascal*Director, Office of Research Integrity, Department of Health and Human Services, USAKeywords: research integrity; data management; training; mentor; principal investigatorABSTRACT: Management of the research data is an extremely importantresponsibility of the Principal Investigator (PI) and other members of the researchteam. Without accurate data, no worthwhile conclusions can be drawn from theresearch study. Integrity in data management is critical to the success of the researchgroup and to public trust in the research outcomes. One of the primary responsibilitiesof the PI is to provide proper training to the junior members of the lab. This effort canbe buttressed by institutional data policies that are implemented at the group level.Extensive and frequent guidance in good research practices by the PI and other seniorresearch staff is critical to the proper training of new scientists.Many scientists think of data management as how you collect the data, record andmaintain it. These are important but not sufficient to ensure the quality of the data forpurposes of interpretation, reporting, and publication. This article is intended to exploredata management in a more comprehensive way that takes into consideration thepositive contributions that can be made by institutions and laboratories in adopting datapolicies, holding regular lab meetings to review and discuss the data, and providingformal training of young scientists in the principles and skills in interpreting andreporting the data.* The opinions expressed herein are those of the author and do not necessarily represent the views ofthe Office of Research Integrity, the U.S. Department of Health and Human Services, or any otherfederal agency.This paper was presented at the 6th International Bioethics Conference on the subject of ‘TheResponsible Conduct of Basic and Clinical Research’, held in Warsaw, Poland, 3-4 June 2005.Address for correspondence: Chris B. Pascal, J.D., Director, Office of Research Integrity, Office ofPublic Health and Science, Department of Health and Human Services, 1100 Wootton Parkway, Suite750, Tower Building, Rockville, MD. 20852, USA; email: cpascal@osophs.dhhs.gov.1353-3452: 2006. Published by Opragen Publications, http://www.opragen.co.uk.Science and Engineering Ethics, Volume 12, Issue 1, 200623
C. B. PascalAs many readers already know, the U.S. Office of Research Intergrity (ORI) has anabiding interest in providing education in the responsible conduct of research (RCR).ORI has provided financial resources for research institutions to develop educationalproducts in various RCR areas such as managing conflicts of interest, peer review,authorship, and data management. Many of these products are web based and areposted on the ORI website.1An important part of ORI’s mission is focused on the integrity of the researchprocess, such as the accuracy of the research data and research publications, and theprevention of, or reduction of, research misconduct. Because accurate scientific resultsare critical to the advancement of science, an argument can be made that the propermanagement of the data, and in particular, the quality and accuracy of the data, is themost important element in ensuring scientific integrity and public confidence inresearch results and findings.Responsible Research at the Individual LevelIn 2002, the Institute of Medicine,2 issued a report on research integrity stating that“the responsible conduct of research is not distinct from research; on the contrary,competency in research encompasses the responsible conduct of that research and thecapacity of ethical decision making.” Another way to address this issue, is to say thatone cannot separate responsible research from the competency or quality of thatresearch. By the same token, lack of knowledge, skills, and experience in research is inmost instances invariably fatal to research quality.In discussing RCR at the individual level, the IOM report2 (p.5) lists the followingqualities needed by investigators to conduct responsible research– intellectual honesty in proposing, performing, and reporting researchaccuracy in representing contributionsfairness in peer reviewtransparency in conflicts of interestprotection of human subjects; humane care of animals andadherence to mutual responsibilities between investigators and researchteamsAlmost all of these principles and concepts are related to the quality of the researchdata in some way. Intellectual honesty in proposing, performing, and reporting researchcertainly encompasses accuracy in describing data collection efforts, reporting datafrom prior studies, and the inclusion of preliminary data in the application in order todemonstrate the feasibility of the research proposal.Accuracy in representing contributions includes a fair and accurate description ofthe contributions of the lead author and making sure that other authors receiveappropriate credit for their data or data interpretations included in the manuscript orproposal.Fairness in peer review requires that the reviewer give appropriateacknowledgment to the author if the data appear sound, data interpretations are24Science and Engineering Ethics, Volume 12, Issue 1, 2006
Managing Data for Integrityappropriate, the methodology is appropriate to the type of research involved and theproposed article makes a contribution to the literature. If data criticisms are made, thereviewer must have adequate justification for those criticisms.Conflicts of interest may affect the interpretation and reporting (or withholding ofdata). When conflicts are disclosed, the reader of a journal article or user of the datamay take into account the potential bias of the investigator. However, this may beinsufficient to ensure the integrity of the data. If the conflict is very impactful, such as alarge stake in the research project, the conflict may lead to apparent or real bias.Although an outsider may be aware that bias could exist, he or she may have a difficultor impossible task in coming up with an accurate account of the level of the bias andhow harmful it may be to the safety of the study and the quality of the data. In suchcircumstances, the institution or investigator should assume direct responsibility forensuring the quality and reliability of the data. If that cannot be accomplished, theinstitution may need to completely bar the investigator from participation in the study.Data validity and reliability is certainly important in both human research andanimal research. If the plan for data collection, interpretation, and reporting is vague orpiecemeal, it may not be possible for researchers to make sound conclusions about thedata. Hence, poorly designed studies may pose unnecessary risks to human subjects oranimals, suggesting that it may not be inappropriate to conduct the study at all.Certainly, if there is more than minimal risk involved, the investigator, institution, andIRB should seriously consider not going forward with the proposed research. If thestudy is worth doing, it is important to collect and use the data in a way that advancesscience and benefits the public. This means the data must be valid, reliable, andinterpreted and reported in a reasonable fashion.Adherence to mutual responsibilities of research teams certainly includes dataresponsibilities. The lead investigator has responsibility to establish the data collectionprocedures, make sure the research team understands its responsibilities, and providetraining and supervision as needed. Likewise, the members of the research team needto be diligent in following the procedures, asking questions as needed, and letting thePI know if there are problems with the data.Data issues are clearly important throughout the research process as indicated byhow often they are linked to the RCR issues identified by the IOM report concerningresponsibilities at the individual level of research.Responsible Research at the Institutional LevelThe IOM2 (p.5) recommends that the institution address RCR by the following– provide leadership in RCRencourage respect for everyone involvedpromote productive interactions between trainees and mentorsadvocate adherence to rules regarding the conduct of researchconduct thorough inquiries and investigations into alleged misconductoffer educational opportunities in RCRScience and Engineering Ethics, Volume 12, Issue 1, 200625
C. B. Pascal monitor and evaluate the institutional environment supporting integrity andRCR and use this knowledge for continuous quality improvementORI also recommends adoption of specific institutional policies and guidelinesthat support research integrityThese institutional RCR elements can impact data integrity in a positive way,especially through RCR education programs, particularly programs that focus onresearch data, regular mentoring of young scientists in the laboratory, and in particular,institutional efforts to adopt formal data policies or guidelines for the institution, thatare then implemented at the laboratory or group level. Before we discuss these positiveinputs to data integrity, let’s discuss the risks involved in handling, interpreting, andreporting data.Risks to Data IntegrityThere are a number of risks to data integrity, including research misconduct,questionable research practices, poor mentoring or training in handling data, lack ofproper guidance from the institution and lab, and poor data practices in the lab.Research misconduct3 is a clear danger to data integrity because experience hasshown that most ORI misconduct cases involve data fabrication or falsification. Bydefinition, such fabrication and falsification is the absolute antithesis of proper datapractices. Also, many cases of misconduct lead to misrepresentations in the scientificliterature. ORI and the institutions that respond to research misconduct allegationsunder the ORI regulation4 have had over 120 scientific articles corrected or retracteddue to research misconduct or, in a few cases, a determination by the institution that thearticle was not supportable.Also, two studies by Kalichman and Eastwood5 on research students and post-docssuggest that they will modify data to get published or funded over 10% of the time.Although the studies do not indicate that these incidents are clear misconductviolations under the Federal definition of research misconduct, i.e., fabrication,falsification, or plagiarism, it is likely that some unknown percentage would constituteformal research misconduct. Also, a recent article in Nature by Martinson,6 et al,indicates that misconduct occurs in young and mid-career investigators funded by NIHover 1% of the time.Questionable Research PracticesQuestionable research practices (QRPs) are also a threat to data integrity and probablymore common than incidents of formal research misconduct. These are practicesidentified by the National Academy of Sciences (NAS)7 as practices that are notacceptable to the community and not in conformity with normative behavior, but notserious enough to be considered research misconduct. The NAS has identified thefollowing behaviors as being QRPs:26Science and Engineering Ethics, Volume 12, Issue 1, 2006
Managing Data for Integrity failure to retain research datamaintaining inadequate research recordsauthorship without a significant research contributionrefusing reasonable requests for access to unique research materials or datamisrepresenting speculations as fact or releasing preliminary research results,without sufficient data to allow critical reviewinadequate supervision or exploitation of subordinatesusing inappropriate statistics to enhance the significance of research findingsThe NAS7 (p6) goes on to say that QRPs erode “confidence in the integrity of theresearch process, violate traditions associated with science, and affect scientificconclusions.”All of the QRPs identified by the NAS affect the data in one way or the other.Certainly, failure to retain the data or maintaining inadequate research records directlyaffects the ability of the research study to reach valid or reliable conclusions.Authorship without a significant contribution may give unfair credit (including creditfor the data) to an investigator who does not deserve it. Refusing reasonable requestsfor data may violate scientific norms and violate the NIH8 data sharing policy.Misrepresenting speculations as fact or releasing preliminary results beforeadequate review may undermine the credibility of the scientific enterprise and unfairlyraise expectations of scientists and the public in new discoveries. Inadequatesupervision of subordinates may lead to poor data practices, and harm the training ofyoung scientists. The use of inappropriate statistics may inappropriately inflate theapparent significance of research and may harm human subjects or the public if thefindings lead to clinical treatments based on faulty data.Poor Data Practices in the Lab Can Harm the Training of YoungScientistsA key responsibility of the Principal Investigator (and other senior members of the lab)is the training of students and young investigators. Training in good data practices iscritical to the training of young scientists and certainly impacts on the quality of datagenerated in the lab. But what happens if little or no training is provided in the lab orworse yet, training is provided but it consists of poor data practices. This can positivelyharm both the trainees and the lab itself by generating poor quality data that may beincorrect and lead to faulty conclusions.Based on ORI experiences with individual cases of research misconduct andallegations of misconduct that do not result in misconduct findings, ORI has identifieda number of lab practices that do not result in good data outcomes.Poor Data
Integrity in data management is critical to the success of the research group and to public trust in the research outcomes. One of the primary responsibilities of the PI is to provide proper training to the junior members of the lab. This effort can be buttressed by institutional data policies that are implemented at the group level. Extensive and frequent guidance in good research practices .
