WIXLIB

Development of a Software Testing Best Practice Frameworkfor Medical Device SoftwareAndrzej Beniamin BujokAbstractSoftware testing is essential to maintain a high quality of software. This is particularlytrue in the medical device domain where software quality is closely related to safety, andsoftware failure can cause injury or death to a patient. The increasing number of adverseevents and recalls of medical devices due to software failures demonstrates the need toimprove the safety of medical device software. Organisations that develop medical devicesoftware are required to test increasingly complex software and detect an increasingnumber of defects. Software safety can be improved by advancing the testing efficiency indetecting software defects that could result in medical device malfunction.International standardisation organisations address these challenges by publishinginternational standards containing information on various aspects of generic and medicaldevice software testing. However, a review of these standards has revealed a lack of aconsolidated set of requirements in the form of a single standard which incorporatessoftware testing best practice and related development and risk management activitiesthat are required for the development of medical device software. This thesis addressesthis lack of consolidated information through the development and validation of thesoftware testing MED-V-STEP framework. This framework maps the activities of the mostrecent generic software testing standard to the relevant activities of the medical devicesoftware development and risk management standards and defines the relationshipsbetween them. The framework enables the implementation of software test processes tomaintain these relationships between test, development and risk management activities.The development of the MED-V-STEP framework generates a contribution by providingorganisations with the knowledge of how to implement generic non-domain-specific testprocesses for testing medical device software to address identified software testingchallenges and improve testing efficiency. The MED-V-STEP framework has beenvalidated by focus groups and questionnaires carried out with medical device softwaredevelopment organisation and software testing organisation. The validation findingsindicate the benefits of testing using the MED-V-STEP framework rather than multiplestandards. The framework was validated as improving the software testing efficiency andenabling implementation of software testing best practice for medical device softwarewith low implementation overhead.x

Map of Thesis – Part 1Chapter 1) IntroductionYou are herePart 1Chapter 2) Literature ReviewPart 2Chapter 3) Research DesignChapter 4) Development ofMED-V-STEP FrameworkPart 3Chapter 5) Validation ofMED-V-STEP FrameworkPart 4Chapter 6) Discussion1

Chapter 1 IntroductionThe potential of software to provide increased functionality has resulted in thedevelopment of increasingly complex and extensive software in various industries overthe past few decades. Using such software offers many benefits but can also bring issuesrelated to quality and safety. Ensuring software quality poses a significant challenge dueto the intangible nature of software. Software development involves software testing andother verification & validation activities to prevent or detect defects that may reduce itsquality or introduce a safety risk to users. This is evident in the healthcare field, in whichthe software functionality improves the services provided by medical devices, but at thesame time may introduce safety problems such as injury to or death of patients. Due tothe significant impact of software on the functionality of medical devices and the servicesthey provide, and the existing software quality and safety issues, this thesis focuses onthe role of software testing in terms of software quality and safety, and the challengesrelated to software testing and improving medical device software testing.Section 1.1 in this chapter outlines the role of testing generic software and medical devicesoftware as well as problems that exist in terms of software quality and safety. Softwaretesting challenges faced by software development organisations in addressing the qualityand safety issues are presented in Section 1.2. Section 1.3 provides an overview of therole of standardisation organisations and international standards in software processimprovement and how they contribute to the improvement of generic software testing andmedical device software testing. This is followed by Section 1.4 outlining the design anddevelopment of an innovative software testing framework to address the challengesassociated with medical device software testing. Section 1.5 specifies the researchquestions to be answered in this thesis, while Section 1.6 specifies the research objectivesto address these research questions. Section 1.7 outlines the contributions generated bythis thesis. The document structure with an overview of the following chapters ispresented in Section 1.8.2

1.1 Software TestingTesting is indispensable in software development to determine software quality (Planning2002, Bertolino 2007). The process of running the software against a number of tests andevaluating the test results is the most frequently used means for assessing software quality(Tian 2005 p.5). Human error in the development process poses a risk of defective andtherefore poor-quality software, as some defects can lead to software malfunction(Graham et al. 2008 p.6). Therefore, one of the main intentions of testing is to detectdefects so that they can be removed. The removal of defects raises the quality of softwareand prevents customer dissatisfaction, loss of revenue and reputation (Mayers 2004 p. 8).Software development in the medical device domain has grown considerably within thelast 20 years, with an increasing number of medical devices relying on software for theirprincipal functionality (Lee and Sokolsky 2010, Rastgarpour and Shanbehzadeh 2011,Toussaint et al. 2007). The functionality of software embedded in medical devices isincreasingly used for diagnostic or treatment purposes, as it improves the quality of thehealth services (National Cancer Institute 2015, Camara et al. 2015, Monti et al. 2010).The growing use of software increases the requirements for software quality, and hencesoftware testing (McHugh et al. 2013, Leveson 2000). Growing functionality increasesthe complexity of software and leads to the evolution of software development to dealwith this complexity (ISO/IEC 2008, IEC 2015). Software testing becomes more complexas a result of an increasing occurrence of defects, the number of which increases due toincreasing complexity of software systems and evolving models of software developmentprocess. In the medical device domain, software testing contributes to safety and reducesthe risk of threat to humans due to defective software (Knight 2002).A series of reports related to generic software quality issues demonstrate a noticeabletrend of challenged and failed software development projects (The Standish Group 2014).There is also an increasing trend in medical device recalls due to software issues (Dix etal. 2016). A recent review reveals an increasing number of recalls of medical devices andadverse events to the patients due to software failures (SRCL 2018). The main causes ofmedical device software quality issues are inappropriate software development processand ineffective software testing (Vector Software 2016, Dix et al. 2016). Since there arequality issues in software development, including that of medical device software, thenext section reviews the literature to identify the most prevalent software testingchallenges in both generic and medical device domains.3

1.2 Software Testing ChallengesIn this thesis, a literature review identified challenges related to testing increasingly largeand complex software systems. Growing software functionality increases software sizeand complexity, requiring more extensive software testing. (Lee et al. 2006). However,software testing must be conducted within a finite time frame, and for both generic andmedical device software, available resources including time and money do not allow forcomplete testing (Vector Software 2016). Software testing represents 20% of the time andmoney spent on the entire development process for generic software, up to 50% for safetycritical software and still, the effective implementation of software testing is challenging(Hailpern and Santhanam 2002, King 2015). Due to these challenges, the emphasis is onimproving the effectiveness of software testing in detecting as many defects as possible,especially those threatening software functionality (Reid 2013).The need to address software quality has resulted in the requirement for standards todescribe good development practices. Currently, there are a number of standards bodiesthat have published standards with the intent to improve software development (Clarkeand O’Connor 2010). The software development life-cycle (SDLC) standards introducevarious life-cycle stages from software development planning to software release andmaintenance (ISO/IEC 2008 p.13). The risk of human error at the early life-cycle stagessuch as requirements specification or design specification brings another challenge tosoftware testing. Incorrect or incomplete requirements specifications are expensive tocorrect when identified at the software testing phase. If, however, human error could beidentified by the software testing activities at early life-cycle stages, it would make thecorrection 100 times less expensive (Langenfeld et al. 2016, Seth et al. 2014). Efficientimplementation of software testing covering early life-cycle stages is a challenge forsoftware development organisations and software testing related standards play a role inthe improvement of its efficient implementation (Gelperin and Hetzel 1988).Greater emphasis on safety in the medical device domain increases the significance ofsoftware testing with respect to the extent of testing (Lee et al. 2006). Providing highquality and safe software is a high priority for medical device software developmentorganisations and is subject to regulatory requirements (European Union 2017). Thecompliance of software development is best achieved with the use of the standard IEC62304 which is harmonised with medical device regulations (European Commission).The importance of harmonised standards for the compliance of medical device softwaredevelopment is discussed in the following section.4

1.3 Standardisation Organisations & International StandardsStandardisation organisations publish international standards that provide recommendedpractices for various professions in various domains (ISO, IEC, IEEE). Compliance withany standard requires the fulfillment of all processes and other requirements specified byit. If the international standard is harmonised with regulations, for example for theEuropean Union or the United States, the fulfilment of standards requirements isconsidered as compliance with relevant regulations (European Commission). If anorganisation using a harmonised standard has been audited by a notified body designatedby a government authority, and was awarded a mark of compliance with regulations, itsproduct can be placed on the market on which it complies with the harmonised standard.This thesis reviewed international standards related to generic software testing and quality(ISO 2018a) and software development in the medical device domain (ISO 2018b), whichhave the potential to contribute to solving software testing challenges in the medicaldevice domain. According to the review’s findings, information on generic softwaretesting and

MED-V-STEP Medical Device-Verification-Software Test Processes MED-SQA Medical Device-Software Quality Assurance QA Quality Assurance RO Research Objective RQ Research Question RSQ Research Sub-Question SDLC Software Development Life-Cycle SLCPs Software Life-Cycle Processes SQA Software Quality Assurance U.S. United States of America