Transcription
WP2DIGIT B1 - EP Pilot Project 645Deliverable 11: Design of the Method for Performing the Code Reviews for theEuropean InstitutionsSpecific contract n 226 under Framework Contract n DI/07172 – ABCIIIMay 2016
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutionsAuthor:DisclaimerThe information and views set out in this publication are those of the author(s) and do not necessarilyreflect the official opinion of the Commission. The content, conclusions and recommendations set out inthis publication are elaborated in the specific context of the EU – FOSSA project.The Commission does not guarantee the accuracy of the data included in this study. All representations,warranties, undertakings and guarantees relating to the report are excluded, particularly concerning – butnot limited to – the qualities of the assessed projects and products. Neither the Commission nor any personacting on the Commission’s behalf may be held responsible for the use that may be made of theinformation contained herein. European Union, 2016.Reuse is authorised, without prejudice to the rights of the Commission and of the author(s), provided thatthe source of the publication is acknowledged. The reuse policy of the European Commission isimplemented by a Decision of 12 December 2011.Document elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 2 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutionsContentsCONTENTS. 3LIST OF TABLES . 5LIST OF FIGURES . 6ACRONYMS AND ABBREVIATIONS . 7123INTRODUCTION . 81.1.OBJECTIVE OF THIS DOCUMENT AND INTENDED AUDIENCE . 81.2.SCOPE . 81.3.DOCUMENT STRUCTURE . 91.4.KEY SUCCESS FACTORS . 91.5.DELIVERABLES . 10APPROACH . 112.1.CODE REVIEW TOOLS . 112.2.COMMUNICATION TOOLS . 112.3.OPERATIONAL STAFF . 12METHODOLOGY . 133.1.CODE REVIEW METHODOLOGY . 133.1.1.3.1.1.1.Preparation . 153.1.1.2.Test Design . 173.1.1.3.Environment Configuration. 183.1.2.Execution . 193.1.2.1.Managed Mode . 213.1.2.2.Defined Mode. 223.1.2.3.Optimised Mode . 233.1.3.Assessment . 243.1.3.1.Technical Report Analysis. 263.1.3.2.Impact Analysis . 273.1.3.3.Finding Prioritisation . 303.1.4.3.2.Planning . 14Reporting . 323.1.4.1.Report . 333.1.4.2.Report Dissemination . 343.1.4.3.Post-audit Support . 35PROJECT EFFORT PLANNING. 36Document elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 3 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutions3.3.RESPONSIBILITIES AND ASSIGNATIONS . 373.4.TEST CATEGORIES . 384REFERENCES . 435ANNEXES . 445.1.ANNEX 1: CONTROL CHECKLIST . 445.1.1.Data/Input Management (DIM) . 445.1.2.Authentication Controls (AUT) . 455.1.3.Session Management (SMG) . 455.1.4.Authorisation Management (ATS) . 455.1.5.Cryptography (CPT) . 455.1.6.Error Handling /Information Leakage (EHI) . 455.1.7.Software Communications . 465.1.8.Logging/Auditing (LOG) . 465.1.9.Secure Code Design (SCD) . 465.1.10. Optimised Mode Controls (OPT) . 465.1.11. Specific JAVA Control Checklist (J*) . 475.1.12. Specific PHP Control Checklist (P*) . 475.2.ANNEX 2: FINAL REPORT STRUCTURE . 485.2.1.Detailed Report . 485.2.2.Executive Report . 495.2.3.Communication Results Formatting . 495.3.ANNEX 3: CODE REVIEW PROCEDURE . 505.3.1.Planning Phase. 505.3.2.Execution Phase . 525.3.3.Assessment Phase . 565.3.4.Reporting Phase . 56Document elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 4 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutionsList of TablesTable 1: Code review tools .11Table 2: Communication tools .11Table 3: Participants .12Table 4: Methodology – 1. Planning phase details .15Table 5: Preparation details .16Table 6: Test design details .18Table 7: Environment configuration details .19Table 8: Methodology – 2. Execution phase details .20Table 9: Managed mode details .21Table 10: Defined mode details .22Table 11: Optimised mode details .24Table 12: Methodology – 3. Assessment phase details .25Table 13: Technical report analysis details .26Table 14: Impact analysis details .27Table 15: Threat, Vulnerability and Impact possible values .28Table 16: Global risk evaluation .29Table 17: Finding prioritisation details .30Table 18: Methodology – 4. Reporting phase details .32Table 19: Reporting details .34Table 20: Report dissemination details .35Table 21: Post-audit support details.36Table 22: RACI matrix .37Table 23: Basic CVRF elements .49Document elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 5 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutionsList of FiguresFigure 1: WP2 Tasks .9Figure 2: Methodology phases .13Figure 3: Test category control levels .14Figure 4: Code review execution tasks order .21Figure 5: Detailed control risk results (sample) .29Figure 6: Checklist control risk results (sample) .29Figure 7: Executive report finding indicators .31Figure 8: Priority levels (sample) .32Figure 9: Possible reporting channels .33Figure 10: Project effort planning (sample) .37Figure 11: Working Queues .53Figure 12: Code Review Process.54Figure 13: Structure of the Tables of the Support Document .55Document elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 6 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutionsACRONYMS AND ABBREVIATIONSCVSSCommon Vulnerability Scoring SystemCWSSCommon Weakness Scoring SystemDDoSDistributed Denial of ServiceDGDirectorate GeneralDoSDenial of ServiceEIEuropean InstitutionsEPEuropean ParliamentFOSSFree and Open Source SoftwareFOSSAFree and Open Source Software AuditingOSOperating SystemSDLCSystem Development Life CycleSEOSearch Engine OptimizationWPWork PackageAPIApplication Programming InterfaceDocument elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 7 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutions1 INTRODUCTION1.1. Objective of this Document and Intended AudienceThis document represents the deliverable 11 included within TASK-08: Design of the method for performingthe code reviews for the European institutions.The objective is to design a code review process to be used in the European Institutions, taking intoaccount the results obtained on TASK-06: “Requirement for the code reviews and their validity for theEuropean Institutions” and TASK-07: “Analysis of the methods for communicating the results of codereviews, targeting their automated communication”.The design will be done based on a close collaboration with the European institutions including, but notlimited to, document review and workshop validation.1.2. ScopeTo entirely understand the scope of the document, it is necessary to understand the aim of the WorkPackage (WP) 2. The WP2 has four tasks: Task 6: Requirements for the code reviews that aim to define the list of requirements for propercode reviews and their validity for the European Institutions, as well as to prepare an analysis ofhow they fit into the working methods of the European Commission and the European Parliament. Task 7: Analysis of the methods for communicating the results of the code reviews, targeting theirautomated communication.Tasks 6 and 7 will provide the requirements that the methodology defined in task 8 needs to fulfil.For this reason, deliverables 9 and 10 (output of tasks 6 and 7) are complementary. Task 8: Design of the code review process to be used in the European Institutions, taking intoaccount the requirements defined in tasks 6 and 7. Task 9: Feasibility study of the method defined to perform code review, to be used in the EuropeanInstitutions.This is deliverable 11, the result of task 8, which covers the design and development of a code reviewmethodology to be used to analyse open-source solutions provided by FOSS communities and EuropeanInstitutions.The selection of the tools that will be used for the code review and communication of the results have beenselected on previous tasks (Task-06 and Task-07).Document elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 8 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutionsFigure 1: WP2 TasksTASK-06: Requirement for the codereviews and their validity for theEuropean InstitutionsTASK-07: Analysis of the methods forcommunicating the results of codereviews, targeting their automatedcommunicationTASK-08: Design of the method for performing the code reviews for the EuropeanInstitutionsTASK-9: Feasibility study1.3. Document StructureThis document consists of the following sections: Section 1: Introduction, which describes the objectives of this deliverable, intended audience andScope. Section 2: Approach, which describes the tools selected for the pilot, based on the requirementsestablished during Task-06 and Task-07. Section 3: Methodology, which defines the methodology to be followed on the code reviewprocesses. Section 4: References. Section 5: Annexes.1.4. Key Success FactorsThe following factors are needed to ensure the success of this phase: Selection of an appropriate tool, or set of tools, that adequately cover the requisites set on Task-06and Task-07. Definition of a detailed code review methodology that ensures that all the requisites are coveredand that it can be replicated easily by different teams or users. Definition of a robust validation process to ensure that any selection, methodology step orrequirement is reviewed and approved by the stakeholders in order to ensure that it covers theirneeds.Document elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 9 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutions1.5. Deliverables1Deliverable 9: List of requirements for code reviews2Deliverable 10: List of methods for communicating the results of code reviewsDocument elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 10 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutions2 APPROACHThe approach followed on this methodology is based on the selection of one (or several) of the code reviewtools filtered on Deliverable 9, as well as considering the methods for communicating the results of thecode reviews based on the tools identified on Deliverable 10. Therefore, the tools selected to be used todefine the steps to follow in the methodology will be fully compliant with the needs of the EuropeanInstitutions.2.1. Code Review ToolsThe following tools have been selected to be used as part of the methodology that is defined on thefollowing chapters. This choice has been made taking into account the results shown in Deliverable 9, andis to be exclusively used on this pilot. It is not to be considered as an official nor general selection madeby the European Institutions.Table 1: Code review toolsToolLanguagesCoverageVCGJAVA, PHP77%YascaJAVA, PHP69%RIPSPHP58%FindBugsJAVA54%2.2. Communication ToolsIn order to define the methods to be followed in order to ensure proper communication and disseminationof the results of the code reviews, several additional, specific tools will also be selected and used.These selections will be done taking into consideration the results obtained on Deliverable 10. As with theprevious case, the tools are selected exclusively for their use on this pilot. They are not to beconsidered as an official nor general selection made by the European Institutions.Table 2: Communication toolsToolCharacteristicsJIRAUsed to distribute final report documents.JoinUpUsed to distribute final report documents.JSReportsUsed to distribute the checks.Document elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 11 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutions2.3. Operational StaffIn order to carry out these code reviews, the team that will be in charge will have to cover the followingskills in order to better execute the phases, tasks and activities that have been proposed on themethodology and therefore present better results.There will be several participants involved on a code review, each one having a specific set ofresponsibilities, and their participation being related directly to very specific tasks. The groups consideredare:Table 3: s Define the scope of the code review.Validate final reports.Manage and support the post-audit phase.Code review team Define the tests to be carried out.Configure and validate the environment to carry out the tests.Carry out the execution of the test cases.Generate the technical report including the results.Develop and evaluate the impact analysis of the findings.Provide an initial prioritisation and action plan.Provide post-audit support.IT team Provide high-level detail of the application to audit.Provide the source code (if developed/maintained by them).Developers Provide high-level detail of the application to audit.Provide the source code (if developed/maintained by them).On the other hand, within these participant groups, there will be a set of roles that have been defined andthat have clearly listed their required skills and capabilities: Deep understanding of programming language, scripting language and other technologies used inthe application. Good Knowledge of the latest testing tools. Good understanding of HTTP communication if they are testing a web application. Knowledge of basic vulnerabilities. Good reporting skills. Teamwork, communication and documentation skills.Document elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 12 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutions3 METHODOLOGYThis chapter will contain a detailed definition of the phases to be followed during a code review analysis,the tasks to be carried out, and the procedure steps to achieve the objectives of these activities.It has been divided into four sections: The first section will cover the code review methodology itself, defining all the phases, tasks andactivities that have to be carried out to finish the code review successfully. The second section will detail the test cases (controls) that are defined for each one of thecategories that are to be reviewed during the Execution Phase. The third section will detail the process to follow when a critical vulnerability is found on a thirdparty solution, either being part of the code being reviewed, or being a support solution. The final section describes the reporting process of the methodology in greater detail, explainingthe steps to follow in order to communicate the results to the interested parties and generate theappropriate reports.3.1. Code Review MethodologyThe code review methodology that is developed in this deliverable is summarized on the following structure(see Figure 2), which covers all the phases from the initial definition of the code review process and up tothe post-audit support whenever required. Each one of these phases will be divided into several main tasksthat will summarize the main objectives of each phase, and which allows a better sorting of the activitiesthat have to be carried out by all the participants that will be involved in the project.Figure 2: Methodology ionManaged ModeTechnical ReportReportTest DesignDefined ModeImpact nOptimised ModeFindingPrioritisationPost-auditDocument elaborated in the specific context of the EU – FOSSA project.Reuse or reproduction authorised without prejudice to the Commission’s or the authors’ rightsPage 13 of 57
DIGIT Fossa WP2 – Governance and Quality of Software Code – Auditing of Free and Open SourceSoftware.Deliverable 11: Design of the method for performing the code reviews for the European institutionsFor each one of these phases, a summary table containing the main description of their objectives, tasks,expected results, constraints and dependencies is included. Now for each of the tasks defined for thephases, a second table is defined to include its own specific objectives and expected results, followed by adetailed description of the steps to be carried out by the code reviewers.The estimated workload can be considered “static” for most phases and activities, with the exception of theExecution ones, which will vary depending on the size of the code and the controls to review. This is clearlymentioned in the corresponding activity detail tables.On the execution phase, a set of controls will be verified by the code reviewers in order to properly analysethe code and its security. The Checklist in “Annex 1” contains a detailed list of the controls to evaluatewithin the categories described on this methodology (Section 3.24).Figure 3: Test category control levelsMethodology test categoriesCategory1Detailed checklistControl 1Category2Control 2Language-specific controlsCheck 1Check 2As there are some controls that are language-specific, a third level of detailed controls is defined to includethem within a supplementary checklist, and which will be carried out within the controls included on thedetailed checklists (see Figure 3).3.1.1.PlanningThis stage will cover the information gathering activities that are required to start a code
DIGIT Fossa WP2 - Governance and Quality of Software Code - Auditing of Free and Open Source Software. Deliverable 11: Design of the method for performing the code reviews for the European institutions
