Transcription
TACKLING BUILD FAILURES IN CONTINUOUS INTEGRATIONbyFOYZUL HASSAN, M.Sc.DISSERTATIONPresented to the Graduate Faculty ofThe University of Texas at San AntonioIn Partial FulfillmentOf the RequirementsFor the Degree ofDOCTOR OF PHILOSOPHY IN COMPUTER SCIENCECOMMITTEE MEMBERS:Xiaoyin Wang, Ph.D., ChairJianwei Niu, Ph.D.Wei Wang, Ph.D.Palden Lama, Ph.D.Lingming Zhang, Ph.D.THE UNIVERSITY OF TEXAS AT SAN ANTONIOCollege of SciencesDepartment of Computer ScienceMay 2020
ProQuest Number: 27957408All rights reservedINFORMATION TO ALL USERSThe quality of this reproduction is dependent on the quality of the copy submitted.In the unlikely event that the author did not send a complete manuscriptand there are missing pages, these will be noted. Also, if material had to be removed,a note will indicate the deletion.ProQuest 27957408Published by ProQuest LLC ( 2020 ). Copyright of the Dissertation is held by the Author.All Rights Reserved.This work is protected against unauthorized copying under Title 17, United States CodeMicroform Edition ProQuest LLC.ProQuest LLC789 East Eisenhower ParkwayP.O. Box 1346Ann Arbor, MI 48106 - 1346
Copyright 2020 Foyzul HassanAll rights reserved.
DEDICATIONTo my parentsTo my wife FarihaTo rest of my family
ACKNOWLEDGEMENTSFirst of all, I would like to thank my adviser, Dr. Xiaoyin Wang. Without his guidance, support, and assistance, it would not be possible for me to finish my dissertation. I very much enjoyedspending time and working with him. Dr. Wang always helped in identifying, formulating, andsolving research problems, and patiently advised the publications and presentations to improve thequality of my research results. Apart from research work, his advises on professional career pathsand preparation for professional carrier have been invaluable. Besides my advisor, I would liketo thank the rest of my dissertation committee: Prof. Jianwei Niu, Prof. Palden Lama, Prof. WeiWang and Prof. Lingming Zhang for their support, insightful comments, constructive feedback,and challenging questions. I am thankful to Prof. Na Meng from Virginia Tech for being a greatcollaborator and imparting her knowledge of fault-localization techniques. I am also grateful to myresearch internship mentors at Microsoft Research Nachi Nagappan, Tom Zimmermann, ChetanBansal and Ahmed Hassan Awadallah for their guidance and fruitful discussion on identifying newresearch problems.I am privileged to be associated with the current software engineering group at UTSA and liketo thank my fellow lab mates: Shaikh Nahid Mostafa, and Xue Qin. Without your great and enjoyable company, it was not possible for me to pass the long journey here.My sincerest gratitude and appreciation go to my lovely wife Fariha Nusrat for constantly supporting me and going through hardships during my PhD years. I would also like to thank myparents, my brother and my sister for their never-ending love, care, and support.Finally, I would like to thank the financial support provided by the Department of Computer Science at UTSA, as well as the funding support from the National Science Foundation(Grant:1464425,1846467).iv
This Masters Thesis/Recital Document or Doctoral Dissertation was produced in accordancewith guidelines which permit the inclusion as part of the Masters Thesis/Recital Document or Doctoral Dissertation the text of an original paper, or papers, submitted for publication. The MastersThesis/Recital Document or Doctoral Dissertation must still conform to all other requirementsexplained in the Guide for the Preparation of a Masters Thesis/Recital Document or Doctoral Dissertation at The University of Texas at San Antonio. It must include a comprehensive abstract, afull introduction and literature review, and a final overall conclusion. Additional material (procedural and design data as well as descriptions of equipment) must be provided in sufficient detail toallow a clear and precise judgment to be made of the importance and originality of the researchreported.It is acceptable for this Masters Thesis/Recital Document or Doctoral Dissertation to includeas chapters authentic copies of papers already published, provided these meet type size, margin,and legibility requirements. In such cases, connecting texts, which provide logical bridges betweendifferent manuscripts, are mandatory. Where the student is not the sole author of a manuscript, thestudent is required to make an explicit statement in the introductory material to that manuscriptdescribing the students contribution to the work and acknowledging the contribution of the otherauthor(s). The signatures of the Supervising Committee which precede all other material in theMasters Thesis/Recital Document or Doctoral Dissertation attest to the accuracy of this statement.May 2020v
TACKLING BUILD FAILURES IN CONTINUOUS INTEGRATIONFoyzul Hassan, Ph.D.The University of Texas at San Antonio, 2020Supervising Professor: Xiaoyin Wang, Ph.D.In popular continuous integration(CI) practice, coding is followed by building, integrationand system testing, pre-release inspection and deploying artifacts. This can reduce integrationtime and speed up the development process. At CI environment, dedicated infrastructure withdifferent build systems such as Make, Ant, Maven, Gradle, Bazel, etc. are used to automate CItasks like compilation, test invocation, etc. For continuous integration, developers describe buildprocess though build scripts such as build.xml for Ant, pom.xml for Maven, and build.gradle forGradle. But with the growing functionality and CI requirement, build scripts can very complexand may require frequent maintenance. Meanwhile, a large number of continuous integrationfailures may interrupt normal software development so that they need to be repaired as soon aspossible. According to the TravisTorrent dataset of open-source software continuous integration,22% of code commits include changes in build script files to maintain build scripts or to resolve CIfailures.CI failures bring both challenges and opportunities to fault localization and program repairtechniques. On the challenge side, unlike traditional fault localization scenarios (e.g., unit testing,regression testing) where only source code needs to be considered, CI failures can also be causedby build configuration errors and environment changes. Actually, the CI practice has make it anecessity for developers to automate the software build process with build scripts and configurationfiles. As a result, the build maintenance needs a substantial amount of developer efforts, anddeveloper’s carelessness may lead to defects in build scripts also.On the opportunity side, the scenario of continuous integration provides rich code commithistory and build logs from previous passing builds and current failing builds. Such informationsources are often not available in other application scenarios. Taking advantage of these additionalvi
information sources, we may be able to further improve the accuracy of automatic repair and faultlocalization for CI failures.Automated program repair techniques have great potential to reduce the cost of resolving software failures, but the existing techniques mostly focus on repairing source code so that they cannotdirectly help resolving software CI failures. To address this limitation, we proposed the first approach to automatic patch generation for build scripts, using fix patterns automatically generatedfrom existing build script fixes and recommending fix patterns based on build log similarity.Apart from build script, CI failures are often a combination of test failures and build failures,and sometimes both source code and various build scripts need to be touched in the repair of onefailure. To address this issue, we proposed a unified technique to localize faults in both source codeand build scripts given CI failures. Adopting the information retrieval (IR) strategy, UniLoc locatesbuggy files by treating source code and build scripts as documents to search, and by consideringbuild logs as search queries. However, instead of naively applying an off-the-shelf IR techniqueto these software artifacts, for more accurate fault localization, UniLoc (Unified Fault Localization) applies various domain-specific heuristics to optimize the search queries, search space, andranking formulas. However, UniLoc can localize faults up to file level and limited to find faultsin source code and build scripts. In the future, we are planning to expand our fault localizationapproach to the source code and build script block level to assist developers and automatic repairapproaches better. Finally, beyond source code and build scripts, there are also other types of filesto be involved during software repair, especially in other scenarios. For example, in the fault localization of web applications, we need to consider Html files, css files, client-side JavaScript files andserver side source code. We plan to adapt our technique to more scenarios with heterogeneous buglocations. Proposed CI fault localization technique with program repair can be a handy solution tofix a range of CI failures.vii
TABLE OF CONTENTSAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ivAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .viList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvChapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.1Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.2Thesis Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41.3Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41.4Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6Chapter 2: Background and Related Work . . . . . . . . . . . . . . . . . . . . . . . . .72.1Analysis of Build Configuration Files . . . . . . . . . . . . . . . . . . . . . . . .72.1.1Automatic Program Repair . . . . . . . . . . . . . . . . . . . . . . . . . .82.2Automatic Fault Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . .92.3Containerized Software Development . . . . . . . . . . . . . . . . . . . . . . . . 10Chapter 3: Automatic Building of Java Projects in Software Repositories: A Study onFeasibility and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.1Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.2Study Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.2.1Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.2.2Study Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3Study on Successfulness of Default Build Commands (RQ1) . . . . . . . . . . . . 163.4A Taxonomy of Root Causes of Build Failures (RQ2) . . . . . . . . . . . . . . . . 17viii
3.4.1Environment Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.4.2Process Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.4.3Project Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.5Identifying Root Causes of Build Failures (RQ3) . . . . . . . . . . . . . . . . . . 243.6Automatic Resolution of Build Failures (RQ4) . . . . . . . . . . . . . . . . . . . 253.73.6.1Build Command Extraction and Prediction . . . . . . . . . . . . . . . . . 263.6.2Version Reverting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.6.3Dummy File Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.6.4Other Types of Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.7.1Threats to Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.7.2Lessons Learned for Automatic Software Building . . . . . . . . . . . . . 303.7.3Lessons Learned for Build-Tool Developers . . . . . . . . . . . . . . . . . 323.7.4Lessons Learned for Project Developers . . . . . . . . . . . . . . . . . . . 323.7.5YML Files and Continuous Integration . . . . . . . . . . . . . . . . . . . 333.8Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.9Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Chapter 4: Change-Aware Build Prediction Model for Stall Avoidance in ContinuousIntegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.1Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.2Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.3DATASET AND CHARACTERISTICS . . . . . . . . . . . . . . . . . . . . . . . 394.44.3.1How Much Time Required for Building? . . . . . . . . . . . . . . . . . . 394.3.2What is the Time Interval in Between Two Commit? . . . . . . . . . . . . 404.3.3How Often Consecutive Build Status Changes? . . . . . . . . . . . . . . . 40Overview of Build Prediction Model . . . . . . . . . . . . . . . . . . . . . . . . . 414.4.1Data Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41ix
4.4.2Feature Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424.4.3Model Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444.5Evaluation and Result Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464.6Threats To Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.7Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50Chapter 5: HireBuild: An Automatic Approach to History-Driven Repair of Build Scripts 515.1Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.2Motivating Example5.3Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555.3.1Gradle Build Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565.3.2Log Similarity Calculation to Find Similar Fixes . . . . . . . . . . . . . . 575.3.3Generation of Build-Fix Patterns . . . . . . . . . . . . . . . . . . . . . . . 595.3.4Generation and Validation of Concrete Patches . . . . . . . . . . . . . . . 63Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.4.1Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.4.2Experiment Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695.4.3Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695.4.4Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695.4.5Threats of Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745.5Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755.6Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765.75.6.1Automatic Program Repair . . . . . . . . . . . . . . . . . . . . . . . . . . 765.6.2Analysis of Build Configuration Files . . . . . . . . . . . . . . . . . . . . 77Conclusion And Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78Chapter 6: UniLoc: Unified Fault Localization of Continuous Integration Failures . . . 806.1Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80x
6.2Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836.2.1Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836.2.2Information Retrieval (IR) . . . . . . . . . . . . . . . . . . . . . . . . . . 846.2.3Project Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 846.3Motivating Example6.4Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 866.5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 856.4.1Query Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 876.4.2Search Space Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 896.4.3Similarity Score-Based File Ranking6.4.4Ranking Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92. . . . . . . . . . . . . . . . . . . . 91Experiments and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 936.5.1Datatset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 936.5.2Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 966.5.3Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 976.5.4Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 986.6Threats to Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1036.7Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1046.8Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1056.96.8.1Automatic Bug Localization . . . . . . . . . . . . . . . . . . . . . . . . . 1056.8.2Fault Localization Supporting Automatic Program Repair . . . . . . . . . 1066.8.3Build Script Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107Chapter 7: RUDSEA: Recommending Updates of Dockerfiles via Software EnvironmentAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1097.1Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1097.2Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1117.3Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112xi
7.47.3.1Extracting Environment-related Code Scope . . . . . . . . . . . . . . . . . 1137.3.2Dockerfile Change Generation . . . . . . . . . . . . . . . . . . . . . . . . 1167.3.3Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1197.4.1Dataset of Dockerfiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1207.4.2Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1217.4.3Evaluation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1217.4.4Threats to Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1237.5Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1237.6Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124Chapter 8: Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125Chapter 9: FUTURE DIRECTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1279.1System Event-Driven Fault Localization of Test and Build Failure in CI Environment1279.2Repair of Build Script and Source Code in Combination CI Failures . . . . . . . . 127List of Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131Vitaxii
LIST OF TABLES3.1Overall Result of Executing Default Build Commands . . . . . . . . . . . . 173.2Root Cause Revealed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.3Resolved Build Failures with Command Extraction and Prediction . . . . . 293.4Resolved Build Failures with Version Reverting . . . . . . . . . . . . . . . 294.1Features Used for Build Prediction Model . . . . . . . . . . . . . . . . . . 454.2Generated Feature Description . . . . . . . . . . . . . . . . . . . . . . . . 454.3Performance Evaluation of Build Prediction Model . . . . . . . . . . . . . 464.4Confusion Matrix of Build Prediction Model . . . . . . . . . . . . . . . . . 474.5Cross Project Performance Evaluation of Build Prediction Model . . . . . . 474.6Confusion Matrix of Cross Project Build Prediction Model . . . . . . . . . 484.7InfoGainAttributeEval Entropy for Ant, Maven, Gradle and Average forTop Ten Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485.1Dataset Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695.2Project-wise Build Failure / Fix List . . . . . . . . . . . . . . . . . . . . . 705.3Cause of unsuccessful patch generation . . . . . . . . . . . . . . . . . . . 736.1Top Software Entities Source Code . . . . . . . . . . . . . . . . . . . . . . 916.2Top Software Entities in Build Scripts . . . . . . . . . . . . . . . . . . . . 916.3Dataset Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 946.4Failure Types and Bug Locations . . . . . . . . . . . . . . . . . . . . . . . 956.5Effectiveness Comparison between Change History Based Approach andUniLoc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.6Effectiveness Comparison between variant approaches, Baselines, and UniLoc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1016.7Effectiveness Evaluation for Different Fix Types . . . . . . . . . . . . . . . 102xiii
6.8Effectiveness for Different Failure Types . . . . . . . . . . . . . . . . . . . 1037.1Effectiveness of RUDSEA on recommending update locations . . . . . . . 1217.2Effectiveness of RUDSEA on recommending updates . . . . . . . . . . . . 122xiv
LIST OF FIGURES3.1Build Failure Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.2Non-Default Build Commands Distribution . . . . . . . . . . . . . . . . . 224.1Ant, Maven and Gradle Build Execution Time Statistics . . . . . . . . . . . 394.2Commit Frequency Time Distribution . . . . . . . . . . . . . . . . . . . . 404.3Build Status Statistics on Consecutive Build Sequence . . . . . . . . . . . 414.4Overview of Build Prediction Model . . . . . . . . . . . . . . . . . . . . . 425.1Hierarchies of Build-Fix Patterns . . . . . . . . . . . . . . . . . . . . . . . 625.2Merged Hierarchies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625.3Breakdown of Build Fixes . . . . . . . . . . . . . . . . . . . . . . . . . . 715.4Patch List Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715.5Amount of Time Required for Build Script Fix . . . . . . . . . . . . . . . 725.6Actual Fix Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 736.1Three *.gradle files used in spockframework/spock declare sub-projects andspecify dependencies between the projects. . . . . . . . . . . . . . . . . 856.2UniLoc Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 856.3Sub-Project Dependency Graph . . . . . . . . . . . . . . . . . . . . . . . 906.4Top-N Comparison between Baselines and UniLoc . . . . . . . . . . . . . 986.5MRR and MAP Comparisons between Baselines and UniLoc6.6MRR for Different Parameter Value on Tuning Dataset . . . . . . . . . . . 1007.1RUDSEA Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111xv. . . . . . . 99
CHAPTER 1: INTRODUCTION1.1MotivationDue to the ever-increasing nature of software requirements and rigorous release practices, Continuous Integration (CI) [80] is gaining more and more popularity. In CI environment, developersmerge code in a codebase, followed by automatic software build process execution, test executionand artifact deployment. This practice allows developers to detect more software faults earlier andalso request developers to resolve the detected fault in a timely manner to make the release pipelineflawless. A study on Google continuous integration [196] finds that more than 5,500 code commitsare merged to the codebase and 100 million test cases are executed per day for validation.At CI environment, dedicated infrastructure with different build systems such as Make, Ant,Maven, Gradle, Bazel, etc. are used to automate CI tasks like compilation, test invocation, etc. Forcontinuous integration, developers describe build process though build scripts such as build.xmlfor Ant, pom.xml for Maven, and build.gradle for Gradle. But with the growing functionality andCI requirement, build scripts can very complex and may require frequent maintenance [140]. Ourstudy [99] on TravisTorrent dataset [62] finds that around 29% of CI trials fail. Seo et al. [178] alsoaddressed a similar issue with 37% build failure proportion at Google CI environment. Rausch etal. [172] find that dependency resolution, compilation, and configuration phases account for 22%of CI failures. On the other hand, 65% failures are categorized as test failures and 13% as qualitychecking errors. Our study on 1,187 CI failures on TravisTorrent Dataset also confirms that 10.8%of CI-failure fix contains only build script revisions, and 25.6% CI-failure contains both build scriptand source code.Apart from CI failure issues, the integration process might be delayed for long build chains,build error fixes, and frequent code changes in version control system. According to the analysis onTravisTorrent [62] data, the median build time for Java project is over 900 seconds and the medianlength of continuous build failure sequences is 4. Thus when multiple developers are committingtheir changes concurrently, their commits may be piled up in the building queue, and they may1
need to wait for a long time to get the build feedback. They also have the option to continueworking on their copy, but they will be at the risk of rolling back their changes if their originalcommit fails the integration.CI failures bring both challenges and opportunities to program repair and fault localizationtechniques. On the challenge side, unlike traditional fault localization scenarios (e.g., unit testing,regression testing) where only source code needs to be considered, CI failures can also be causedby build configuration errors and environment changes. Actually, the CI practice has made it anecessity for developers to automate the software build process with build scripts and configurationfiles. As a result, the build maintenance needs a substantial amount of developer efforts, anddevelopers’ carelessness may lead to defects in build scripts also. This has been confirmed byvarious empirical studies. The study by Kumfert et al. [113] found that build maintenance mayimpose 12% to 36% overhead on software development, and the study by McIntosh et al. [140]shows that, on average, build scripts account for 9% of all maintained files, and 20% of codecommits involve build scripts. Furthermore, although it is possible to differentiate test failures andbuild failures, it is often not possible to ascribe the failure to source code or build scripts just basedon the failure type. On the opportunity side, the scenario of continuous integration provides richcode commit history and build logs from previous passing builds and current failing builds. Suchinformation sources are often not available in other application scenarios of program repair andfault localization. Taking advantage of these additional information sources, we may be able tofurther improve the accuracy of program repair and fault localization for CI build failures. Ourproposed method focuses on fault localization and program in a CI environment.Apart from build repair and fault localization, it is desirable to have a recommendation systemthat predicts the build feedback of a code commit and thus gives developers more confidenceto continue their work and reduce the chance of rolling back. With our proposed approach, weanalyzed software build execution time, commit and consecutive build status change to study thenecessity and possibility of a change-aware build prediction model. Furthermore, we propose arecommendation system to predict build outcome based on the TravisTorrent data and also the2
code change information in the code commit such as import statement changes, method signaturechanges to train the build prediction model.For faster deployment and integration, OS-level container such as Docker is heavily used forcontinuous deployment (CD). A container image is a stand-alone and executable package of a pieceof software with all its environment dependencies, including code, runtime, system tools, libraries,file structures, settings, etc. Despite the large benefit brought by container images during softwaredeployment, they also increase the effort of software developers because they need to generate andmaintain the image configuration files which describe how the container images can be constructedwith all environment dependencies, such as what tools and libraries should be installed and howthe file structure should be set up. Our work on Dockerfile can help developers to update containerimage configuration files more easily and with more confidence. We use novel string analysisfor DockerFile and source code and performed dependency analysis for environment dependencymapping. Finally, to generate DockerFile update suggestion, we used Change Impact Analysisto provide a suggestion based on code changes. Our evaluation on 1,199 real-world instructionupdates shows that RUDSEA can recommend correct update locations for 78.5% of the updates,and correct code changes for 44.1% of the updates. In the future, we planned to utilize system traceinformation such as Ptrace, Strace, etc. to our earlier developed fault localization research workand develop a complete tool suite to repair build failures involving heterogeneous failure
process though build scripts such as build.xml for Ant, pom.xml for Maven, and build.gradle for Gradle. But with the growing functionality and CI requirement, build scripts can very complex . developer's carelessness may lead to defects in build scripts also. . proach to automatic patch generation for build scripts, using fix patterns .
