Transcription
Performance and ConcurrencyBug Detection Tools forJava ProgramsShan LuUniversity of Chicago1
Fighting software bugs is crucial Software is everywhere– http://en.wikipedia.org/wiki/List of software bugs Software bugs are widespread and costly– Lead to 40% system down time [Blueprints 2000]– Cost 312 Billion lost per year [Cambridge 2013]2
Fighting bugs in cloud systems
is crucial
Different aspects of fighting bugsIn-housebug detectionIn-fieldfailure recoveryIn-fieldfailure diagnosisIn-housebug fixing5
Work from my group (local systems)In-housebug detectionconcurrencybugsIn-fieldfailure recoveryIn-fieldfailure diagnosisIn-housebug SPLOS11]; V13];[ICSE15]performancebugs6
Work from my group (local systems)In-housebug detectionconcurrencybugsIn-fieldfailure recoveryIn-fieldfailure diagnosisIn-housebug SPLOS11]; V13];[ICSE15]performancebugs7
Work from my group (cloud systems)In-housebug detectionIn-fieldfailure recoveryIn-fieldfailure diagnosisIn-housebug ugsperformancebugs[CIKM’17];On-going8
Our bug-tools for Java programsdistributedconcurrencybugsEmpiricalbug studiesIn-housebug eldfailure diagnosisIn-housebug fixing[OOPSLA14];[ICSE17b][ICSE15]performancebugs9
Empirical Bug StudiesUnderstanding and detecting real-world performance bugs [PLDI '12]10TaxDC: A Comprehensive Taxonomy of Non-Deterministic Concurrency Bugs in Cloud Distributed Systems [ASPLOS '16]
Performance bug studies Why did we do this study?The most cited paper in PLDI 2012Understanding and detecting real-world performance bugs [PLDI '12]11
Benchmark SuiteApplication Software TypeLanguageBug DBMLOC HistoryTags# BugsApacheCommand-line Utility Server LibraryC/Java0.45 13 yN/A25ChromeGUI ApplicationC/C 14.0N/A10GCCCompilerC/C 5.713 y time-hogCompile-10MozillaGUI ApplicationC /JS4.714 yperf36MySQLServer SoftwareC/C /C# 1.310 yS5284yUnderstanding and detecting real-world performance bugs [PLDI '12]12
What/How did we study? Read on-line discussion Check patches Check source codeAll Manual Bug root causes Bug locations Bug triggeringconditions Bug fix strategies Bug symptoms Bug-related inputsUnderstanding and detecting real-world performance bugs [PLDI '12]13
What could have been better? Read on-line discussion Check patches Check source codeAll Manual Bug root causes Bug locations Bug triggeringconditions Bug fix strategies Bug symptoms Bug-related inputsUnderstanding and detecting real-world performance bugs [PLDI '12]14
Distributed concurrency bug studies Why did we do this study?TaxDC: A Comprehensive Taxonomy of Non-Deterministic Concurrency Bugs in Cloud Distributed Systems [ASPLOS '16]15
Benchmark SuiteBug DBMLOC HistoryApplication Software TypeCassandra Distributed Key-Value StoreJava 0.06HadoopDistributed computingJavaHBaseDistributed Key-Value StoreZookeeper Distributed Synch. Service# Bugs9y191.212 y36Java0.210 y30Java0.110 y19TaxDC: A Comprehensive Taxonomy of Non-Deterministic Concurrency Bugs in Cloud Distributed Systems [ASPLOS '16]16
What/How did we study? Read on-line discussionCheck patchesCheck source codeRead software tutorial Triggering conditions Errors & Failures Fix strategiesAll Manual17TaxDC: A Comprehensive Taxonomy of Non-Deterministic Concurrency Bugs in Cloud Distributed Systems [ASPLOS '16]
What could have been better? Read on-line discussionCheck patchesCheck source codeRead software tutorial Triggering conditions Errors & Failures Fix strategiesAll Manual18TaxDC: A Comprehensive Taxonomy of Non-Deterministic Concurrency Bugs in Cloud Distributed Systems [ASPLOS '16]
Bug Detection & FixingDCatch: Automatically Detecting Distributed Concurrency Bugs in Cloud Systems [ASPLOS'17]Toddler: Detecting Performance Problems via Similar Memory-Access Patterns [ICSE '13]CARAMEL: Detecting and Fixing Performance Problems That Have Non-Intrusive Fixes [ICSE'15]19
Dynamic DCbug Detection DCbugsBugs caused by improper timing among distributed operations Why? Why not applying single-machine detectors?– Distributed triggering– Different happens-before model– Distributed error propagation– Much larger scalesDCatch: Automatically Detecting Distributed Concurrency Bugs in Cloud Systems [ASPLOS'17]20
DCatch ToolTraceHBTriageTriggerDCatch: Automatically Detecting Distributed Concurrency Bugs in Cloud Systems [ASPLOS'17] 21
DCatch Tool ImplementationTraceJavassistHBWALATriageTrigger JavassistDCatch: Automatically Detecting Distributed Concurrency Bugs in Cloud Systems [ASPLOS'17]22
Benchmark SuiteApplication Software TypeWorkloadCassandra Distributed Key-Value Storestartup1HadoopDistributed Computingwordcount2HBaseDistributed Key-Value Storeenable, split, alter2startup2Zookeeper Distributed Synch. Service# BugsDCatch: Automatically Detecting Distributed Concurrency Bugs in Cloud Systems [ASPLOS'17]23
Dynamic PerfBug Detection Loop inefficiency bugs– Inefficient data structure– Redundant computation Why?Toddler: Detecting Performance Problems via Similar Memory-Access Patterns [ICSE '13]24
Toddler ToolTraceAccess-Value AnalysisToddler: Detecting Performance Problems via Similar Memory-Access Patterns [ICSE '13] 25
Toddler ToolTraceAccess-Value Analysis SootToddler: Detecting Performance Problems via Similar Memory-Access Patterns [ICSE '13]26
Benchmark SuiteApplication Software TypeAntKLOCKnownBugsNewBugsJava11018Apache Col. Collections libraryJava51120GroovyDynamic languageJava13710Ggl Core Lib Collections libraryJava156210JFreeChart Chart frameworkJMeterLoad testing 178373211000Build toolText search enginePDF frameworkSearch serverHow to get inputs?Toddler: Detecting Performance Problems via Similar Memory-Access Patterns [ICSE '13]27
Static PerfBug Detection & Fixing Missing-break bugsHow IsComputationWasted?Where Is Computation No-ResultType 1Type 2Type YUseless-ResultType XType 3Type 4 Why?CARAMEL: Detecting and Fixing Performance Problems That Have Non-Intrusive Fixes [ICSE'15]28
Caramel FixSuggestionCARAMEL: Detecting and Fixing Performance Problems That Have Non-Intrusive Fixes [ICSE'15] 29
Caramel Tool .WALASideEffectDataFl.FixSuggestion WALACARAMEL: Detecting and Fixing Performance Problems That Have Non-Intrusive Fixes [ICSE'15]30
Benchmark Suite (1)Application Software TypeAntKLOCNewBugsJava1101Apache Col. Collections libraryJava5120GroovyDynamic languageJava1379Ggl Core Lib Collections libraryJava15610JFreeChart Chart frameworkJMeterLoad testing 837314102Build toolText search enginePDF frameworkSearch serverCARAMEL: Detecting and Fixing Performance Problems That Have Non-Intrusive Fixes [ICSE'15]31
Benchmark Suite (2)Application Software TypeKLOCNewBugsLog4JLogging frameworkJava526SlingWeb app. frameworkJava2026StrutsWeb app. frameworkJava1754TikaContent extractionJava501TomcatWeb serverJava2954No input requirementsCARAMEL: Detecting and Fixing Performance Problems That Have Non-Intrusive Fixes [ICSE'15]32
Failure DiagnosisPerformance Diagnosis for Inefficient Loops [ICSE'17]33
Conclusion 2 bug studies, 3 bug detection tools 18 Java benchmark software (4 distributed) Workloads– Functionality & Performance Analysis mechanisms– Human– WALA, Soot, Javassist Others– Bug-tracking systems34
Thanks!Shan LuUniversity of Chicagoshanlu@uchicago.edu35
Our bug-tools for Java programs 9 [PLDI12] performance bugs In-field failure diagnosis [OOPSLA14]; [ICSE17b] In-house bug fixing [ICSE15] [ASPLOS16] In-house bug detection [ICSE13] [ASPLOS17] . Build tool Java Java Java Java KLOC Known Bugs 137 156 51 110 2 1 1 1 New Bugs 8 20 0 10 JFreeChart JMeter Lucene PDFBox PDF framework Load testing .
