WIXLIB

CopyrightbyRobert Henry Bell, Jr.2005

The Dissertation Committee for Robert Henry Bell, Jr. Certifies that this is theapproved version of the following dissertation:Automatic Workload Synthesis for Early Design Studies andPerformance Model ValidationCommittee:Lizy K. John, SupervisorEarl E. Swartzlander, Jr.Douglas C. BurgerAdnan AzizLieven Eeckhout

Automatic Workload Synthesis for Early Design Studies andPerformance Model ValidationbyRobert Henry Bell, Jr., B.A.; M.S.E.E.DissertationPresented to the Faculty of the Graduate School ofThe University of Texas at Austinin Partial Fulfillmentof the Requirementsfor the Degree ofDoctor of PhilosophyThe University of Texas at AustinDecember 2005

DedicationTo my wife, Susan Carol HonnAnd my parents, Robert H. Bell and Joyce W. Bell

AcknowledgementsThis work would not have been possible without the support of many people.I would like to thank my advisor, Dr. Lizy Kurian John, for her advice, support,wisdom, and guidance. Dr. John had a profound influence on both the overall direction ofthis research and the specific content of this dissertation. Her unfailing passion for thesubject matter and sound advice in the face of sometimes difficult issues always pointedin the correct direction. Her research in the complex fields of computer architecture andcomputer performance analysis continues to inspire researchers and developers in bothacademia and industry.I would like to thank my graduate committee for their advice and friendship overthe years. Many thanks are extended to Earl Swartzlander, who co-authored my firstpaper as a graduate student at the University of Texas at Austin; Adnan Aziz, whose classon logic synthesis inspired many analogous thoughts on the automatic synthesis ofworkloads; Lieven Eeckhout, for our collaboration on statistical simulation that launchedthis work, and his many friendly and helpful comments over the years. Special thanks goto Doug Burger, whose charm, wisdom, intellect and complete mastery of computerdesign are truly an inspiration.I would also like to thank the many characters that I interacted with in theLaboratory on Computer Architecture at the University of Texas at Austin, especially Dr.v

Tao Li, Dr. Madhavi Valluri, Dr. Ravi Bhargava, and Dr. Juan Rubio. Their researchideas, their knowledge in many diverse disciplines, willingness to listen and comment inmany talks and discussions, and friendly help were much appreciated.I would like to thank the faculty, staff, and administration of the University ofTexas at Austin for providing the venue and resources for a world-class graduateeducation in computer architecture and performance analysis. Many thanks go to IBM,the IBM Systems and Technology Division, and the Advanced Learning AssistanceProgram at IBM for providing funding for this work. Thanks to Doug Balser, TomAlbers, Sam Thomas of IBM, and a special thanks to Jeff Stuecheli. I would like to thankDr. Ann Marie Maynard of the IBM Austin Center for Advanced Study for her energyand enthusiasm in enhancing the interaction between IBM and academia. Thanks to Dr.James H. Aylor and Dr. Jim Jokl of the University of Virginia, who guided me throughmy M.S.E.E. and inspired continuing studies in engineering.Many thanks go to my family. I would like to thank my father, Robert H. Bell,who inspired my work in the sciences and engineering, and whose ever-cheerfuldemeanor, friendly and helpful company, intellectual curiosity, and unfailing support aregreat comforts. Thanks also to my mother, Joyce W. Bell, whose support, kindness,extensive scientific education, and quest for knowledge are most important to me.Finally, to my dear wife and friend, Susan C. Honn, a special thanks for enduringthe long hours, late nights, and lost weekends that inevitably result from such anendeavor; and for her enthusiasm, loving support, and patience over the years - and inthree states - I will be eternally thankful.Robert H. Bell, Jr.The University of Texas at AustinDecember 2005vi

Automatic Workload Synthesis for Early Design Studies andPerformance Model ValidationPublication No.Robert Henry Bell, Jr., Ph. D.The University of Texas at Austin, 2005Supervisor: Lizy Kurian JohnComputer designers rely on simulation systems to assess the performance of theirdesigns before the design is transferred to silicon and manufactured. Simulators are usedin early design studies to obtain projections of performance and power over a large spaceof potential designs. Modern simulation systems can be four orders of magnitude slowerthan native hardware execution. At the same time, the numbers of applications and theirdynamic instruction counts have expanded dramatically. In addition, simulation systemsneed to be validated against cycle-accurate models to ensure accurate performanceprojections. In prior work, long running applications are used for early design studieswhile hand-coded microbenchmarks are used for performance model validation.One proposed solution for early design studies is statistical simulation, in whichstatistics from the workload characterization of an executing application are used tocreate a synthetic instruction trace that is executed on a fast performance simulator. Inprior work, workload statistics are collected as average behaviors based on instructionvii

types. In the present research, statistics are collected at the granularity of the basic block.This improves the simulation accuracy of individual instructions.The basic block statistics form a statistical flow graph that provides a reducedrepresentation of the application. The synthetic trace generated from a traversal of theflow graph is combined with memory access models, branching models and novelprogram synthesis techniques to automatically create executable code that is useful forperformance model validation. Runtimes for the synthetic versions of the SPEC CPU,STREAM, TPC-C and Java applications are orders of magnitude faster than the runtimesof the original applications with performance and power dissipation correlating to within2.4% and 6.4%, respectively, on average.The synthetic codes are portable to a variety of platforms, permitting validationsbetween diverse models and hardware. Synthetic workload characteristics can easily bemodified to model different or future workloads. The use of statistics abstractsproprietary code, encouraging code sharing between industry and academia. Thesignificantly reduced execution times consolidate the traditionally disparate workloadsused for early design studies and model validation.viii

Table of ContentsTable of Contents. ixList of Tables . xiiiList of Figures .xvChapter 1: Introduction .11.1 Early Design Studies and Application Simulation.31.2 Performance and Power Model Validation.51.3 Simulation Strategies .51.4 The Problems and Proposed Solutions .71.5 Thesis Statement .91.6 Contributions.91.7 Organization.12Chapter 2: Workload Modeling and Statistical Simulation .132.1 Performance Simulation Strategies and Statistical Simulation.132.2 Overview of Statistical Simulation in HLS .182.3 Simulation Results .212.3.1 Experimental Setup and Benchmarks .212.3.2 The HLS Graph Structure .222.3.3 The HLS Processor Model.242.3.4 Issues in the Experimental Setup of HLS .252.3.5 Challenges Modeling the STREAM Loops .282.4 Improving Processor and Workload Modeling in HLS .292.4.1 Improving the Processor Model.292.4.2 Improvements to Workload Modeling.312.4.2.1 Basic Block Modeling Granularity .312.4.2.2 Basic Block Maps .342.4.2.3 Basic Block Maps for Strong Phases .382.5 Implementation Costs .39ix

2.6 Summary .41Chapter 3: Automatic Workload Synthesis.433.1 Introduction to Performance Model Validation.433.2 Synthesis of Representative Workloads.463.3 Synthesis Approach .503.3.1 Workload Characterization .513.3.2 Graph Analysis.523.3.2.1 Instruction Miss Rate and I-cache Model .543.3.2.2 Instruction Dependences and Instruction Compatibility.553.3.2.3 Loop Counters and Program Termination .573.3.2.4 Memory Access Model .573.3.2.5 Branch Predictability Model .603.3.3 Register Assignment .613.3.4 Code Generation .623.4 Evaluation of Synthetic Testcase Performance.643.4.1 Methodology .643.4.2 Evaluation of Synthetic Workload Characteristics .653.4.3 Evaluation of Design Changes.693.4.4 TPC-C Study.753.5 Drawbacks and Discussion .753.6 Early Synthesis and Related Workload Synthesis Research.773.7 Summary .80Chapter 4: Quantifying the Errors in Workload Characteristics Due to theWorkload Synthesis Process .824.1 Introduction to Errors in Synthetic Workloads.824.2 Sources of Error in Workload Synthesis.844.2.1 Sources of Error in Workload Characterization.854.3.2 Sources of Error in Graph Analysis .864.3.2.1 Instruction Miss Rate and I-cache Model .864.3.2.2 Instruction Dependences.87x

4.3.2.3 Loop Counters and Program Termination .884.3.2.4 Memory Access Model .884.3.2.5 Branching Model .894.3.3 Sources of Error in Register Assignment.904.2.4 Sources of Error in Code Generation.914.3 The Flexibility of Statistical Simulation .924.4 Simulation Results .964.4.1 Experimental Setup and Benchmarks .964.4.2 Sensitivities to Changes in Workload Characteristics inStatistical Simulation .974.4.3 Sensitivities to Changes in Workload Characteristics fromTestcase Synthesis .1024.5 Summary .105Chapter 5: Efficient Power Analysis using the Synthetic Workloads .1075.1 Introduction to Power Dissipation Studies .1075.2 Synthetic Testcases and Power Dissipation.1095.3 Power Simulation Results .1115.3.1 Experimental Setup and Benchmarks .1125.3.2 Base Power Dissipation Results.1125.3.3 Analysis of Design Changes .1165.4 Summary .121Chapter 6: Performance Model Validation Case Study for theIBM POWER5 Chip .1226.1 Introduction to the POWER5 Chip .1226.2 IBM PowerPC Synthesis and Model Validation .1246.2.1 The POWER5 M1 Performance Model.1246.2.2 PowerPC Performance Model Validation.1256.2.2.1 Validation using RTL Simulation.1256.2.2.2 Validation using a Hardware Emulator.1286.3 Synthesis for PowerPC .128xi

6.3.1 Workload Characterization .1286.3.2 Graph Analysis.1296.3.2.1 Instruction Miss Rate and I-cache Model .1306.3.2.2 Instruction Dependences and Compatibility.1306.3.2.3 Loop Counters and Program Termination .1326.3.2.4 Memory Access Model .1326.3.2.5 Branch Predictability Model .1376.3.3 Register Assignment .1396.3.4 Code Generation .1396.4 POWER5 Synthesis Results .1406.4.1 Experimental Setup.1416.4.2 Synthesis Results .1416.4.3 Design Change Case Study: Rename Registers and DataPrefetching .1456.5 Performance Model Validation Results .1466.5.1 RTL Validation .1466.5.2 Hardware Emulation .1496.6 Summary .150Chapter 7: Conclusions and Future Work.1517.1 Conclusions.1527.2 Future Work .156Bibliography .159Vita .169xii