Transcription
International Society of Parametric AnalystsParametricEstimating HandbookFourth Edition – April 2008
International Society of Parametric AnalystsParametric Estimating Handbook, Fourth EditionCopyright 2008 (Unlimited Rights Granted to the US Government)ISBN # 0-9720204-7-0.ISPA/SCEA Joint Office527 Maple Avenue East, Suite 301Vienna, VA 22180703-938-5090Fax 703-938-5091www.ispa-cost.org
Table of ContentsList of FiguresPrefaceIntroductionPart OneThe Basics of Parametric AnalysisCHAPTER 1PARAMETRIC ANALYSIS OVERVIEW1.11.21.3Best Practices of the Parametric Analysis Process . 1-11.1.1 The Parametric Model Building Process . 1-31.1.2 Parametric Model Use. 1-13Cost Estimating Relationships . 1-161.2.1 Data Characteristics and Sources. 1-161.2.2 Data Normalization. 1-201.2.3 CER Development . 1-211.2.4 CER Validation. 1-23Complex Models. 1-241.3.1 Comparison of Complex Models and CERs. 1-241.3.2 Complex Tools in the Estimating Process . 1-251.3.3 Development of Complex Tools. 1-261.3.4 How Good Is the Estimate? . 1-33Part TwoThe Application of Parametric AnalysisCHAPTER 2DATA COLLECTION AND ANALYSIS2.12.22.32.4Data Types and Collection. 2-1Data Sources . 2-2Routine Data Normalization Adjustments . 2-52.3.1 Inflation. 2-62.3.2 Non-Recurring and Recurring Costs. 2-62.3.3 Cost Improvement Curve. 2-72.3.4 Production Rate. 2-7Significant Data Normalization Adjustments . 2-82.4.1 Adjustment for Consistent Scope. 2-82.4.2 Adjustment for Anomalies. 2-8International Society of Parametric Analystsi
TABLEOFCONTENTS2.52.62.4.3 Data Adjustment Analysis Example . 2-9Government Evaluation Issues . 2-10Other Considerations . 2-112.6.1 Resources . 2-112.6.2 Information in the Wrong Format. 2-112.6.3 Differences in Definitions of Categories . 2-122.6.4 The Influence of Temporal Factors. 2-122.6.5 Comparability Problems . 2-122.6.6 Database Requirements. 2-12CHAPTER 33.13.23.33.43.53.63.73.8iiCOST ESTIMATING RELATIONSHIPSCER Development . 3-23.1.1 Cost CERs. 3-23.1.2 Overall CER Development Process . 3-33.1.3 Development Database . 3-43.1.4 Testing the Logic of a CER . 3-5CER Development Examples . 3-53.2.1 Developing Simple CERs . 3-63.2.2 Lessons Learned from CER Implementation. 3-12Curve Fitting and OLS Regression Analysis . 3-123.3.1 Graphical Method . 3-133.3.2 OLS Regression Analysis . 3-133.3.3 Assumptions, Limitations, and Caveats of OLS. 3-153.3.4 Multiple Regression . 3-173.3.5 Curvilinear Regression and Cost Improvement Curve Analysis3-18Testing the Significance of the CER. 3-19When to Use a CER . 3-233.5.1 Strengths . 3-233.5.2 Weaknesses . 3-23Examples of CERs in Use. 3-243.6.1 Construction. 3-243.6.2 Electronics. 3-243.6.3 Weapons Procurement . 3-24CERs from the Defense Procurement and Acquisition Policy . 3-253.7.1 Cost Estimating Relationship Definition . 3-253.7.2 Steps for Developing a Cost Estimating Relationship. 3-253.7.3 Identifying Situations for Use. 3-263.7.4 Developing and Using Estimating Factors . 3-273.7.5 Developing and Using Estimating Equations . 3-273.7.6 Identifying Issues and Concerns . 3-28Evaluating CERs. 3-293.8.1 Government Evaluation Criteria . 3-293.8.2 Logical Data Relationships . 3-293.8.3 Credible Data . 3-31International Society of Parametric Analysts
PARAMETRIC ESTIMATING HANDBOOK3.8.4 Strength of CER Relationships . 3-313.8.5 CER Validation. 3-313.8.6 Summary of CER Evaluation. 3-32CHAPTER 44.14.24.34.44.54.6Background . 4-24.1.1 General Definitions . 4-24.1.2 Examples of Proprietary Models . 4-3The Model Development Process . 4-44.2.1 Step 1: Identifying the Parametric Opportunity . 4-64.2.2 Step 2: Preliminary Model Design . 4-64.2.3 Step 3: Information Systems Needs . 4-64.2.4 Step 4: Data Collection and Analysis . 4-74.2.5 Step 5: Model Development . 4-84.2.6 Step 6: Calibration and Validation . 4-104.2.7 Step 7: Estimating System Policies and Procedures . 4-114.2.8 Step 8: Internal Approval Process . 4-124.2.9 Step 9: External Approval Process . 4-124.2.10 Step 10: Model Maintenance . 4-13Evaluation Criteria . 4-154.3.1 Cost Benefit Analysis . 4-164.3.2 Information System Controls . 4-16Summary Example: Jet Engine Cost Model . 4-17Lessons Learned . 4-17Best Practices . 4-18CHAPTER 55.15.25.35.45.55.65.7COMPANY DEVELOPED COMPLEX MODELSCOMPLEX HARDWARE MODELSBackground . 5-1Overview of Hardware Cost Modeling . 5-2The Parametric Cost Modeling Process . 5-65.3.1 Define Objectives . 5-65.3.2 Data Collection and Analysis . 5-75.3.3 Model Calibration . 5-75.3.4 Model Validation . 5-105.3.5 Forward Estimating . 5-11Commercially Available Hardware Models . 5-12Lessons Learned from the Use of Complex Hardware Models . 5-12Best Practices . 5-14Conclusions . 5-14International Society of Parametric Analystsiii
TABLEOFCONTENTSCHAPTER 66.16.26.36.46.56.66.76.8Background . 6-16.1.1 Software and Programming Languages . 6-46.1.2 Software Development Methodologies. 6-46.1.3 Software Support . 6-86.1.4 Software Capability Maturity Model Integration (CMMI)Models. 6-96.1.5 Manager’s Checklist for Validating Software Cost andSchedule Estimates . 6-136.1.6 Software Estimating Techniques . 6-13Overview of Software Parametric COST Models . 6-156.2.1 Function Point Models. 6-156.2.2 Conversion of Function Points to SLOC . 6-186.2.3 Object Points. 6-196.2.4 Use Case Points (UCPs). 6-196.2.5 Normalized Use Cases (NUCs) . 6-20Cost Model Selection. 6-206.3.1 Step 1: Determine User Needs. 6-206.3.2 Step 2: Select Candidate Models . 6-216.3.3 Step 3: Choose the Most Appropriate Model or Models . 6-216.3.4 Step 4: Reevaluate the Choice . 6-22Intelligent Use of Software Models . 6-226.4.1 Input Data. 6-236.4.2 Model Validation . 6-236.4.3 COCOMO II . 6-246.4.4 Example: REVIC – Revised COCOMO Model . 6-246.4.5 Example: Software Experience. 6-25Future Directions of Software Estimating . 6-266.5.1 New Development and Support Concepts . 6-266.5.2 Reuse and COTS Integration . 6-266.5.3 New Cost Models. 6-26Lessons Learned. 6-27Best Practices . 6-28Conclusions. 6-29CHAPTER 77.1ivCOMPLEX SOFTWARE MODELSGOVERNMENT COMPLIANCERegulatory Compliance . 7-17.1.1 The Truth in Negotiations Act (TINA) . 7-27.1.2 FAR, DFARS, and Estimating Systems . 7-37.1.3 Cost Accounting Standards (CAS) . 7-57.1.4 Federal Acquisition Regulations and Sundry Issues . 7-77.1.5 Subcontracts . 7-87.1.6 Earned Value Management System (EVMS) andContractor Cost Data Reports (CCDR) . 7-10International Society of Parametric Analysts
PARAMETRIC ESTIMATING HANDBOOK7.27.37.1.7 Best Practices . 7-10Parametrics and the Government Review Process . 7-117.2.1 General Estimating System Requirements . 7-127.2.2 Parametric Estimating System Requirements . 7-137.2.3 Parametric-Based Proposals . 7-197.2.4 Best Practices . 7-21Technical Evaluation of Parametrics . 7-227.3.1 Authority to Evaluate Parametric Techniques . 7-227.3.2 Cost Modeling Process for CERs . 7-237.3.3 Projecting Beyond the Historical Range . 7-277.3.4 Breaks in Production . 7-297.3.5 Personnel Reassignments and Relearning . 7-297.3.6 Best Practices . 7-30CHAPTER 88.18.28.38.48.5Tailor Applications to Customer Needs. 8-1Considerations When Applying Tools. 8-2When and How to Use the Various Tools and Techniques . 8-3General Applications . 8-38.4.1 Forward Pricing Rate Models . 8-48.4.2 Subcontractor Price or Cost Analysis Using Vendor Data . 8-68.4.3 Cost As an Independent Variable (CAIV) . 8-98.4.4 Risk Analysis . 8-118.4.5 Bid/No Bid Analysis . 8-138.4.6 Conceptual Estimating . 8-138.4.7 Independent Cost estimates (ICEs). 8-148.4.8 Design to Cost (DTC) . 8-158.4.9 Life Cycle Cost Estimates. 8-158.4.10 Budget Planning Analysis. 8-178.4.11 Proposal Evaluation, Red Team Reviews. 8-178.4.12 Should Cost Studies . 8-178.4.13 Estimate at Completion (EACs). 8-188.4.14 Costing by Phase of Contract. 8-188.4.15 Trade Studies . 8-188.4.16 Sensitivity Analysis . 8-198.4.17 Basis of Estimates (BOE’s) . 8-198.4.18 Affordability and Cost Realism . 8-208.4.19 Cost Spreading . 8-22Summary . 8-23CHAPTER 99.1OTHER PARAMETRIC APPLICATIONSINTERNATIONAL USE OF PARAMETRICSFrance. 9-19.1.1 General Applications . 9-19.1.2 Examples. 9-19.1.3 Regulatory Considerations. 9-2International Society of Parametric Analystsv
TABLEOFCONTENTS9.29.3Germany. 9-59.2.1 General Applications . 9-59.2.2 Examples. 9-59.2.3 Regulatory Considerations. 9-5United Kingdom. 9-89.3.1 General Applications . 9-89.3.2 Examples. 9-89.3.3 Regulatory Considerations. 9-8Appendix AModel Builders - Commercial Software DescriptionsAppendix BDetailed Math of Cost Estimating RelationshipsAppendix CFrequently Asked QuestionsAppendix DRelated Web Sites and Supplementary InformationAppendix EParametric Estimating ChecklistsAppendix FMemorandum of Understanding for Parametric ModelsAppendix GParametric Cost Estimating Initiative Closure ReportAppendix HSpace Systems Cost Analysis Group Risk SummaryAppendix ISpace System Cost Analysis Group, Nonrecurring andRecurring Cost Definitions and Cost Accounting GuidelinesAppendix JEstablishing a Parametric Implementation TeamAppendix KPreparation of Subsystem Level DatasheetsGlossaryReferencesHandbook User CommentsviInternational Society of Parametric Analysts
PARAMETRIC ESTIMATING HANDBOOKList of FiguresFigure 1.1Figure 1.2Figure 1.3Figure 1.4Figure 1.5Figure 1.6Flowchart of Cost Model Development by the Practitioner. 1-3More Hypothetical Plot Points . 1-9Poor Choice of Cost Driver . 1-9Non-linearity of X-Y Relationship. 1-12Typical Linear Scatter Plot. 1-22Comparison of Additive and Multiplicative Errors. 1-23Figure 2.1 Sources of Data. 2-3Figure 2.2 Data Collection, Evaluation, and Normalization. 2-4Figure 2.3 Data Normalization Process Flow . 2-5Figure 3.1Figure 3.2Figure 3.3Figure 3.4Figure 3.5Figure 3.6Figure 3.7Figure 3.8Figure 3.9Figure 3.10Figure 3.11CER and Parametric Estimating Techniques . 3-2CER Development Process. 3-4Examples of Simple CERs . 3-6Estimating Relationship Validation Process . 3-8Estimating Relationship Validation Process Activities. 3-9Summary of Estimating Relationship Report Card . 3-10Joint Estimating Relationship Oversight Panel Membership. 3-11CER IPT Lessons Learned . 3-12OLS Regression Analysis. 3-15Descriptions of CER Statistical Indicators . 3-22Cost Driver Survey Example. 3-30Figure 4.1 Example of Model with CERs Matched to Product . 4-4Figure 4.2 Typical Model Development Process – Company DevelopedModels . 4-5Figure 4.3 Example of a Computerized Data Collection Form . 4-8Figure 4.4 The Space Sensor Cost Model Engineering Cost Drivers. 4-9Figure 4.5 Example of Model Validation by Comparing the Model’sEstimate to Another Estimate . 4-11Figure 4.6 Example of Model Documentation Which FacilitatesMaintenance . 4-15International Society of Parametric Analystsvii
TABLEOFCONTENTSFigure 5.1Figure 5.2Figure 5.3Figure 5.4Hardware Model Input and Output Parameters. 5-5Complex Parametric Hardware Model Estimating Process . 5-6Detailed Calibration Process Flow . 5-9Forward Estimating Process . 5-11Figure 6.1Figure 6.2Figure 6.3Figure 6.4Figure 6.5Figure 6.6Figure 6.7Figure 6.8Figure 6.9Figure 6.10Figure 6.11Critical Software Estimating Process Elements . 6-2Software Waterfall Model Development Phases. 6-5Software Hierarchy. 6-6Software Support Categories. 6-9Staged CMMI-SE/SW Maturity Level Descriptions. 6-11Process Areas for Staged CMMI-SE/SW Maturity Levels . 6-12Categories of Software Cost Models. 6-14Traditional Function Point Computations . 6-17SLOC-Per-Function Point Ratios . 6-18The Weighted Factors Approach. 6-21Key Software Metrics. 6-28Figure 7.1Figure 7.2Figure 7.3Figure 7.4Figure 7.5Audit Process Flowchart. 7-12Types of Data Collected . 7-14Sample Inputs for Complex Parametric Models . 7-16Typical CER Development Process . 7-22Different Learning Formulations. 7-29Figure 8.1 General Parametric Applications. 8-3Figure 8.2 Data Collection Details. 8-7Figure 8.3 Projecting from Historical Data. 8-8viiiInternational Society of Parametric Analysts
Preface to the Fourth EditionThis Parametric Estimating Handbook is a comprehensive, “all in one place”guide for Industry and Government acquisition professionals who prepare,evaluate, or negotiate proposals which use parametric estimating techniques or forproject stakeholders who wish to gain a better understanding of the applicationand use of parametric cost estimating. This handbook also serves as a foundationfor companies that want to make more use of parametric tools in developing thebasis of estimate (BOE) for their proposals to the Government or otherwiseexpand their use of parametrics.Fourth Edition UpdatesThis Fourth Edition of the handbook updates the Third Edition (Spring of 2003).The Third Edition was not a comprehensive update. The motivation for the ThirdEdition was a general edit, a refresh for certain technical data, and an update tothe various web sites that reference parametric applications. The Second Edition(Fall of 1999) replaced the First (Fall of 1995) by adding technical informationand new chapters concerning Government regulations, implementation teams, andtechnical evaluations. The Second Edition also incorporated results from theParametric Estimating Reinvention Laboratory.This Fourth Edition is an extension of the third. The Third Edition was primarilya cosmetic and slightly modernized version of the Second Edition. The FourthEdition includes new material and rewrites with the best parts of the Third Editionretained.The focus of this edition is on process and benefits with less emphasis placed ontools and detailed mathematics. Although tools and math are important, thereader/practitioner is encouraged to go to the sources for detailed informationabout understanding, training, use, and other special topics (e.g.,calibration/validation). Although such topics are discussed in the Fourth Edition,they are discussed primarily in the context of process. The goal was to make thisedition more readable and understandable. All in all, we believe this edition to besuperior in both content and quality to all previous editions.International Society of Parametric AnalystsPreface-1
PREFACEAbout the International Society of Parametric Analysts (ISPA)ISPA (www.ispa-cost.org) was founded more than 25 years ago. The firstnational conference was held in 1979. The genesis of ISPA began in 1978 at thethird PRICE Users meeting in San Francisco in April of that year. At thatmeeting, ISPA was founded as an “international” organization. The first ISPA“Town Meeting” was in April, 1979. Three hundred participants frominternational locations attended and it was at this meeting that the ISPA wasorganized.The PRICE User’s Bulletin (PUB) was replaced by the ISPA News (premiered in1981) with Charley Hopkins as Editor. Other ISPA publications include: Parametric World; Membership Directory; Journal of Parametrics; Conference Proceedings; This Parametric Estimating Handbook; Training materials based on the handbook.ISPA has always provided excellent conferences and educational programs for itsmembers. Truly international in scope, the ISPA conferences are held annually,and each leap year are at an international location.In the estimating process, parametricians understand the technical and other costdrivers, thus making the parametrician a valuable member of a proposal team.The goal of ISPA is to continue to support parametricians throughout their careerby stimulating tool development and by encouraging professional contributions.ISPA will continue to be a powerful force within the estimating community in theforeseeable future.AcknowledgementsThe International Society of Parametric Analysts (ISPA), as well as the generalIndustry/Government Estimating Community would like to thank the followingpeople and organizations for their support during the effort to compile this editionof the Parametric Estimating Handbook. We apologize if anyone was overlooked. The ISPA Board of Directors for their overall support and funding. The Content Writing and Review Team consisted of the following people:o David Eck, DCAA;o John C. Smith, DCAA;o Jason Dechoretz, MCR;o Rich Hartley, USAF;Preface-2International Society of Parametric Analysts
PARAMETRIC ESTIMATING HANDBOOKo Jerry McAfee, DCAA;o Dan Ferens, USAF (retired);o Evin Stump, Galorath;o Dan Galorath, Galorath;o Bob Hunt, Galorath;o Neil Albert, MCR;o Hank Apgar, MCR;o Sherry Stukes, NASA JPL;o Dick Basner, Northrop Grumman;o Cheryl Saul, DCMA;o Keith Flaherty, DCMA;o Michelle Auwaerter, DCAA;o Dennis Lowery, USA, AMCOM;o Bruce Fad, PRICE Systems;o George Teologlou, PRICE Systems;o Madeline Ellis, Boeing;o Tony Fettig;o Bill Brundick, WEB Enterprises;o George Barbic, Lockheed Martin;o Steve Book, MCR. Joan Ugljesa (MCR) and Diane Welschmeyer (DCAA) for general editingand publishing support.Parametric Estimating Handbook SourcesElectronic copies of the current edition of the Parametric Estimating Handbookcan be obtained from the following sources: ISPA Web Site atwww.ispa-cost.org SCEA Web Site atwww.sceaonline.net DoD Acquisition Deskbook Web Site at:www.deskbook.dau.mil NASA Web Site onal Society of Parametric AnalystsPreface-3
PREFACE Galorath Incorporatedwww.galorath.comUser CommentsAs in previous editi
International Society of Parametric Analysts Parametric
