Historical Mass, Power, Schedule, And Cost Growth

Transcription

Historical Mass, Power, Schedule & Cost Growthfor NASA Instruments & SpacecraftMarc Hayhurst, Robert Bitten, Daniel Judnick, Ingrid Hallgrimson, Megan YoungsThe Aerospace CorporationStephen ShinnNASA Goddard Space Flight CenterPresented at 2016 NASA Cost Symposium23-25 August 2016 2016 The Aerospace Corporation

Agenda Background Instrument Data Overview Instrument Growth Spacecraft Data Overview Spacecraft Growth Comparison to Guidelines/ Recommendations Summary2

Average Percent Growth from Phase B StartHistorical NASA Data Indicates Payload Mass and Cost GrowthSignificantly Greater than Spacecraft Mass & Cost 40%20%0%MassCost11Data Indicated Payload Resource has Greater Uncertainty than SpacecraftNote: 1) As measured from Current Best Estimate, not including reserves* Taken from “Inherent Optimism In Early Conceptual Designs and Its Effect On Cost and Schedule Growth: An Update”,Freaner C., Bitten R., Emmons D., 2010 NASA PM Challenge, Houston, Texas, 9-10 February 20103

Agenda Background Instrument Data Overview Instrument Growth Spacecraft Data Overview Spacecraft Growth Comparison to Guidelines/ Recommendations Summary4

Instrument Growth Introduction Science instruments are typically the most immature part of any NASAmission development As the building of spacecraft become less challenging for a mature industry,NASA’s continual need to push the cutting edge of science requires therevolutionary and evolutionary development of instruments to meet sciencerequirements Because of this challenge, however, instruments run into substantial issuesthat result in significant increases in mass, power, cost and schedule Although previous studies have identified such issues, there are no industrystandard reserve/contingency design and programmatic guidelines forinstruments This study investigates the historical mass, power, cost and schedule growthof NASA science instruments to more fully understand the growth throughouta mission’s lifecycle5

MissionLarge Diversity of MissionsIncluded in Analysis The data set used for the studyrepresents 80 instruments covering 30missions launched since 1999 The missions include instrument datacollected IFTMESSENGERMRODeep ImpactCloudSatSTEREOCALIPSONew oGRAILNuSTARRBSPLDCMIRISMAVEN8 Astrophysics,5 Heliophysics,7 Earth Science, and10 Planetary missions The missions provide a fairly robustrepresentation of different instrumenttypes and science objectives Collected data at primary historicalmilestones KDP-B or Start of Phase B,PDR, CDR and Final Actual at Launch6ScienceTypeEarth ScienceEarth ScienceAstrophysicsEarth ryPlanetaryPlanetaryEarth ScienceHeliophysicsEarth ophysicsHeliophysicsAstrophysicsAstrophysicsEarth ophysicsEarth 66114213

Agenda Background Instrument Data Overview Instrument Growth Spacecraft Data Overview Spacecraft Growth Comparison to Guidelines/ Recommendations Summary7

Instrument Mass Growth by Milestone50%Average Mass Growth (%)46.7%Median Mass Growth (%)50%40%40%30%30%19.8%20%0%Phase B toDeliveryPDR toDelivery18.0%20%9.0%10%40.4%6.9%10%0%CDR toDeliveryPhase B toDeliveryPDR toDeliveryMass growth percentage reduces as design matures8CDR toDelivery

Instrument Power Growth by Milestone80%Median Power Growth (%)Average Power Growth (%)73.9%50%40%60%40%0%30%29.9%PDR toDelivery12.8%10%0%Phase B toDelivery15.6%20%14.3%20%42.3%CDR toDeliveryPhase B toDeliveryPDR toDeliveryPower growth percentage also reduces as design matures;Median growth is substantially different than average growth9CDR toDelivery

Instrument Cost Growth by Milestone80%Median Cost Growth (%)Average Cost Growth (%)75.6%60%80%60%47.2%20%20%0%0%Phase B toDeliveryPDR toDelivery42.1%40%32.3%40%70.7%CDR toDelivery29.8%Phase B toDeliveryPDR toDeliveryCDR toDeliveryCost growth percentage does not reduce as much as design maturesDemonstrated by substantial uncertainty still existing at CDR10

Instrument Schedule Growth by MilestoneMedian Schedule Growth (%)Average Schedule Growth (%)40%40%35.5%30%30%20.0%20%16.8%10%0%0%PDR toDelivery17.3%20%10%Phase B toDelivery26.1%CDR toDeliveryPhase B toDeliveryPDR toDeliverySchedule growth percentage also decreases as design matures1112.6%CDR toDelivery

Agenda Background Instrument Data Overview Instrument Growth Spacecraft Data Overview Spacecraft Growth Comparison to Guidelines/ Recommendations Summary12

Spacecraft Growth Introduction For the past several decades, industry spacecraft developers have been movingtowards standardized product lines that satisfy the needs of multiple customer basesand missions More standardized bus designs appeal to customers for potential savings in cost andschedule, reduced design uncertainty, and also increased reliability from high heritagedesigns Often customer needs require additional modification of the standardized design,especially in the case of NASA and other government agency customers The modification of existing designs or addition of new designs naturally leads togreater overall uncertainty in the design and potential for growth of spacecraftresources over time This study assesses historical mass, power, cost, and schedule growth for multipleNASA spacecraft buses from the last twenty years and compares to industry reserveguidelines to understand where the guidelines may fall short13

Average Percent Growth from Phase B StartSpacecraft Study Builds from Previous Research120%100%PayloadSpacecraft101%50%Median Mass Growth (%)50%40%80%60%Average Mass Growth 8%20%Phase B toDeliveryPDR toDelivery40.4%CDR toDelivery18.0%20%6.9%10%0%Phase B toDeliveryPDR toDeliveryCDR toDeliveryCost2010 research* indicated that payload resourceshad greater uncertainty than spacecraft2014 research** examined instrument growth in depthat the start of Phase B, PDR, and CDR milestones*“Inherent Optimism In Early Conceptual Designs and Its Effect On Cost and Schedule Growth: An Update”, Freaner C., Bitten R., Emmons D., 2010 NASA PMChallenge, Houston, Texas, 9-10 February 2010**“Historical mass, power, schedule, and cost growth for NASA science instruments,” R. Bitten and S. A. Shinn, 2014 IEEE Aerospace Conference, 2014, pp. 1–10.This study*** examines growth of spacecraft buses indepth at the start of Phase B, PDR, and CDR milestonessimilar to what was performed for instrumentsAdditionally, a comparison of NASA in-house and RapidSpacecraft Development Office (RSDO) catalog buses hasbeen performedAnalysis of spacecraft subsystem growth is also presentedCalculated growth for mass, power, cost, or schedule fromeach milestone, PDR for example, is calculated ��� 𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 𝑃𝑃𝑃𝑃𝑃𝑃 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹 ���𝐶𝐶𝐶𝐶𝐶@𝑃𝑃𝑃𝑃𝑃𝑃 CBE@PDR represents the current best estimate without reservesfor the total mass, power, cost, or schedule at PDR (totalcost/schedule, not cost/schedule to go) and final value represents the final total mass, power, cost, or scheduleat delivery/launch***M. R. Hayhurst, D. C. Judnick, R. E. Bitten, I. E. Hallgrimson, S. A. Shinn and M. A. Youngs, "Historical mass, power, schedule, and cost growth for NASA spacecraft," 2016IEEE Aerospace Conference, Big Sky, MT, 2016, pp. 1-17.14

Large Diversity of Missions Included in Analysis Missions used in the study include 47 spacecraft bus developments launched since 1996Missions representNASA sciencethemes 10 Astrophysics 8 Heliophysics 12 Earth Science 16 Planetary MissionScience TypeLaunch YearNEARCassiniTRMMStardustLandsat SSENGERSwiftDeep ImpactMRONew netaryEarth SciencePlanetaryEarth ScienceEarth ScienceEarth ScienceAstrophysicsPlanetaryHeliophysicsEarth ryAstrophysicsPlanetaryPlanetaryPlanetaryEarth ssionScience TypeLaunch Suomi lanetaryAstrophysicsHeliophysicsEarth csEarth SciencePlanetaryPlanetaryEarth SciencePlanetaryAstrophysicsHeliophysicsEarth ScienceAstrophysicsPlanetaryPlanetaryEarth ScienceEarth ScienceEarth 01420142015Missions provide afairly robustrepresentation ofdifferent scienceobjectives thatinfluence bus designCollected data at primary historical milestones KDP-B or Start of Phase B, PDR, CDR and FinalActual at Launch– Not all missions have data available at every milestone so some analyses have fewer than 47 data points– For missions with multiple identical spacecraft, the first build was examined– For landed missions, the cruise stage was considered as the spacecraft bus15

Agenda Background Instrument Data Overview Instrument Growth Spacecraft Data Overview Spacecraft Growth Comparison to Guidelines/ Recommendations Summary16

Spacecraft Mass Growth by MilestoneMedian Mass Growth (%)Average Mass Growth 5%9%0%9%0%Phase B toDeliveryInstrumentGrowth26%48%PDR to Delivery CDR to Delivery20%Phase B toDeliveryInstrument40%Growth9%PDR to Delivery18%CDR to Delivery7%Mass growth shrinks by about 10% on average every milestoneSpacecraft growth is significantly less than instruments at start of Phase B17

Spacecraft Power Growth by MilestoneMedian Power Growth (%)Average Power Growth %10%16%5%10%5%7%0%0%Phase B toDeliveryInstrumentGrowth22%74%Phase B toDeliveryPDR to Delivery CDR to Delivery30%InstrumentGrowth14%42%PDR to Delivery14%4%CDR to Delivery12%Power growth reduces as design matures and is significantly reduced by the CDRSpacecraft power growth lower than instruments at all milestones18

Spacecraft Bus/I&T Cost Growth by MilestoneMedian Cost Growth (%)Average Cost Growth %30%15%37%15%10%10%5%5%0%23%0%Phase B toDeliveryInstrumentGrowth40%76%PDR to Delivery46%CDR to DeliveryPhase B toDeliveryInstrumentGrowth32%71%PDR to Delivery39%CDR to Delivery30%Cost growth does not reduce significantly as substantial uncertainty remains at CDRSpacecraft growth is lower than instruments at start of Phase B19

Spacecraft Bus Schedule Growth by MilestoneMedian Schedule Growth (%)Average Schedule Growth %5%15%5%0%0%Phase B toDeliveryInstrumentGrowth18%36%PDR to Delivery20%CDR to DeliveryPhase B toDeliveryInstrumentGrowth17%27%PDR to Delivery17%CDR to Delivery13%Similar to cost, schedule growth does not reduce as significantly by CDRSpacecraft schedule growth appears in family with instruments20

Spacecraft Bus Subsystem Mass Growth by MilestoneSubsystem Average Mass Growth from Milestone80% se B to DeliveryPDR to Delivery48%CDR to Delivery28% 27%24% 22%20%10% 7%14% 10%-3%3%3%-3%Subsystem Median Mass Growth from Milestone60%Phase B to Delivery48% 47%50%40%30%20%10%PDR to Delivery32%CDR to Delivery31%26%20%22% 22%15%13%7%6%0%4%7%2%-1%7%0%-1% -2% -3%-10%“Interconnected” systems appear to have the highest growth: Thermal, EPDS (Harness), SMS (Brackets/Support Structure)“Box-like” systems appear to have the lowest growth: C&DH, TT&C, ADCS21

Agenda Background Instrument Data Overview Instrument Growth Spacecraft Data Overview Spacecraft Growth Comparison to Guidelines/ Recommendations Summary22

Example Reserve Discussion References “Goddard Space Flight Center Rules for the Design, Development, Verification, and Operation of FlightSystems,” GSFC-STD-1000F, February 2013. “Goddard Space Flight Center Rules for the Design, Development, Verification, and Operation of FlightSystems,” GSFC-STD-1000E, August 2009 GSFC Goddard Procedural Requirement (GPR) 7120.7 “Schedule Margins and Budget Reserves to beused in Planning Flight Projects and in Tracking Their Performance,” May 2008 NASA Mission Design Process, An Engineering Guide to the Conceptual Design, Mission Analysis, andDefinition Phases, The NASA Engineering Management Council, December 22, 1992 JPL Design Principles, Design, Verification/ Validation and Operations Principles for Flight Systems (D17868), Rev. 2, March 3, 2003 ANSI/AIAA Guide for Estimating and Budgeting Weight and Power Contingencies for SpacecraftSystems, AIAA-G-020-1992, April 16, 1992 “Mass Properties Control for Space Systems Draft for Public Review”, AIAA S-120A-2015, 2015. “Mass Properties Control for Space Systems”, AIAA S-120-2006, December 2006 “JSC Cost Estimating Handbook Cost Reserve nes.html.23

Instrument Mass & Power Contingency vs. GrowthMass Contingency GuidelinesSourceHistoricalMedianGrowthNASA “GreenBook” [7]Goddard GoldRules [8]JPL DesignPrinciples [9]AIAAStandard [10]Power Contingency GuidelinesRelative to:Phase ument30%25%10%SourceHistoricalMedianGrowthNASA “GreenBook” [7]Goddard GoldRules [8]JPL DesignPrinciples [9]AIAAStandard [12]Relative to:Phase tSystemInstrumentHistorical Mass & Power growth percentage at Phase B Starttypically higher than guidelines while PDR & CDR are more in line24

Instrument Mass Growth by Milestone100%Percentile Distribution90%Phase B to LRDPDR to LRD80%CDR to LRD70%GSFC KDP-B Guidelines60%@ PDR:72% DistributionGSFC PDR GuidelinesGSFC CDR Guidelines@ Phase B Start:40% Distribution50%40%@ CDR:61% DistributionAverage Phase B to Launch Mass Growth 46.7%30%Average PDR to Launch Mass Growth 19.8%20%Average CDR to Launch Mass Growth 9%10%0%-50%-25%0%25%50%75%Mass Growth from Milestone Current Best Estimate to Launch (i.e. without reserve)Mass growth percentage reduces as design matures25100%

Instrument Cost & Schedule Contingency vs. GrowthCost Contingency GuidelinesPhase BStartSourceRelative to:HistoricalMedianInstrument 71%GrowthNASA “GreenMission35%Book” [7]GSFC GPRMission30%7120.7 [13]JPL DesignMission30%Principles [9]JSC CostFlight35-50%Handbook [14]SystemSchedule Contingency SourceHistoricalMedianGrowthNASA “GreenBook” [7]GSFC GPR7120.7 [13]JPL DesignPrinciples [9]Industry Ruleof ThumbRelative to:Phase ical Cost & Schedule growth percentages are significantly higherthan guidelines at most milestones26

Instrument Cost Growth by Milestone100%90%80%Percentile Distribution70%60%50%40%30%Phase B to LRDPDR to LRDCDR to LRDGSFC KDP-B GuidelinesGSFC PDR GuidelinesGSFC CDR Guidelines@ CDR:45% Distribution@ PDR:32% DistributionAverage Phase B to Launch Cost Growth 75.6%Average PDR to Launch Cost Growth 47.2%20%Average CDR to Launch Cost Growth 32.3%10%0%-75%@ Phase B Start:28% t Growth from Milestone Current Best Estimate to Launch (i.e. without reserve)200%Cost growth percentage also reduces as design matures but is mostly above guidelines27

Spacecraft Mass & Power Contingency vs. GrowthPower Contingency GuidelinesMass Contingency GuidelinesSourceHistoricalMed. GrowthHistorical Avg.GrowthNASA “GreenBook” [6]Goddard GoldRules [7]JPL DesignPrinciples [8]AIAA Standard[9]Relative to:PhaseB lMed. GrowthHistorical Avg.GrowthNASA “GreenBook” [6]Goddard GoldRules [7]JPL DesignPrinciples [8]AIAA Standard[10]Relative to:Phase FlightSystemFlightSystemFlightSystemGuidelines appear mostly adequate compared to historical mass & power growth28

Spacecraft Bus Mass Growth by Milestone@ CDR:77% Distribution@ PDR:64% Distribution@ Phase B Start:48% Distribution29

Spacecraft Cost & Schedule Contingency vs. GrowthCost Contingency GuidelinesSourceHistoricalMed. GrowthHistorical Avg.GrowthNASA “GreenBook” [6]GSFC GPR 7120.7[11]JPL DesignPrinciples [8]JSC Cost Handbook(Within SOTA) [12]JSC Cost Handbook(Beyond SOTA) [12]Relativeto:Schedule Contingency GuidelinesPhase HistoricalMed. GrowthHistorical Avg.GrowthNASA “GreenBook” [6]GSFC GPR7120.7 [11]JPL DesignPrinciples [8]Industry Rule ofThumbRelative to:Phase Mission8%8%8%Historical cost & schedule growth percentages are significantly higher than guidelinesat most milestones30

Spacecraft Bus Cost Growth by Milestone@ CDR:52% Distribution@ PDR:39% Distribution@ Phase B Start:26% Distribution31

Spacecraft Bus Schedule Growth by Milestone@ CDR:38% Distribution@ PDR:30% Distribution@ Phase B Start:26% Distribution32

Instrument Recommendations Data indicates that instrument designs are typically immature at the start ofPhase B There is a need to guard against growth and/or increase the maturity levels ofthe instrument prior to mission Phase B start This may be accomplished by:– Significantly increasing mass, power, cost and schedule reserve beyond currentguidelines– Perform analogous technical comparison of in-family instruments so as to help moreconservatively scope the initial mass, power, cost and schedule resources– Start development of the instrument prior to mission Phase B start so as to increasethe maturity of the instrument before mission development begins33

Maturing Instrument Prior to Mission Phase B Start A potential alternative consideration, developed by the NASA Earth ScienceTechnology Office (ESTO), is to start the instrument development prior to missionstart - entitled an Instrument First, Spacecraft Second (IFSS) approach – whichbrings the instruments to a CDR level of maturity prior to starting a missionIFSS has been identified as an approach to significantly reduce the collateralmission cost growth due to instrument delays and results in more missions beingfunded for less cost when implemented for a portfolio of missionsBased on the historical data from the study, an IFSS approach would reduce therequired reserve levels for instrument development to 10% for mass, 15% forpower, 30% for cost, and 20% for schedule at the start of mission developmentThis is much more manageable and closer to current industry guidelines formission developmentResource@ Instrument CDR10%15%30%20%MassPowerCostSchedule34

Spacecraft Recommendations The historical mass and power growth data collected for spacecraft and spacecraftsubsystems in this analysis and that of our previous work for instruments firstlyindicates that that these items behave differently in terms of growth However, several of the guidelines only specify single overall reserve values withoutrespect to spacecraft or instrument– The growth of different elements might be better controlled if specific tailored guidelines wereimplemented at th

3 Historical NASA Data Indicates Payload Mass and Cost Growth Significantly Greater than Spacecra