WIXLIB

A Reflection on AnalysisThe antithesis of intellectual humility is intellectual hubris, a quality that may be most harmful in the intelligencefield because of the impact on analysis.it takes 10,000 hours of deliberatepractice to be considered “worldclass” in a subject.a While his rulerefers to performance that transcendsthe expert level and into the elitelevel, it supports the argument thatanalysts should work a portfolio fora longer time period to develop depthand continuity, instead of movingevery few years. This is not to say,however, that analysts should become“hedgehogs”—the term Tetlock usesto describe those who know one bigthing well—rather than “foxes”—Tetlock’s term for nimbler thinkers.bTetlock’s research suggests that foxesare superior forecasters.cIn my 13 years in the IC, I canthink of a handful of analysts I wouldconsider true experts. The most prominent is one who shared the commonanalyst’s experience by studying thetarget country at a university, traveling there, and working the portfoliofor more than a decade, signalinga commitment to a focus area. Hecould recite all of the country’s majorfacets—the size of its army, headsof state tenures, geographic features,and economic imperatives. Thisanalyst’s knowledge, however, wentmuch deeper.a. Malcolm Gladwell, Outliers: The Story ofSuccess (Little, Brown & Company, 2011),35–40.b. Philosopher Isiah Berlin, drawing on anancient Greek poem, originally developedthe fox and hedgehog framework, whichTetlock incorporated into his work. (Superforecasting, 69.)c. Tetlock and Gardner, Superforecasting, 69.How did the analyst cross into theexpert threshold? The analyst’s housewas filled with nearly every bookwritten on his target country, he hadregular dialogue with leading academics who had done field researchin his country, and he had colleagueswho were natives of the country orstill lived there. He also traveledregularly to the country, often for extended periods, and journeyed outsideof the capital where he could meetvarious stakeholders, including nongovernmental organizations, religiousofficials, and citizens outside of elitecircles. He understood how elites inhis target country thought and hisexpertise was in high demand frompolicymakers.True expertise alone does notprevent intellectual hubris and cansometimes foster it. Tetlock, in one ofhis research’s main points, says thatspecialists in a field are often not significantly more reliable than non-specialists in forecasting what is goingto happen in their region of study.dHis work is a reminder of the valueof self reflection for anyone workingin a field like intelligence analysis,no matter what their experience orexpertise.Dr. Kjetil Hatlebrekke and Dr.M. L. R. Smith, in their 2010 articleexamining the human and cognitiveunderpinnings of intelligence shortfalls, allude to the dangers of notpracticing self-awareness. They advocate for “the importance of humilityand responsibility in relation to threatd. Phillip Tetlock, Expert Political Judgement, How Good is It, How Can We Know?(Princeton University press, 2005), 42–43.perception and what the results maybe if such humility and self-criticismare not taken seriously.”e(U) Reason 3: Lack of Humility Can Harm AnalysisThe antithesis of intellectual humility is intellectual hubris, a qualitythat may be most harmful in the intelligence field because of the impact onanalysis. Analysts are often passionate about their work and employ whatis usually a healthy defense to anychanges to it. The pressures analystsface to answer salvos of policymakerdemands may contribute to arroganceamong analysts. Sometimes, however, analysts become entrenched intheir views and become unwilling tohear alternative views.This mentality is commonlymanifest in coordination or peerreview processes in which analystsreceive feedback on articles they havewritten from colleagues who maytake different views. Many analystsinstinctively prepare for the defensiveby crafting counterpoints, which riskdevolution into verbal or e-mail sparring. It is during these moments thatPaul and Eider’s guidance to examinethe logical foundations of our beliefsis particularly relevant. I once heardthat coordination is “free feedback,”which is a mindset that helps advancecollaboration, no matter how challenging the coordination. Psychologist Adam Grant’s “givers and takers”framework encourages us to mirrorthis philosophy when providinge. Kjetil Hatlebrekke and M. L. R. Smith,“Towards a New Theory of IntelligenceFailure? The Impact of Cognitive Closureand Discourse Failure,” Journal of Intelligence and National Security, Vol 25, No. 2(5 July 2010): 147.

A Reflection on Analysisfeedback to a colleague. In discussinggivers, Grant says “a willingness tohelp others achieve their goals is atthe heart of effective collaboration,innovation, quality improvement, andservice excellence.”aIntellectual hubris impedes collaboration in the process of sincerelyaddressing dissent. Paul and Elder’sdefinition of intellectual humilityincludes awareness of the deceptiveness of one’s native egocentrism.This egocentrism is what Tetlock says“prevents us from seeing any worldbeyond the one visible from the tipsof our noses.”bAn analyst’s ego is also anautomatic defense against a dissentbecause of the stigma associated withit. Few analysts welcome disagreement with their theses and insertionof dissent into their work. Striving toalter this mindset into one that seesa dissent as a means to strengthenanalysis would enrich such analysis.Dissent also underscores the complexity of the issues the IC faces, theuncertainty ingrained in them, andpresents policymakers with additionalperspectives.Another important facet in evaluating humility is how we respondto criticism or to instances where weare wrong. There are scores of pagesdevoted to the study of intelligencefailures, which focus on system-leveldeficits. Lowenthal and former CIAofficer Ronald Marks advocate for a“robust ‘lessons learned’ capability”a. Adam Grant, “In the Company of Giversand Takers,” Harvard Business Review,April 2013 at d-takers.b. Tetlock and Gardner, Superforecasting, 233.Another important facet in evaluating humility is how werespond to criticism or to instances where we are wrong.to reflect on such analytic lapses.cThis same mentality is importantat the individual level too. How doanalysts reconcile their own shortfalls, whether missing an assessment or facing disagreement from apolicymaker during a briefing? Amidthe steady production grind, it isimportant to pause to admit one waswrong and reflect on the respectivecauses that Tetlock and Gardner callan “unflinching postmortem.”d Thismay entail revisiting one’s evidenceand critiquing one’s own assessment.It could also involve outreach to akey stakeholder who may have had adivergent view that has since becomemore pertinent. Tetlock and Gardner also suggest that people facingsituations with great uncertainty keepa journal to “create an immutable record” that they can reference.e Theseare tough steps to take, particularlywhen one is on the losing side.Proposals to Cultivate HumilityHow then can the IC encourageand institutionalize humility, andhow can it keep people humble? Thefollowing recommendations seek toachieve these goals: A Recertification process. Anintelligence analysis aptitudeassessment, akin to a physician’sc. Mark Lowenthal and Ronald Marks,“Intelligence Analysis: Is It As Good As ItGets?”: 664.d. Tetlock and Gardner, Superforecasting, 283e. Dan Gardner and Philip Tetlock,“Anxious about the election? Here’s someperspective,” Washington Post, 4 November2016requirement to pass a board certification exam every 10 years, wouldwork toward institutionalizinghumility. The project managementprofessional’s certification processoffers a gold standard model andthe Defense Department’s “Certified DoD All-Source Analysis”program is a step toward this, inthe way it requires DoD analyststo pass a comprehensive knowledge exam.f Continuing education. The IC Advanced Analyst Program (ICAAP)provides a standard of continuingeducation for analysts. The ICshould seek more informal waysto promote continuing educationthrough avenues such as periodic“coffee” or “lunch” talks in whichanalysts give presentations ontheir current research. Double down on analytic outreach. Analytic outreach is amongthe fastest ways to overcome thelimits of our knowledge. The StateDepartment’s Intelligence andResearch Bureau’s outreach eventsserve as a community model.Individual analysts should seekto develop their own outreachnetworks through the appropriateanalytic outreach avenues. A new ODNI analytic standard.The analytic standards and associated tradecraft standards embodyaspects of humility but do notexplicitly address them. Incorporating intellectual humility as ananalytic standard would be amongf. Catherine Johnston et al., “TransformingDefense Analysis,” Joint Forces Quarterly,Issue 79 (Fourth Quarter 2015): 16.

A Reflection on AnalysisAt its core, intelligence analysis is an intellectual—notmechanical—activity.the strongest means to apply theprinciple. Embrace a customer servicementality. Remembering ourraison d’etre and that we supportthe greater good is imperative topromoting humility as a community. We exist, not for self-promotion, but to support policymakingclients. Utilize humor. Humor is the sisterof humility, and self-deprecatinghumor can go a long way towardpromoting humility. Personal ownership. No formal program is a substitute forindividual efforts to incorporatehumility into analysis and devotetime to personal reflection. Humility is a private quality and carriesdifferent meaning to differentpeople.Intelligence analysis pioneerSherman Kent famously said thatif given three wishes, intelligenceprofessionals would desire to knoweverything, to be believed, and to influence policy for the good.a Imbuedin this, however, is intellectual humility and recognition of the limits ofour knowledge and openness to otherviewpoints.At its core, intelligence analysisis an intellectual—not mechanical—activity, as Lowenthal has adeptlydescribed.b This activity is nestedin a business dominated by uncertainty, a paradigm we must acceptwhile constantly pushing for new anda. Sherman Kent, “Estimates and Influence”in Sherman Kent and the Board of NationalEstimates—Collected Essays (Originallyin Studies in Intelligence, Summer 1968)at essays/4estimates.htmlb. Mark Lowenthal, “A Disputation onIntelligence Reform and Analysis: My18 Theses,” International Journal of Intelligence and Counterintelligence, Vol. 26Issue 1 (30 November 2012), 32.vvdeeper knowledge. Intelligence focusareas are intrinsically complex andthe community needs to understandthe limits of its knowledge so as tonot overplay its hand. Doing so willreflect more accurately the servicesthe community provides to clients,such as providing niche value.We also must constantly guardagainst hubris and, in our most confident moments, recall the early daysof our careers, when we sometimesseemed overwhelmed by the scopeof a new portfolio or the magnitudeof our work. All passionate analystsbecome the chief advocates anddefenders of their work, owing tothe intimate study of a topic and theslog of written production. While thismentality fuels our work, we needto pause to remind ourselves that wework in a service industry in whicha service philosophy will advancehumility. Ensuring humility is ingrained in all aspects of intelligenceanalysis will help uphold the intelligence community’s longstanding,quiet professionals, credo, instilled inPresident Kennedy’s words that our“successes are unheralded.”vThe author: John S. Mohr is a DIA analyst who currently works in the NORAD and USNORTHCOM J2. He has servedin DIA in Washington and as a country director on the National Security Council Staff.

forecasting: The Art and Science of Pre-diction (Crown Publishing Group, 2015), 3 and 212-13. Reason 2: Humility Re-veals Subject Matter Experts are Rare Developing expertise is among the IC's primary tools to fight uncer-tainty, but true expertise is rare and takes considerable time and effort to develop. Showcasing expertise often