2021 REN-ISAC Blended Threat Workshop Series

Transcription

2021 REN-ISAC BlendedThreat Workshop SeriesFinal Findings ReportOctober 27, 2021This document is marked TLP:WHITE. TLP:WHITE information may be distributed withoutrestriction.

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop SeriesFOREWORDOn behalf of the Research and Education Networks Information Sharing and Analysis Center(REN-ISAC), thank you for taking the time to read this report. More importantly, thanks toeveryone who contributed to the success of the 2021 REN-ISAC Blended Threat ResilienceWorkshop Series, including our hosts, our planning teams, our REN-ISAC staff, our colleaguesat Gate 15, and our participants.The 2021 Blended Threat Workshop Final Findings Report is full of actionable information thatyou can apply at your organization. The report is also evidence of how the REN-ISACcommunity can continue to learn from each other and how to turn our collectively generatedideas into innovative ways to protect, defend, and respond efficiently and effectively.This year’s workshop series was different from previous years. As with all other aspects of ourlives, the COVID-19 pandemic impacted the workshops. We took a long pause during the earlyplanning to confirm the educational necessity the workshops provide and to evaluate, with ourGate 15 partners, virtual delivery options. The virtual offerings allowed us to blast out thetraditional borders of space and region, making the workshops available to many moreparticipants.We decided on the topic of ransomware early and never varied from it. By the time we startedplanning, it was clear that ransomware was a beast (and not in a good way) that wasn’t goingaway. That certainly played out in 2021 as we saw institution after institution victimized. Somenews media are now declaring ransomware attacks as inevitable. Discussions on how to lessenthe impact and what to do during a ransomware attack are necessary. I was inspired to see theworkshop participants take full advantage of the workshop opportunity to ask hard questions andlearn from each other. Allowing individuals and organizations to deeply analyze and discuss asignificant, stress-inducing—sometimes even catastrophic—ransomware event helped toprioritize guiding principles, articulate processes, and even develop preventative controls.As we continue to offer education and workshops to and with the REN-ISAC community, wetake the information articulated in this report as opportunities for change, inspiration foradditional offerings, and suggestions for improving services. The REN-ISAC staff (includingme) get so motivated when we hear participants say “Couldn’t the REN-ISAC ” or “What ifthe REN-ISAC ” during workshops and educational events. It inspires us to try new things andto keep our service mission close to our hearts.Best regards,Kim Milford, Executive DirectorREN-ISACForeword/Executive Summary1TLP:WHITEREN-ISAC

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop SeriesExecutive SummaryThis Final Findings Report compiles observations from all seven workshops of the multi-national2021 REN-ISAC Blended Threat Workshop Series. Representatives from approximately 322higher education institutions and 39 other organizations attended the workshop.We would like to thank the institutions that hosted individual workshops. The following reportprovides an overview of the series’ discussion. The scenario was developed over the course offour modules, each providing situation updates that led to a facilitated discussion. This reportpresents participants’ observations from these discussions as Best Practices, Areas ofImprovement, and Challenges. All participant comments have been anonymized.Appendices provide the 2021 workshop schedule, acronyms, the full text of the scenario’smodules and questions, Core Capabilities linked to each observation, and references andresources.Foreword/Executive Summary2TLP:WHITEREN-ISAC

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop SeriesHostsREN-ISAC would like to thank the following organizations that generously volunteered to hostthe 2021 Blended Threat Workshop Series. REN-ISAC would like to especially acknowledgeJon Garvin, Jill Kowalchuk, Rich Nagle, Dave Robinson, Greg Sawyer, Theresa Semmens, andLisa Zerkle for the effort they put into making these events as successful as they were. Withouttheir help, this report would not have been possible.Foreword/Executive Summary3TLP:WHITEREN-ISAC

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop SeriesBLENDED THREAT WORKSHOP SERIES OVERVIEWSeries BackgroundThe REN-ISAC Blended Threat Workshop Series began in 2018Definition: Workshopout of a desire to improve the Education Facilities CriticalThe HSEEP definition is anInfrastructure (CI) Subsector’s1 capability to respond to threatsinformal discussion “employedwith both cyber and physical components. These threats, whetherto build specific products, suchthey are labeled blended, complex, or some variation, areas a draft plan or policy.”increasing in number and scope as network devices continue to beintegrated into the life of the everyday citizen. REN-ISAC established the Blended ThreatWorkshop Series to create opportunities for ISAC members and other higher educationprofessionals across a wide range of disciplines to confront threat-informed, risk-based scenariosin order to improve their institutions’ security posture before any major incidents occur.Following the format of prior years’ series, the 2021Blended Threat Workshop Series presented sevenThe Wikipedia definition is a natural,workshops containing a scenario with a ransomware-basedaccidental, or purposeful physical orthreat to Internet of Things (IOT) devices commonly usedcyber danger that has or indicates theon college campuses. Due to the complexities created bypotential for crossover implications tothe COVID-19 pandemic, all 2021 workshops were virtual,harm life, information, operations, thea first for the series. At each event, participants interactedenvironment, and/or property.with each other through plenary sessions and breakoutrooms. The sessions were organized into four modules that helped focus discussion, while stillallowing attendees to speak to the broader topic of cyber and physical threats. REN-ISACcollected data from these events to produce two types of reports: seven TLP:AMBER individualreports summarizing the conversation for each workshop and this final TLP:WHITEcomprehensive report that looks across all seven workshops to pull out actionable strategies andtactics.Definition: Blended ThreatThe observations from this workshop are divided intothree categories:Definition: Complex ThreatTwo or more separate attacks aimed atthe same general or specific target orobjective Best Practices – Processes, procedures, or otherobservations identified as valuable or effective. Areas of Improvement – Opportunities for thesector to enhance its security posture. Challenges – Inherent issues that, in today’s threat environment, are unable to beeliminated but may be mitigated.1DHS CISA: The Education Facilities Subsector covers pre-kindergarten through 12th grade schools, institutions ofhigher education, and business and trade schools. The subsector includes facilities that are owned by bothgovernment and private sector entities.Blended Threat Workshop Series Overview4TLP:WHITEREN-ISAC

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop SeriesIn order to inform future emergency preparedness efforts stemming from this report, eachobservation is tied to the appropriate Core Capabilities, as identified in the Federal EmergencyManagement Agency’s (FEMA’s) National Preparedness Goal. Core Capabilities for eachobservation can be found in Appendix D.Purpose and DesignScopeThe following is the scope of the 2021 Blended Threat Workshop Series as approved by the 2021Blended Threat Workshop Series planning team: In CY2020/2021, REN-ISAC will lead the development and conduct seven securityfocused discussion-based exercises, anticipated to be workshop events. These exercisesare anticipated to be conducted virtually between Sep 2020 – June 2021 and to beapproximately six-hours each. Exercise participants are expected to include institutions’leadership personnel, physical and cybersecurity, emergency management, informationtechnology (IT), administration, student affairs, and other key personnel, leaders and / orstaff from REN-ISAC and other higher education institutions, as well as other partnersand subject matter experts, as may be appropriate.ObjectivesThe following are the objectives of the 2021 Blended Threat Workshop Series as approved bythe 2021 Blended Threat Workshop Series planning team:1. Provide a forum for Higher Education organizations to use a complex or blendedransomware threat scenario to prompt discussion and share approaches from leaders inthe community regarding physical and cybersecurity preparedness, coordination, andresponse (these exercises will not focus on recovery) to help inform organizationalpreparedness.2. Provide participants an opportunity to interact with one another and discuss issues,concerns, best practices, and other salient points to help inform organizationalpreparedness.3. Provide feedback to members and the broader higher education community on bestpractices, preparedness gaps, and opportunities for improvement identified through theexercise series to help inform organizational and community security preparedness.4. REN-ISAC will provide participants and their organizations a summary of the discussionwithin 90 days of exercise completion to help inform organizational preparedness.5. REN-ISAC will provide participants and their organizations a roll-up summary of thecomplete 2021 exercise series no later than 90 days after completion of the exerciseseries in order to help inform organizational preparedness.ScenarioIn 2020, the 2021 Blended Threat Workshop Series planning team was formed from thefollowing REN-ISAC stakeholders: Kim Milford, Todd Herring, Andy Jabbour, Joe Potchanant,Amy Starzynski Coddens, Damian Wilk, Sarah Bigham, Brett Zupan, Al Arboleda, Keith Barros,Blended Threat Workshop Series Overview5TLP:WHITEREN-ISAC

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop SeriesJon Garvin, Donald King, Tim Krabec, Rich Nagle, James Offer, Dave Robinson, TheresaSemmens, and Lisa Zerkle.This team developed a four module scenario focused on a ransomware threat to a genericuniversity with a subsequent IOT-based threat to campus devices. At the beginning of eachmodule, a situation update from “Gotham University” was presented to participants. Facilitatorled discussion in a plenary session guided attendees through questions established by theplanning team, as well as any comments, concerns, or other issues raised during the natural flowof conversation. Module Three contained additional breakout discussions that were reported backout to the plenary session. Module Four had three variants, each representing different IOTthreats the planning team felt were plausible and topical. Each host chose one variant from thelist to present at their event. The following are summaries of the four modules exercised duringthe 2021 Blended Threat Workshop Series. The full text of the 2021 modules can be found inAppendix C.Module One: Secret OriginDuring this module, participants were informed that a new strain of malware, PuRevil, had beendiscovered and was targeting non-education sectors. The infection rate of this particularransomware strain saw significant growth over multiple weeks.Module Two: ConfidentialDuring this module, participants were informed that PuRevil infections have begun to appearacross the higher education community. Institutions that do not pay the ransom would have theirdata sold online. A subset of institutions was infected with a PuRevil variant that targeted oncampus IOT devices.Module Three: Speeding BulletsDuring this module, participants were informed their institution had been compromised byPuRevil and that the threat actor behind it had already delivered a ransom threat.Module Four: The Dark Side (Facilities/Housing Variant)During this module, participants were informed their institution declined to pay the ransom.Because of this, the threat actor behind PuRevil used the malware’s capability to compromiseIOT devices. The HVAC, lighting systems, and access control systems within student residencesare being manipulated to create an uncomfortable living environment.Module Four: The Dark Side (Remote Learning Variant)During this module, participants were informed that their institution decided to not pay theransom. Because of this, the threat actor behind PuRevil used the malware’s capability tocompromise IOT devices. Faculty, staff, and students connected to the institution’s remotelearning systems have had devices on their home network infected with PuRevil.Module Four: The Dark Side (Research Variant)During this module, participants were informed that their institution decided to not pay theransom. Because of this, the threat actor behind PuRevil used the malware’s capability tocompromise IOT devices. Vulnerable devices within research labs are being manipulated todamage equipment and sabotage research data.Blended Threat Workshop Series Overview6TLP:WHITEREN-ISAC

TLP:WHITE2021 REN-ISAC Blended Threat Workshop SeriesFinal Findings ReportOBSERVATIONSBest PracticesDefinitionProcedures, processes, or other observations identified as valuable or effective.Observations1. Utilizing REN-ISAC as a Cyber Information Broker During IncidentsDuring the workshops, REN-ISAC observed that there was an opportunity for institutions whoare being consumed by incident response activity to use the ISAC as an information broker.When initially responding to an incident, many institutions do not think about dedicatingresources towards information sharing, as those resources are prioritized towards putting out thebiggest fires; however, timely information sharing is the most powerful way to help othermembers in information sharing communities. One participant noted the benefit of being able tooffload information sharing onto REN-ISAC; the institution can save time and focus on directincident response while knowing the information is being shared and used to assist otherorganizations.REN-ISAC already consistently monitors threats to the Education Facilities CI Subsector incoordination with multiple public and private security partners. This puts REN-ISAC in a uniqueposition where they can take on the burden of distributing IOCs and other critical threatintelligence from institutions suffering severe impacts to these partners. Anyone intending to useREN-ISAC in this manner will be in full control of their information. This includes the TrafficLight Protocol (TLP) level at which the information is shared, which partners (government,private sector, other ISACs) the information is allowed to be shared, whether or not thecontributing organization is anonymous, and any other requirement the information providerwants to set. Once those decisions have been made, REN-ISAC staff can take on the burden ofcoordinating with those partners to get the information out to other organizations that could use itto proactively defend their own networks.2. Preserving Secure, Reliable BackupsBackups are commonly cited as a mitigation against ransomware infections, but multipleparticipants discussed that it is necessary to establish a truly effective backup process. The firstset of variables involves the backup process itself. Security leaders should be able to identifywhere backups are stored, how often those backups are executed, and the methods used toprotect backups from intrusion or infection. Backups do not work if, in the process of infiltratingthe network to plant ransomware, the threat actors are present long enough to also compromisethe backup system.The second set of variables involves educating and informing stakeholders, especially leaders,about the backup process. With the current international focus on ransomware, institutionalleaders should be fully educated on the elements of the organization’s backup policies and theimplications they pose on recovering from an incident. This not only helps decision makingObservations7TLP:WHITEREN-ISAC

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop Seriesduring an incident but also, as one participant noted, creates an opportunity for the IT team toreceive resources to help fill any gaps. Another piece of information critical for ransomwareresponse is knowing the average time it takes to recover from backups or to rebuildcompromised systems. This metric is critical when determining the impact of a successfulransomware attack and whether or not the institution should pay.3. Taking Advantage of TemplatesREN-ISAC observed that multiple workshop participants utilized the events as a way tojumpstart the creation of their institutions’ Incident Response Plan (IRP) or ransomware-specificplan. It can be a difficult task to build a comprehensive IRP from start to finish with noassistance, and those participants found the workshops to be a source of inspiration and ideas thatcould serve as a potential foundation. Many participants shared existing templates they haddiscovered (see Appendix D) while researching the creation of a departmental or institutionalplan.Discussed at multiple workshops, a critical piece of advice for these participants was thatinstitutions without an IRP should consider duplicating a template from another source to use astheir initial policy. To that end, a number of useful resources were shared by participants of theworkshops. This document, whether from a peer institution providing a copy of their plan or asample document created by experts, can be tweaked to fit the details of any organization andrapidly enshrined as policy. As the institution becomes used to the plan through exercises orincidents, it can become more customized to the needs of all stakeholders. The benefit of thisapproach is that there is no delay; the institution has an existing document and can begin workingon continually updating the plan based on resource availability and real-world needs. This is incontrast to an institution that spends a lot of resources to make an initial plan from scratch inhopes it will perfectly fit the institution but, in reality, will still require continuous updates basedon the lessons learned from exercises and actual incidents.4. Preplanning the Decision of Paying the RansomWorkshop participants were at different stages of the policy conversation when it came todeciding whether to pay and, if so, how to pay or not pay the ransom when their institution waseventually infected with ransomware. Some institutions had no guidelines, some institutions hadset guidelines, and some institutions were in the process of building guidelines. Setting a policyfor when to pay a ransom is a difficult topic, especially since there is the potential for damage ifcriminal actors or the media are able to access that policy. However, it is important forinstitutional stakeholders, especially leaders, to have put effort into planning if and how to pay aransom when an incident occurs. It is not a decision any leadership team wants to be suddenlyconfronted with during an incident event.One participant offered a best practice at their institution: establishing a clearly defined group ofroles that have the responsibility to decide whether or not to pay the ransom for any piece ofmalware on the university’s network, whether it is a single professor’s computer or largenumbers of machines. The participant’s institution chose the CISO, CIO, CFO, Provost, andChief Legal Officer as the stakeholders for their group; however, the makeup of this group couldvary. The key consideration here is to include the leaders who understand the current impact ofany ransomware infection and possess the capability to facilitate a payment if necessary. A sideObservations8TLP:WHITEREN-ISAC

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop Seriesbenefit of creating this decision-making body is that it does not commit the institution to eitherpaying or not paying, while still proactively assigning responsibility for how to handle thataspect of a ransomware incident.5. Using the Incident Command SystemThe use of the National Incident Management System (NIMS) Incident Command System (ICS)was a topic at multiple workshops. Multiple participating organizations used it as the frameworkto organize their incident response activities. The NIMS ICS is a nationally recognized processdeveloped by FEMA and utilized by law enforcement, paramedics, fire fighters, and other firstresponders that have to confront complex natural disasters or incidents. It is designed withinteroperability in mind, so that all responders are working off a common language. Other publicand private sector organizations have adopted it as their own incident management processes forthat reason.There are many free resources available for individuals and organizations who want to becomecertified with NIMS ICS. FEMA offers a suite of online classes that can help professionals getup to speed on the foundational concepts of the ICS process. Institutions should consideridentifying critical incident response decision makers and make the training mandatory for thoseroles, though any stakeholder involved would find the information useful. Furthermore, currentplans and processes should be examined for their compatibility with NIMS ICS and, if it makessense, aligned to better fit with the system. Since NIMS ICS is a macro-scale methodology forresponding to incidents, this alignment would be more beneficial for general plans, rather thanincident-specific playbooks.6. Maintaining an Exercise ProgramAcross all workshops, it was agreed: the act of training and exercising existing plans andprocedures is the only way to truly be ready for an event. Having a plan on file does not conferexperience on how to execute it or familiarity with how it works under real world conditions. Anorganization will gain a benefit from their plans only through dedicating resources, time, and theattention of its leaders. Institutional muscle memory for incident response is built throughpractice and multiple participants noted that this muscle memory can be more important than theplan itself, as it allows staff to be flexible and know their role even if a situation does not fit theplan or becomes unpredictable.There is no secret to consistent training and improvement; it is simply a function of what isinvested into the exercise program. Effort has to be spent to create useful training scenarios, timehas to be set aside to get key stakeholders together for long discussions, and a champion isneeded to keep the process moving between exercises. Thankfully, exercise programs aresomething that easily scale and an institution can start small before expanding outwards. Regularworkshops are an easy beginning goal since they use short, informal discussions to brainstormapproaches to different types of incidents that could target an institution. From there, anorganization can work up to more formal workshops and, eventually, to a tabletop exercisewhere a fully developed plan is tested against a rigorous scenario. When ready, organizations canmove from discussion-based exercises towards more challenging operational testing, includingdrills, functional and full-scale exercises.Observations9TLP:WHITEREN-ISAC

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop Series7. Preparing to Determine the Value of Compromised DevicesWhen looking at the question of paying the ransom from an economic perspective, it is acomparison of the cost of the ransom versus the cost of downtime for all infected devices.However, multiple workshop participants noted that accurately calculating the cost and providingthat to leaders in a timely manner requires resources. This capability should be prepared andpracticed ahead of an incident, so organizations can avoid the pitfall of improvising such criticalnumbers while also battling the pressure tactics of the criminal actors who have their data.Determining an accurate cost of ransomware-related downtime requires understanding what datathe ransomers have in their possession, the relative value and priority of all assets within theorganization, and the amount of time it would take for the IT team to restore the network.Generating these numbers is difficult and requires investing in initiatives (such as businessimpact studies focused on network infrastructure and system owners) or internal drills todetermine the average time it takes to restore from backups. These are resource-intensive andtake time to integrate, but leaders will benefit from a much deeper understanding of theirorganization’s networks and the consequences of any incidents targeting it.8. Controlling IOT ProactivelyOne best practice offered by cybersecurity leaders focusing on IOT device management withintheir organization was to engage in proactive IOT control through asset inventories and creatingand maintaining relationships with other IOT-related teams in the institution. IOT devices areparticularly insecure. Not knowing what is being connected to the network or who is doing itcreates multiple points of vulnerability a network defender has to manage. The best way to gainthis awareness is to coordinate with all IOT stakeholders.Creating and maintaining proactive IOT control requires a multi-pronged approach. Multiple ITparticipants cultivated a relationship with other teams in charge of deploying or maintaining IOTdevices, especially with facilities management personnel. Many of these participants had regularmeetings with their counterparts, some even attending the other team’s weekly meetings. Thisallowed for the informal flow of information, which helped to provide context or alert managersof potential incidents before they became an issue. These relationships assisted with maintainingmore robust asset inventories. While cataloging all devices connected to the network is an eternalchallenge, building relationships and educating peers helped IT leaders get support from otherteams for processes that recorded when new IOT devices were procured and added to thenetwork.9. Prioritizing ThreatsMany participants shared the methods they utilized to prioritize threats to their institution’snetworks on a daily basis, and there were a few common themes that were observed across allworkshops. Smaller IT shops, due to a lack of resources, tended to use third parties to signpostpressing issues. Alerts from government organizations like CISA, information sharingorganizations like REN-ISAC, and similar partners were triggers for those teams to abandonsteady state operations to examine the potential threat. Larger shops were able to handle thevulnerability management side of operations, but only the largest were able to dedicate full-timeObservations10TLP:WHITEREN-ISAC

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop Seriesresources to threat hunting. Teams of every size utilized personal connections built with peers atother institutions to further prioritize the multitude of threats faced by cyber defenders every day.Another consideration when it came to threat prioritization is redundancy. For most teams, threathunting, if done at all, is only one component of team members’ daily duties. Participantsdiscussed the different methods they used to share that burden. In terms of redundancy, the bestresults were noted to come from assigning multiple staff members to the job part time, eachusing a personalized collection of intelligence sources. In terms of uniformity, the best resultswere noted to come from assigning one or two staff members to the job in a more permanentrole, producing a regular threat intelligence product. Either way, IT leaders noted the importanceof staff members being able to communicate and coordinate with each other horizontally in orderto quickly disseminate threats and their priority.Observations11TLP:WHITEREN-ISAC

Final Findings ReportTLP:WHITE2021 REN-ISAC Blended Threat Workshop SeriesAreas of ImprovementDefinitionOpportunities for the sector to enhance its security posture.Observations1. Confronting the Difficulty of Timely Emergency ProcurementAs one participant noted after their institution underwent a major internal exercise, anorganization could potentially ad hoc every other aspect of their incident response process exceptfor procurements. Responding to an incident might require sudden and unusual purchases ofspecialized equipment and steady state procurement processes are often not capable of achievingthis expectation. One participant reported that all emergency procurement requests had to gothrough their institution’s board of trustees, which could be a challenge in the middle of anincident. Emergency procurement is an especially important topic in relation to ransomwareattacks, as the best way to ensure the network is free from infections is to replace compromisedmachines.Security leaders should think about testing their institution’s procurement policies against itsincident response policies to see if they are capable of working in tandem. If they are not, then itis critical to bring together the appropriate stakeholders to discuss how procurement should workduring emergencies. Consider running a conversational drill to expose areas of concern, such asdiscussing how to recover from a theoretical ransomware incident that involved a large numberof infected devices on the network. Once discovered, these concerns can then be addressed bythe group as a whole. This is not a conversation that can be had in the middle of an incident,which makes it a major priority for organizations trying to proactively stre

Oct 27, 2021 · workshop participants take full advantage of the workshop opportunity to ask hard questions and . module, a situation update from “Gotham University” was presented to participants. Facilitator-led discussion in a plenary sess