WIXLIB

Multicore Processing (MCP)Asymmetric Multiprocessing (ASP)APP 14APP 13APP 12APP 10APP 11APP 1APP 2APP 3APP 4APP 5APP 6APP 7APP 8APP 9Heterogeneous cores (but homogeneous OS): Compare with single core processor separate graphics card Today, GPU cores treated as a peripheral used by CPU coresMiddlewareMulticore Host Operating System (OS)Board Support Package (boot loader, OEM Adapters, and device drivers)CPU CoresGPU CoresDSP Core(s)Fast Core(s)i-Caches d-Cachesi-Caches d-Cachesi-Caches d-Cachesi-Caches d-CachesL2 CachesL2 CachesL2 CachesL2 CachesL3 CacheMain MemoryInfrastructureLayerHardwareLayerSystem BusMulticoreProcessor Memory ControllerApplicationSoftwareLayerI/O ControllerI/O DeviceMulticore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.15

Multicore Processing (MCP)Current TrendsMulticore processors are replacing traditional single coreprocessors: Fewer single core processors are being produced and supported. Single-core processors are increasingly technologically obsolete(as technical advances are primarily applied to multicoreprocessors)The number of cores continues to increase.Asymmetric (e.g., computer on a chip) processors becoming morecommon.User demand for significantly-increased performance in SWAP-Cconstrained environments increases need for multicore processing.Multicore processors are starting to be used in real-time, safety- andsecurity-critical, cyber-physical systems.Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.16

Multicore Processing (MCP)Pros – Increased Energy EfficiencyDecrease number of separate embedded computersOvercomes increased heat generation due to Moore’s Law Reduces the need for coolingReduces power consumption Increases battery lifeReduces SWAP-C (Size, Weight, and Power and Cooling/Cost)Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.17

Multicore Processing (MCP)Pros – True ConcurrencyIncreased intrinsic support for actual (as opposed to virtual)parallel processing of: Individual software applications Multiple SW applications (server and cloud computing)Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.18

Multicore Processing (MCP)Pros – Increased PerformanceDepends on number of cores, level of real concurrency(multithreading) of the software, and use of shared resourcesDecreased distance between cores on integrated chips enableshorter resource access latency and higher cache speeds Compared to having separate processors/computersMulticore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.19

Multicore Processing (MCP)Pros – Improved IsolationTypically improves (but does not guarantee) spatial and temporalisolation (segregation) compared to single core architectures: SW running on one core less likely to affect SW on another corethan if both are executing on same single core- Spatial isolation of data in core-specific cashes- Temporal isolation of cores because thread on one core is notdelayed by thread on another core(except for interference due to overlapping access to sharedresources) May improve robustness by localizing impact of defects to singlecoreThis increased isolation is particularly important in the “independent”execution of mixed-criticality applications (mission-critical, safetycritical, and security-critical).Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.20

Multicore Processing (MCP)Cons – Shared ResourcesCores share: Processor-internal resources (L3 cache, system bus, memorycontroller, I/O controllers, and interconnects) Processor-external resources (main memory, I/O devices, andnetworks)Shared resources imply: Single points of failure Two applications running on same core can interfere with eachother. Software running on one core can impact software running onanother core (i.e., interference can violate spatial and temporalisolation because multicore support for isolation is limited).Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.21

Multicore Processing (MCP)Cons – InterferenceInterference occurs when software executing on one core impactsthe behavior of software executing on other cores in the sameprocessor: Failure of spatial isolation (due to shared memory access) Failure of temporal isolation (due to interferencedelays/penalties)Multicore processors may have special hardware that can be usedto enforce spatial isolation to prevent software running on differentcores from accessing the same processor-internal memory. Temporal isolation is a bigger problem than spatial isolation.The number of interference paths increase very rapidly withnumber of cores. Exhaustive analysis of all interference paths is often impossible. Representative selection of paths is necessary.Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.22

Multicore Processing (MCP)Cons – Example Interference PathsAPP 14APP 13APP 12APP 11APP 10APP 9APP 8APP 7APP 6APP 5APP 4APP 3APP 2APP 1Three example interference paths with shared resources indicated:MiddlewareMulticore Host Operating System (OS)Board Support Package (boot loader, OEM Adapters, and device drivers)CoreCoreCoreCorei-Cache d-CacheL2 Cachei-Cache d-CacheL2 CacheBus / Interconnecti-Cache d-CacheL2 CacheMulticore Processor Memory ControllerMainMemoryi-Cache d-CacheL2 CacheL3 CacheI/O Device ControllerI/O DeviceMulticore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.23

Multicore Processing (MCP)Cons – Increased Concurrency DefectsIncreased potential for concurrency defects due to cores executingconcurrently: Deadlock Livelock Starvation Suspension (Data) race conditions Priority inversion Order violations Order vulnerabilities Atomicity violationsIncreased amount and difficulty of testing needed to uncoverconcurrency defectsMulticore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.24

Multicore Processing (MCP)Concurrency DefectsDeadlock is a failure condition that exists when one thread or process cannot proceed because itneeds to obtain a resource that is held by a second thread, while the first thread holds a resourcethat the second thread needs. All involved threads are in a waiting state as they wait for otherthreads to release the resource they need.Livelock is a failure condition that exists when one thread or process is waiting on a resource thatwill never become available, while a CPU is busily releasing and acquiring the shared resource.The state of the waiting thread is constantly changing, with the thread frequently executing butnever reaching completion.Starvation is a failure condition that exists when a thread or process is ready to execute but isindefinitely delayed because other processes are continually given preference.Suspension is a failure condition that exists when a thread or process is forced to wait too longbefore it can access a shared resource. The thread eventually obtains the resource but too late.Data Race is a failure event that occurs when at a thread or process writes to an unprotectedmemory location while others are simultaneously accessing it.Priority Inversion in which a higher priority thread or process is forced to wait on a lower priorityone.Order Violation is a failure event that occurs when two or more threads or processes execute inan incorrect order.Order Vulnerability exists when the expected order of at least two memory accesses is notenforced.Atomicity Violation is a failure event that occurs when a code block that must run to completionwithout disruption is interrupted by the execution of another code block.Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.25

Multicore Processing (MCP)Cons – Increased Non-DeterminismI/O Interrupts have top-level hardware priority Note that this is also a problem with single core processors.Lock thrashing is the existence of excessive lock conflicts due tosimultaneous access of kernel services by different cores, resultingin decreased concurrency and performance.The resulting behavior is non-deterministic, unpredictable, and thesource of related failures.Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.26

Multicore Processing (MCP)Cons – Analysis is more complex and difficultReal concurrency requires: Different memory consistency models than virtual interleavedconcurrency Breaks traditional analysis approaches that work on single coreprocessorsTemporal analysis of maximum time limits is: More difficult May be overly conservativeMemory access analysis of spatial interference is more complex.Although interference analysis becomes more complex as thenumber of cores per processor increases, overly restricting corenumber may not provide adequate performance.Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.27

Multicore ProcessingCons – Safety RamificationsMoving to a multicore architecture may require recertification.Interference between cores can cause missed deadlines andexcessive jitter: Can cause faults (hazards) and failures (accidents) Requires:- Proper real-time scheduling and timing analysis and/or- Specialized performance testingSafety policy guidelines are based on obsolete assumptions.Safety policy guidelines need to be updated based on theguidelines in the recommendations section.Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.28

MCP, Virtualization, and ContainerizationVirtualization (V)[DISTRIBUTION STATEMENT A] This material has beenMulticore Processing,Virtualization,& ,&[DISTRIBUTIONThismaterialhas been approved forapprovedSTATEMENTfor public releaseA]andunlimiteddistribution.February 26, onUniversity.University 201829public releaseandunlimitedPleaseuseseePlease seeCopyrightnoticedistribution.for non-US GovernmentandCopyright noticedistribution.for non-USGovernment use and distribution.

Virtualization (V)Definition – Virtual MachinesA virtual machine (VM), also called a guest machine, is asoftware simulation of a hardware platform that provides a virtualoperating environment for guest operating systems.A platform VM, also called a system VM and full virtualizationVM, is a VM that: Runs on top of a hypervisor Simulates a complete hardware platformAn application VM, also called a process VM, is a VM that: Runs as a language-specific software application (e.g., JavaVM) on top of the host OS process Provides a platform-independent programming environmentFor the rest of this presentation, we will restrict ourselves toplatform VMs.Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.30

Virtualization (V)Definition - HypervisorsAPP 7APP 6APP 3APP 4APP 5APP 2APP 1APP 7APP 6APP 3APP 4APP 5APP 2APP 1A hypervisor, also called a virtual machine monitor (VMM), is asoftware program that runs on an actual host hardware platformand supervises the execution of the guest operating systems onthe virtual SVM 1VM 2VM 3VM 1VM 2VM 3Type 1 HypervisorType 2 HypervisorHost Hardware PlatformHost OSHost Hardware PlatformMulticore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.31

Virtualization (V)RTOSVM 1VM 2VM 3VM 4VM 5VM 6VM 7APP 14APP 12RTOSAPP 13APP 9APP 10APP 11WINDOWSAPP 7LINUXAPP 6LINUXAPP 3APP 4APP 5LINUXAPP 2WINDOWSAPP 1NotionalDiagramApp 8Type 1 (“Bare Metal”) Hypervisor on MCPType 1 (“Bare Metal”) HypervisorCoreCoreCoreCorei-Cache d-Cachei-Cache d-Cachei-Cache d-Cachei-Cache d-CacheL2 CacheL2 CacheL2 CacheSystem BusMain MemoryInfrastructureLayerVirtualizationLayerL2 CacheL3 CacheMulticoreProcessor Memory LayerI/O ControllerI/O DeviceMulticore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.32

Virtualization (V)RTOSVM 1VM 2VM 3VM 4VM 5VM 6VM 7APP 14APP 12RTOSAPP 13APP 9APP 10APP 11WINDOWSAPP 7LINUXAPP 6LINUXAPP 3APP 4APP 5LINUXAPP 2WINDOWSAPP 1NotionalDiagramApp 8Type 2 (“Hosted”) Hypervisor on MCPType 2 (“Hosted”) HypervisorCoreCoreCoreCorei-Cache d-Cachei-Cache d-Cachei-Cache d-Cachei-Cache d-CacheL2 CacheL2 CacheSystem BusMain MemoryVirtualizationLayerL2 CacheL3 CacheMulticoreProcessor Memory ControllerInfrastructureLayer (Guest)InfrastructureLayer (Host)Host OSL2 I/O ControllerI/O DeviceMulticore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.33

Virtualization (V)Current Trends – 1Virtualization is reaching saturation at the server level for: IT applications Data centers Cloud computingVirtualization is increasingly being used for: Storage virtualization (mass storage) Network virtualization Mobile devices (especially testing on virtual mobile devices)Virtualization is only just beginning to be used for real-time, safetycritical, and security-critical systems such as: Automotive software Internet of Things (IoT) Military softwareMulticore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.34

Virtualization (V)Current Trends – 2Virtualization is being combined with Containerization.Where appropriate, VMs are being replaced by lighter-weightcontainers.Security is increasingly important as vulnerabilities (VM escapes)in virtual machines and hypervisors are discovered.Multicore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government use anddistribution.35

Virtualization (V)Pros – Increased Hardware IsolationIncreased hardware isolation: Supports reuse of software written for different, potentially olderoperating systems and hardware Enables upgrade of obsolete hardware infrastructure software Improves portability to multiple hardware and OS platforms Enables virtualized test bedsMulticore Processing, Virtualization, & ContainerizationFebruary 26, 2019 2019 Carnegie Mellon University[DISTRIBUTION STATEMENT A] This material has beenapproved for public release and unlimited distribution.Please see Copyright notice for non-US Government

Virtualization (V) - via virtual machines (VMs) Containerization (C) - via containers. These systems are: Not just weapons systems, aircraft, etc. with embedded software Not just data processing systems in the cloud For example, ground control stations. What are the significant ramifications on performance, reliability,