Oracle Solaris And Sun SPARC Systems—Integrated And Optimized For .

Transcription

An Oracle White PaperJune 2010Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for EnterpriseComputing

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise ComputingExecutive Overview . 1Introduction—Oracle Datacenter Integration . 1Overview . 3The Oracle Solaris Ecosystem . 3SPARC Processors . 4Architected for Reliability . 7Oracle Solaris Predictive Self Healing . 8Highly Reliable Memory Subsystems . 9Oracle Solaris ZFS for Reliable Data . 10Reliable Networking . 10Oracle Solaris Cluster . 11Scalable Performance . 14World Record Performance . 16Sun FlashFire Storage . 18Network Performance . 18Security . 19Integrated with Sun SPARC Enterprise T-Series Servers . 19The Oracle Solaris Cryptographic Framework Library . 20Preventing Attacks . 21Least Privilege . 21Common Criteria . 22Oracle Server Virtualization . 23Oracle VM Server for SPARC . 24Oracle Solaris Containers . 25Dynamic Domains and Dynamic Reconfiguration . 26Comprehensive Management with Oracle EnterpriseManager Ops Center . 27Developer Tools Optimizations . 27Conclusion . 29Resources . 30

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise ComputingExecutive OverviewThis document is intended for IT architects, system administrators, and developers that want tounderstand the details of how Oracle Solaris and SPARC can improve your applicationsolution environment. This paper will provide technical information on how Oracle Solaris andthe SPARC processor have been highly optimized for each other, improving throughput,security, and resiliency throughout the application solution stack, driving maximum ROI andminimum TCO. It includes brief technical descriptions of how specific Oracle Solaris featuresand capabilities are implemented in a system-wide approach to optimize the specificfunctionality of the SPARC processor family in the areas of scalable performance, advancedreliability, security, and cost-effective virtualization—and enhance your Oracle solution set.Introduction—Oracle Datacenter IntegrationOracle offers customers a complete integrated stack, from the applications layer at the top to diskstorage systems at the bottom, as shown in Figure 1. Oracle is the number one vendor in the top threesoftware segments (applications, middleware, and database), and Oracle Solaris is today the numberone deployment platform for Oracle Database applications in the market. Oracle offers customers acomplete top-to-bottom solution that is open and fully integrated.Figure 1: Complete. Open. Integrated. Oracle Solaris and Oracle’s Sun SPARC Enterprise servers are the optimal solution stack for OracleDatabase and Applications.Oracle and Sun have a long history of optimizing the platforms for scalability, reliability, and security.Working together, these improvements have enhanced and optimized the entire stack and leveragedinnovation throughout. This paper offers a high-level discussion of the benefits of Oracle Solarisrunning on Oracle’s Sun SPARC Enterprise M-Series and T-Series servers, and drill-down informationon specific optimizations and advantages for increased reliability, scalability, security, and virtualization.Resources that can provide more information are listed at the end of each section, and acomprehensive list is available at the end of the paper.1

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise ComputingHere are some examples of how cooperative innovation improved the application performance andreliability on Oracle Solaris, Sun SPARC Enterprise servers, and Oracle Database and Applications.Scalability and Performance Solaris was the first commercially available UNIX to offer a 64-bit version. This enabled the 64-bitversion of Oracle 8i to scale to beyond the 4 GB memory barrier. This was necessary to make use ofthe 64 GB of memory available on the Sun Enterprise 10000 (―Starfire‖) servers. Recently1 Oracle Solaris, Oracle 11g Enterprise Edition with Real Application Clusters andPartitioning, and Sun SPARC Enterprise T5440 server combined to set a new world-record TPC-Cbenchmark: 7,646,486.7 tpmC. This is the latest in a long tradition—10 years ago Sun and Oraclebroke the 100,000 tpmC barrier with a result of 115,395.73 tpmC running Oracle 8i with Solaris 7 ona Sun Enterprise 10000. Large page support and multiple page size support (MPSS) expanded memory page sizes up to256 MB, and increased the performance of Sun SPARC T-Series and M-Series servers runningOracle Database. Memory Placement Optimization (MPO) enables processors to have an affinity for the closestmemory on Non-uniform Memory Access (NUMA) systems—the types of multisocket, largememory systems that are powered by SPARC processors and Oracle Solaris. Sun collaborated withOracle to define and use the lgroup API, lgrp init (3LGRP), and enable Oracle to optimizelocal versus remote access to the System Global Area (SGA, the database buffer cache) on NUMAmachines. These optimizations were made default on Oracle 10g running on Sun NUMA basedmachines. These optimizations help increase the locality of reference for the SGA and ProcessGlobal Area (PGA, a dedicated memory cache). The performance improvements can be quite drasticdepending on the server. Oracle Solaris MPO innovations are key to scaling on servers with highNUMA ratios. Intimate shared memory (ISM) shares translation tables involved in the virtual to physical addresstranslation for shared memory pages, as opposed to just sharing the actual physical memory pages.ISM was a critical technology which enabled Oracle to efficiently scale on large SMP systems as wellas smaller machines.Availability 1Dynamic ISM enabled Oracle support for the dynamic SGA feature introduced in Oracle9i. Thisallowed a DBA to dynamically increase or decrease the size of the SGA (up to a limit defined bysga max size) without needing to restart the Oracle instance. Using the Solaris Reconfigurationblogs.sun.com/BestPerf/entry/new tpc c world record2

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise ComputingCoordination Manager (RCM), it is also possible to write a script that allows Oracle to be alertedwhen new CPUs/memory are to be removed from the domain, so that the SGA can be dynamicallyscaled back to allow the board to be removed without shutting down the database. For many years Oracle Solaris Cluster software has been evolving to complement and integrate withOracle Database solutions including Oracle Real Application Clusters (RAC). The result isthoroughly tested, tightly integrated, end-to-end solutions that extend the advantages of OracleSolaris and Sun SPARC Enterprise systems into multiserver, high-availability environments.SecurityRole-based access control (RBAC) is a feature of Oracle Database, Oracle E-Business Suite, andOracle Solaris. In the RBAC model in Oracle Solaris, users log in as themselves and assume roles thatenable them to run restricted administration graphical tools and commands. RBAC is considered a bestpractice across all Oracle products.While there are many integration synergies to come, today Oracle offers end-to-end management forthe complete hardware software stack, from application to disk. Oracle Enterprise Manager offerscustomers visibility into underlying Sun servers, Oracle Solaris and associated virtualization, helpingthem to resolve issues that could impact application, middleware and database service levels. Thisincludes extensive capabilities for managing physical and virtual Sun environments.OverviewOracle Solaris is the centerpiece on which Oracle delivers integrated hardware and software solutionsthat are reliable, scalable, and secure. Thousands of customers worldwide depend on SPARC-basedsystems and Oracle Solaris to run their business, usually for one simple reason: these platforms simplydon’t quit. Maximum scalability is achieved when multicore servers and highly threaded operatingsystems host middleware and applications that are tuned to take advantage of these capabilities.SPARC Enterprise servers offer up to 512 hardware processing threads and four terabytes (4 TB) ofmemory. Oracle Solaris offers an industry-leading threading model, the result of nearly two decades ofinnovation. Oracle Database and Middleware products have been tuned to maximize performance andscalability on this platform. Oracle Solaris offers an exceptionally secure environment, including onchip encryption capabilities, a robust cryptographic framework, Trusted Extensions, and virtualizationcapabilities. Finally, a comprehensive development platform enables organizations to create newapplications that maximize solution performance while improving reliability.The Oracle Solaris EcosystemOracle’s comprehensive portfolio of operating system, virtualization, and cluster technologies includesOracle Solaris, the OpenSolaris development community, Oracle VM, Oracle Solaris Cluster, and theOracle Solaris Studio software development tools, which form the core of a large developer ecosystem.Oracle Solaris is a proven, industry-leading operating system with features designed to handleenterprise, business-critical operations. In fact, Oracle Solaris 10 provides key functionality for3

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise Computingvirtualization, optimal utilization, high availability, unparalleled security, and extreme performance forboth vertically and horizontally scaled environments. Oracle Solaris 10 runs on a broad range ofSPARC (and x86-based) systems and compatibility with existing applications is guaranteed. This is whythere are over 50,000 businesses and institutions running over 11,000 certified applications on OracleSolaris today.Powering Oracle’s Sun SPARC Enterprise servers, Oracle Solaris continues to set world records forperformance, scalability, and cost-effectiveness. Oracle is investing more in Solaris than Sun did priorto the acquisition, and will continue to develop innovative technologies and enhance Oracle Solaris.Oracle Solaris includes many unique and innovative technologies that are uncommon to otheroperating system vendors—including: Oracle Solaris ZFS, Oracle Solaris DTrace, Predictive SelfHealing, built-in virtualization, independent security verification, binary compatibility, and the OracleSolaris Cluster high availability and disaster recovery solutions. Oracle protects your IT investments byguaranteeing that existing Oracle Solaris 8 and 9 applications will run unmodified on Oracle Solaris 10.As enterprise system hardware often has a service life of 8-10 years or more, it is comforting tounderstand Oracle’s commitment to providing a long-lived platform for the software environment.SPARC ProcessorsSPARC (Scalable Processor ARChitecture) is a RISC instruction set architecture developed by SunMicrosystems (now Oracle). The ―Scalable‖ in SPARC comes from the fact that the SPARCspecification allows implementations to scale from embedded processors up through large serverprocessors, all sharing the same (non-privileged) core instruction set. A single version of Oracle Solarisruns across all Oracle Sun SPARC Enterprise servers, including M-Series and T-Series servers. Thismeans datacenters can run a single OS—Oracle Solaris—across all systems, including x86-basedsystems, from the smallest to the largest, greatly simplifying administration. Combined with OracleSolaris, Sun SPARC Enterprise servers provide record-setting performance, extreme scalability,mainframe-class reliability and availability, and strong security.Table 1 provides and overview of the key features of the SPARC processor architectures.TABLE 1: KEY FEATURES OF THE SPARC PROCESSOR ARCHITECTURE BY FAMILYFEATURET-Series with UltraSPARC T2/T2 Plus*M-Series with SPARC64 VIICores/Threads/Sockets8 cores/8 threads/4 sockets4 cores/2 threads/64 socketsUp to 256 processing threadsUp to 512 processing threadsChip Multithreading (CMT)Simultaneous Multithreading (SMT)Maximum frequency1.6 GHz2.88 GHzL2 cache4MB on chip6 MB on chipOn-chip supportPCI Express bridge, integrated dual 10GbEL2 cachenetworking with XAUI, crypto acceleration,L2 cache4

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise ComputingMaximum memory512 GB4 TBReliability featuresPredictive Self Healing, hot-swapEnd-to-end ECC protection; guaranteed datacomponents, ECC everywhere, redundantpath integrity; automatic recovery with instructioncomponents and networking.retry; total SRAM and register protection; ECCand Extended ECC protection for memory,memory mirroring, and Predictive Self Healing;full hardware redundancy; fault-isolated dynamicdomains; dynamic reconfiguration;autodiagnosis, and recovery; guaranteed datapath integrity, total SRAM and registerprotection.SecurityMultiple on-chip cryptographic capabilities,Available add-in crypto-accelerator cardsplus additional protectionsVirtualization (V12N)Oracle VM Server for SPARC (previouslyDynamic Domains and Oracle SolarisIncluded at no extra charge—third-called Logical Domains or LDOMs) andContainersparty products also availableOracle Solaris ContainersTarget environmentsNetwork-facing: consolidation andData-facing. Optimized for 24x7 mission-criticalvirtualization, Web, Media, security, OLTP,computing: DSS, ERP, CRM, BIDW, largemiddleware/SOA, batch processing,databases, large-scale OLTP, anddatamart, application serversHPC/scientific/engineering applications, thatrequire mission-critical RAS features.* UltraSPARC T2 Plus is the multisocket version of the UltraSPARC T2 processor. Up to fourUltraSPARC T2 Plus processors can be used in a single server.As shown in Table 1, the SPARC processor family is designed and optimized for different types ofapplication environments. The same Oracle Solaris provides commonality across both hardwareplatforms in myriad of applications and different datacenter tiers. The SPARC processor family spans awide range of enterprise servers to create architectures that are suitable for best efficiency and security,such as with the T-Series, to massive scalability and availability, such as the M-Series. These twoplatforms create a potent mix of solutions such as CRM systems and Java/Web middlewareinfrastructure with the T-Series to ERP systems and backend OLTP/DW systems with the M-Series.The SPARC processors provides a range systems—one to four sockets for T-Series, up to 64 socketsfor M-Series—to run critical systems for the business from the edge of the network to deep in thedatacenter. We recommend that the server choice in an architecture is based purely on specificapplication scenarios and expectations, and can be mixed and matched.A specific recommendation is out of the scope of this paper and we encourage you to understand theSPARC server application scenarios and case studies on Oracle.com or by discussing with your Oraclerepresentative. The exact sizing and capacity planning can be undertaken with the help of Oracle’sexperts. Your specific scenarios can be tried and tested at Oracle facilities before deployment. Thefollowing section describes the specific SPARC servers.5

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise ComputingSun SPARC Enterprise T-Series Servers with Chip Multithreading (CMT)The UltraSPARC T2 processor with CoolThreads technology implements the industry’s first massivelythreaded ―system on a chip.‖ These processors power the Sun SPARC Enterprise T-Series servers.With support for up to 8 cores/8 threads per core (64 threads per chip)—and up to four sockets—thisprocessor provides breakthrough performance and energy efficiency. In addition, the UltraSPARCT2/T2 Plus processors are the first to integrate 10 Gb Ethernet, PCI Express I/O, and cryptographicacceleration directly onto the processor chip. Combined with Oracle Solaris, this approach providesleading levels of performance and scalability with extremely high levels of efficiency. CMT architectureis ultimately very flexible, and working with Oracle Solaris allows different modular combinations ofprocessors, cores, and integrated components, which offer: Increasing computational capabilities to meet the growing demand from Web applications Supporting larger and more diverse workloads with greater floating point performance Powering faster networking to serve new network-intensive content Providing end-to-end datacenter encryption Increasing service levels and reducing downtime Improving datacenter capacities while reducing costsClosely orchestrated with Oracle Solaris, these systems provide record-setting performance andexcellent RAS characteristics, ideal for maximizing the uptime and ROI of mission-critical enterpriseapplications. Note that there are additional features that contribute to enhanced reliability, includingadvanced integration—significantly lower parts component count—and superior energy efficiency thatcontributes to a reduction of faults due to thermal conditions.Sun SPARC Enterprise servers running Oracle Solaris are built to achieve high levels of uptime andfast recovery from failures. Administrators can utilize Oracle Solaris commands to remove and replacedisks, power supplies, and fan units while the system continues to operate. One PCI Express rootcomplex per processor combined with the ability to configure multiple CPUs, memory FB-DIMMs,and I/O cards add to the resiliency of Sun SPARC Enterprise T5140, T5240, and T5440 servers. Hotswap and hot-plug chassis mounted hard drives, fan units, and power supplies improve serviceabilityand availability.Sun SPARC Enterprise M-Series Servers with SPARC64 VIISPARC64 processors power Oracle’s Sun SPARC Enterprise M-Series servers. Running Oracle Solaris,these platforms offer mainframe-class features and sustainable levels of record-setting applicationperformance. SPARC64 VII processors provide four cores, with two strands (threads) per core. Incombination with Oracle Solaris, SPARC64 VII provides simultaneous multithreading (SMT)scalability to support parallel execution of all eight threads across all available processors (from 1–64processors). SPARC Enterprise servers feature memory subsystems as large as 4 TB, and highthroughput I/O architectures.6

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise ComputingSun SPARC Enterprise M3000, M4000, M5000, M8000, and M9000 servers running Oracle Solarisdelivers a mainframe-class system architecture for high availability (HA) running Oracle Solaris 10.Furthermore, the range of compute power offered by these servers provides the levels of verticalscalability required for server consolidation and many other deployment classes. SPARC EnterpriseM4000 and M5000 servers fulfill mid-range system requirements, while SPARC Enterprise M8000 andM9000 servers deliver the massive processing power needed for high-end computing.Many design features of Sun SPARC Enterprise M3000, M4000, M5000, M8000, and M9000 serverswork together with Oracle Solaris in contributing to a comprehensive and integrated architecturalapproach that is designed for high availability of key systems at lower total costs. Mainframe-class RASfeatures come standard in the Sun SPARC Enterprise M-Series servers, including automatic recoverywith instruction retry, up to 4 TB of system memory error-correcting code (ECC) protection withextended ECC support, guaranteed data-path integrity, total static random access memory (SRAM) andregister protection, configurable memory mirroring, and many more.What’s more, most major system components are redundant and hot-swappable, for increasedavailability and serviceability. This includes processors, memory, disk drives, I/O cards, powersupplies, and more. The Sun SPARC Enterprise M8000 and M9000 servers add the ability to hot-swapCPUs, memory, and the service processors. These systems are able to recover from most hardwarefailures, often with no impact to users or system functionality. Sun SPARC Enterprise M4000, M5000,M8000, and M9000 servers can recover quickly from many component failures, including serious faultssuch as the failure of a CPU or a critical ASIC. In fact, no single hardware component failure prohibitsSun SPARC Enterprise M9000 servers from booting.These innovative CPU designs help Sun SPARC Enterprise M3000, M4000, M5000, M8000, andM9000 servers offer better performance than competing systems. At the same time, these servers offerfull binary compatibility and complete investment protection for owners of previous generations ofSun systems.Architected for Reliability“Our Sun SPARC Enterprise M-Series servers, combined with Solaris OS and Oracle database, offers rock-solidreliability and uptime along with unmatched investment protection and scalability. We reduced our response time perdatabase transaction by 98.6%, a 72x performance boost, and achieved a positive ROI in three months.”— Bill Dougherty, Director of Site Operations, StubHubOracle Solaris is designed for reliability. Built with a small, compact kernel, Oracle Solaris limits thepotential for operating system faults and subsequent platform downtime. In addition, Oracle Solarisestablishes a clear distinction between the kernel, shared libraries, and applications in order to limit theimpact of application failures. Furthermore, the ability to install most patches and other incrementalsoftware updates for Oracle Solaris without taking the system offline helps organizations increaseuptime and ease serviceability.7

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise ComputingThere are many complementary features built into Oracle Solaris, SPARC64 and UltraSPARC T2/T2Plus processors, the M-Series and T-Series servers, and Oracle Solaris Cluster to promote mainframeclass reliability. On all Sun SPARC Enterprise systems, Oracle Solaris Predictive Self Healing andOracle Solaris Cluster enhance reliability. On Sun SPARC Enterprise M-Series servers, DynamicDomains (discussed in the Virtualization section) further improves uptime and availability.Oracle Solaris Predictive Self HealingOracle Solaris Predictive Self Healing software proactively monitors and manages system componentsto help organizations achieve maximum availability of IT services. Predictive Self Healing is aninnovative capability in Oracle Solaris 10 that automatically diagnoses, isolates, and recovers frommany hardware and application faults. This enables business-critical applications and essential systemservices to continue uninterrupted in the event of software failures, major hardware componentfailures, and even misconfigured software. The Oracle Solaris Fault Manager Architecture (FMA) andOracle Solaris Service Manager Facility (SMF) are the two main components of Predictive Self Healing.The FMA, a common system that works across platforms running Oracle Solaris, reduces complexityby automatically diagnosing faults in the system and initiating self-healing actions to help preventservice interruptions. This software helps increase availability by configuring problem components outof a system before a failure occurs—and in the event of a failure, this feature initiates automaticrecovery and application re-start using SMF. The FMA diagnosis engine produces a fault diagnosisonce discernible patterns are observed from a stream of incoming errors. Following diagnosis, FMAprovides fault information to agents that know how to respond to specific faults.The FMA offers comprehensive reliability and availability capabilities on all Sun SPARC Enterprisesystems. For example: CPU ―offlining‖ takes cores and threads (strands) deemed faulty offline. They are recorded andremain offline on reboot until the faulty processor has been replaced, at which point they are madeavailable again. Memory page retirement retires pages of memory marked as faulty. They are recorded and remainoffline on reboot until the faulty memory has been replaced, at which point it is made availableagain.In addition, Sun SPARC Enterprise M-Series servers running Oracle Solaris also provide FMA supporton their service processors, or eXtended System Control Facility (XSCF). This allows the XSCF toreport faults in the system even if there are no domains running. The alerts are in exactly the sameformat as the reports from FMA running in a domain.The SMF facility creates a standardized control mechanism for application services by turning theminto first-class objects that administrators can observe and manage in a uniform way. These servicescan then be automatically restarted if they are accidentally terminated by an administrator, if they areaborted as the result of a software programming error, or if they are interrupted by an underlyinghardware problem. Specifically, SMF enables administrators to do the following tasks easily andefficiently with Sun SPARC Enterprise servers running Oracle Solaris:8

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise Computing Observe and manage system-wide services Identify ―misbehaved‖ or failed services Securely delegate administrative tasks to non-root users Automatically restart failed services in the appropriate order of dependency Persist the enable/disable of services across system upgrades and patches Preserve compatibility with legacy services Automatically configure snapshots for backup, restore, undo Provide consistent configuration handlingPredictive Self Healing offers comprehensive reliability and availability capabilities on all Sun SPARCEnterprise systems.Solaris Memory Page RetirementAs a part of the Oracle Solaris Predictive Self Healing technology framework, the Oracle Solarismemory page retirement (MPR) capability works to isolate memory issues without system interruption.Fault Manager examines hardware on a continual basis, notifying the MPR subsystem of pages in needof retirement. MPR retires memory pages containing correctable errors and relocatable clean pagescontaining uncorrectable errors without interrupting user applications. In addition, MPR can alsoisolate relocatable dirty pages containing uncorrectable errors with limited impact on affected userprocesses and avoids forcing an outage of an entire system. By utilizing MPR on Sun servers, systeminterruption rates can be reduced by as much as 35-40 percent2.Highly Reliable Memory SubsystemsOracle Solaris and Sun SPARC Enterprise servers work together to ensure the reliability of systemmemory. Some Sun SPARC Enterprise M-Series servers offer the following: Memory patrol: Memory patrol periodically scans memory for errors, proactively preventing the useof faulty areas of memory before they can cause system or application errors, improving systemreliability. Memory Extended ECC: The memory Extended ECC function of these servers enables single-biterror correction, enabling processing to continue despite events such as burst read errors that aresometimes caused by memory device failures.2Assessment of the Effect of Memory Page Retirement on System RAS Against Hardware Faults9

Oracle Solaris and Sun SPARC Systems—Integrated and Optimized for Enterprise Computing Memory mirroring: Memory mirroring on the Sun SPARC Enterprise M4000 to M9000 is anoptional, high-availability feature appropriate for execution of applications with the most stringentavailability requirements. Memory mirroring duplicates the data on write and compares the data onread to each side of the memory mirror. In the event that errors occur at the bus or dual inlinememory module (DIMM) level, normal data processing continues through the other memory busand alternate DIMM set.Oracle Solaris ZFS for Reliable Data“Solaris provides a couple of key advantages over any other OS. One is just the base reliability of the operatingsystem with storage, things like retrying I/Os. But on top of that there are two key technologies that, frankly, you can’tget anywhere else That’s MPxIO for multipath I/O and the other is ZFS.”— Jason Williams, CIO, DigiTAROracle Solaris ZFS technology offers a dramatic advancement in data management with a virtualstorage pool design, integrated volume manager, and data services that provide an innovative approacht

SPARC (and x86-based) systems and compatibility with existing applications is guaranteed. This is why there are over 50,000 businesses and institutions running over 11,000 certified applications on Oracle Solaris today. Powering Oracle's Sun SPARC Enterprise servers, Oracle Solaris continues to set world records for