Hitachi Solution For Databases - Oracle Composable .

Transcription

Hitachi Solution for Databases - OracleComposable Platform with VSP G900 and HitachiAdvanced Server DS220 with NVMe SSDs forDatabase Smart Flash CacheReference Architecture GuideBy Libo JiangMarch 2019

FeedbackHitachi Vantara welcomes your feedback. Please share your thoughts by sending an email message toSolutionLab@HitachiVantara.com. To assist the routing of this message, use the paper number in the subject and the titleof this white paper in the text.Revision HistoryRevisionMK-SL-143-00ChangesInitial releaseDateMarch 13, 2019

Table of ContentsSolution Overview2Business Benefits2High Level Infrastructure2Key Solution Components4Hitachi Virtual Storage Platform G Series Family7Hitachi Storage Virtualization Operating System7Hitachi Advanced Server DS220 Server7Hitachi Advanced Server DS120 Server7Oracle Database Smart Flash Cache8Hitachi Infrastructure Analytics Advisor8Hitachi Storage Advisor9Oracle Enterprise Manager9Hitachi Storage Adapter for Oracle Enterprise Manager9Hitachi Server Adapter for Oracle Enterprise Manager9VMware ESXi9VMware vCenter Server Appliance10Brocade Switches10Cisco Switches10Solution Design10Storage Architecture10Server and Application Architecture16SAN Architecture17Network Architecture18Hitachi Applications21Engineering Validation22Test Methodology22Database Configuration23Test Results24Conclusions33Appendix A – Intel SSD DC P4610 Series Drives Configuration Information for Oracle Database Smart FlashCache33Step 1 - Make Partitions from Intel SSD DC P4610 Series Drives33Step 2 - Setup udev Rules for ASM Disk Management for Intel SSD DC P4610 Series Drive Disk Partitions33Step 3 - Create a Local ASM Disk Group from each Oracle with Disk Portions from Intel SSD DC P4610 SeriesDrives34Step 4 - Enable Database Smart Flash Cache34

1Hitachi Solution for Databases - Oracle Composable Platformwith VSP G900 and Hitachi Advanced Server DS220 with NVMeSSDs for Database Smart Flash CacheReference Architecture GuideUse this reference architecture guide to see how Hitachi Solution for Databases provides a high performance, integrated,converged solution for Oracle using Intel SSD DC P4610 series drives with NVMe interface as Oracle Database SmartFlash Cache. The environment uses Hitachi Virtual Storage Platform G900 (VSP G900), Hitachi Advanced Server DS220with Intel Xeon Scalable processors, and Hitachi Advanced Server DS120 servers with Intel Xeon Scalable processors.With these products, design an Oracle converged infrastructure to meet your requirements and budget.This solution uses Intel SSD DC P4610 series drives in Hitachi Advanced Server DS220 servers as Oracle DatabaseSmart Flash Cache to boost performance primarily for read intensive queries. The dedicated servers run Oracle Database12c R2 with the Oracle Real Application Clusters (RAC) option and use Oracle Linux 7.5 for the operating system. Thisreference architecture document is for you if you are in one of the following roles: Database administrator Storage administrator Database performance analyzer IT professional with the responsibility of planning and deploying an Oracle Database solutionTo use this reference architecture guide, you need familiarity with the following: Hitachi Virtual Storage Platform G900 Hitachi Advanced Server DS220 Hitachi Advanced Server DS120 Storage area networks Intel SSD DC P4610 series drives Oracle RAC Database 12c Release 2 Oracle Automatic Storage Management (Oracle ASM) Hitachi Adapters for Oracle Database Hitachi Storage Adapter for Oracle Enterprise Manager Hitachi Server Adapter for Oracle Enterprise Manager Oracle Linux1

2Note — Testing of this configuration was in a lab environment. Many things affect production environments beyondprediction or duplication in a lab environment. Follow the recommended practice of conducting proof-of-concept testingfor acceptable results in a non-production, isolated test environment that otherwise matches your productionenvironment before your production implementation of this solution.Solution OverviewHitachi Unified Compute Platform Converged Infrastructure (UCP CI) architecture for Oracle Database is engineered, pretested, and qualified to provide high performance and high reliability in demanding, dynamic Oracle environments. Thereference architecture implements Hitachi Unified Compute Platform CI for Oracle Real Application Clusters on two nodesusing Hitachi Virtual Storage Platform G900. This environment addresses the high availability, performance, and scalabilityrequirements for OLTP and OLAP workloads. Based on the existing reference architecture, we developed this newreference architecture using server side NVMe SSDs as Oracle Database Smart Flash Cache.Using Intel SSD DC P4610 series drives as Oracle Database Smart Flash Cache on the server side is an alternative way tooptimize query performance at a lower cost. Generally speaking, using larger quantities of server RAM would cost more.The Intel SSD drives are locally configured on each server. This simplifies the overall architecture design and eliminatesthe cost for the dedicated inter-connections for NVMe SSDs often found in other vendors’ solutions.Business BenefitsThis reference architecture provides the following benefits: Complementary to the current UCP CI for Oracle offering from Hitachi Vantara Relatively lower cost than sever RAM Predictable, repeatable, reliable results that are pre-validated Faster speed to deploy and increased ability to meet changing needs, with a single source for components andprescriptive guides Extreme reliability of Hitachi storage Simplified and centralized storage managementHigh Level InfrastructureFigure 1 shows the high-level infrastructure for this solution.The configuration of Virtual Storage Platform G900 and Hitachi Advanced Server DS220 has the following characteristics: Fully redundant hardware Dual fabric connectivity between hosts and storageTwo Hitachi Advanced Server DS120 servers are being used as management servers in this reference architecture.2

3Figure 1To avoid any performance impact to the production database, Hitachi Vantara recommends using a configuration with thefollowing: A dedicated storage system for the production database A dedicated storage system for storing backup data, if neededUplink speed to the corporate network will be dependent on the customer environment and requirements. The Cisco Nexus93180YC-EX switches can support uplink speeds of 40 GbE or 100 GbE if higher bandwidth is required.3

4Key Solution ComponentsThe key solution components for this solution are listed in Table 1, Table 2, and Table 3.TABLE 1. HARDWARE COMPONENTSHardwareHitachi Virtual StoragePlatform G900Detailed Description12 16 Gbps Fibre Channel A10.H3216 12 Gbps Backend SAS Ports512 GB cache memory64 1.9 TB SSDs4 3 TB 7.2k rpm SAS drivesHitachi Advanced Server 2 Intel Xeon Gold 6140 18C CPU @DS220 servers2.30GHzBMC: 4.23.06256 GB (32 GB 8) DIMM DDR4Synchronous Registered (Buffered) 2666MHz2 Intel Corporation Ethernet ControllerXXV710 for 25 GbE SFP28Driver: i40eVersion: 2.1.14-kFirmware: 6.022 Emulex LightPulse LPe31002-M6 2Port 16 Gb Fibre Channel AdapterBoot: 11.4.204.34Firmware:11.4.204.34Driver: 11.4.0.72 6.4TB Intel SSD DC P4610 seriesdrivesFirmware:VDV101404

5TABLE 1. HARDWARE COMPONENTSHardwareDetailed DescriptionHitachi Advanced Server 2 Intel Xeon Sliver Processor 4110, 8DS120 servercore, 2.1 GHz, 85W256 GB (32 GB 8) DIMM DDR4Synchronous Registered (Buffered)2666MhzFirmware/VersionBIOS: 3A10.H8Quantity2BMC: 4.23.06CPLD:101 64 GB SATADOM1 Intel Corporation Ethernet ControllerXXV710 for 25 GbE SFP28Driver: i40eVersion: 1.5.6Firmware: 6.0.11 Emulex LightPulse LPe31002-M6 2Port 16 Gb Fibre Channel AdapterFirmware:11.4.204.27Driver: 11.4.341.0Brocade Fibre ChannelSwitchesG620Kernel: 2.6.34.648 port Fibre Channel switchFabric OS: v8.2.0b216 Gbps SFPsBrocade hot-pluggable SFP , LCconnectorCisco Nexus93180YC-EXBIOS: version 07.61 248 10/25 GbE network portsNXOS: version7.0(3)I4(7)6 40/100 Gbps Quad SFP (QSFP28)portsCisco Nexus3048TPBIOS: version 4.0.01 GbE 48-Port Gb Ethernet SwitchNXOS: version7.0(3)I4(7)1Note that spare drives are recommended for production environments.1.6 TB or 3.2 TB Intel SSD DC P4610 series drives can be used as well. 6.4 TB drives were available for lab verification.Certain components may be optional depending on the existing infrastructure and required interconnect topology. This mayinclude the SAN, IP switches, and/or the management servers.5

6However, this documents the environment tested in the lab to support a full deployment of the architecture includingsupporting components.TABLE 2. SOFTWARE COMPONENTS FOR COMPUTE NODESSoftwareVersionFunctionOracle Linux7.5Operating SystemOracle 12c12c Release 2 (12.2.1.0)Database SoftwareOracle Real Application Cluster12c Release 2 (12.2.1.0)Cluster SoftwareOracle Grid Infrastructure12c Release 2 (12.2.1.0)Volume Management, File SystemSoftware, and Oracle AutomaticStorage ManagementTABLE 3. SOFTWARE COMPONENTS FOR MANAGEMENT NODESSoftwareVersionFunctionVMware ESXi6.7.0 Build 10302608ESXi for management nodesVMware vCenterServer6.7.0 Build 10302608Management clusterHitachi StorageAdvisor (HSA)2.3Storage orchestration softwareHitachi InfrastructureAnalytics Advisor(HIAA)4.2.0-01Analytics SoftwareManager for Hitachiadapters for OracleDatabase2.2.3Hitachi adapters managementHitachi StorageAdapter for OracleEnterprise Manager2.2.3Virtual appliance softwareStorage management softwareHitachi Server Adapter 2.2.3for Oracle EnterpriseManagerServer management softwareOracle EnterpriseManager CloudControl 13c13c Release 2 (13.2.0.0)OEM softwareOracle EnterpriseManager CloudControl 13c plug-ins13c Release 2Hitachi Storage and Server OEM plugins6

7Hitachi Virtual Storage Platform G Series FamilyThe Hitachi Virtual Storage Platform G series family enables the seamless automation of the data center. It has a broadrange of efficiency technologies that deliver maximum value while making ongoing costs more predictable. You can focuson strategic projects and to consolidate more workloads while using a wide range of media choices.The benefits start with Hitachi Storage Virtualization Operating System RF. This includes an all new enhanced softwarestack that offers up to three times greater performance than our previous midrange models, even as data scales topetabytes.Virtual Storage Platform G series offers support for containers to accelerate cloud-native application development.Provision storage in seconds, and provide persistent data availability, all the while being orchestrated by industry leadingcontainer platforms. Move these workloads into an enterprise production environment seamlessly, saving money whilereducing support and management costs.Hitachi Virtual Storage Platform G900, used in this reference architecture, supports Oracle Real Application Clusters.Hitachi Storage Virtualization Operating SystemHitachi Storage Virtualization Operating System (SVOS) spans and integrates multiple platforms. It integrates storagesystem software to provide system element management and advanced storage system functions. Used across multipleplatforms, Storage Virtualization Operating System includes storage virtualization, thin provisioning, storage service levelcontrols, dynamic provisioning, and performance instrumentation.Storage Virtualization Operating System includes standards-based management software on a Hitachi Command Suite(HCS) base. This provides storage configuration and control capabilities for you.Hitachi Advanced Server DS220 ServerHitachi Advanced Server DS220 is a high-performance two-socket rackmount server designed for optimal performanceand power efficiency. This allows owners to upgrade computing performance without overextending power consumptionand offers non-latency support to virtualization environments that require the maximum memory capacity. Hitachi AdvancedServer DS220 provides flexible I/O scalability for today’s diverse data center application requirements. DS220 serverssupport SSD drives with an NVMe interface. These were used in this reference architecture for Database Smart FlashCache.Hitachi Advanced Server DS120 ServerOptimized for performance, high density, and power efficiency in a dual-processor server, Hitachi Advanced Server DS120delivers a balance of compute and storage capacity. This rack mounted server has the flexibility to power a wide range ofsolutions and applications.The highly-scalable memory supports up to 3 TB RAM using 24 slots of 2666 MHz DDR4 RDIMM. DS120 is powered bythe Intel Xeon scalable processor family for complex and demanding workloads. There are flexible OCP and PCIe I/Oexpansion card options available. This server supports up to 12 small form factor storage devices with up to 4 NVMe.7

8The following applications were installed in individual virtual machines in this architecture and would be installed in mostcases. vCenter Oracle Enterprise Manager (OEM) 13c Hitachi Storage Adapter for Oracle Enterprise Manager Hitachi Server Adapter for Oracle Enterprise Manager Oracle Adapter Manager Hitachi Storage Advisor (HSA) Hitachi Infrastructure Analytics Advisor / Hitachi Datacenter Analytics (HIAA/HDCA) HDCA ProbeOther management applications may be installed on additional virtual machines depending on customer needs andrequirements.Oracle LinuxOracle Linux (OL, formerly known as Oracle Enterprise Linux) is a Linux distribution packaged and freely distributed byOracle, available partially under the GNU General Public License. It is compiled from Red Hat Enterprise Linux sourcecode, replacing Red Hat branding with Oracle branding.Oracle Database With the Real Application Clusters OptionOracle Database has a multi-tenant architecture so you can consolidate many databases quickly and manage them as acloud service. Oracle Database also includes in-memory data processing capabilities for analytical performance. Additionaldatabase innovations deliver efficiency, performance, security, and availability. Oracle Database comes in two editions:Enterprise Edition and Standard Edition 2.Oracle Real Application Clusters (Oracle RAC) is a clustered version of Oracle Database. It is based on a comprehensivehigh-availability stack that can be used as the foundation of a database cloud system, as well as a shared infrastructure.This ensures high availability, scalability, and agility for any application.Oracle Automatic Storage Management (Oracle ASM) is a volume manager and a file system for Oracle database files.This supports single-instance Oracle Database and Oracle Real Application Clusters configurations. Oracle ASM is therecommended storage management solution that provides an alternative to conventional volume managers, file systems,and raw devices.Oracle Database Smart Flash CacheOracle Database Smart Flash Cache feature is a transparent extension of the database buffer cache using solid statedevice (SSD) technology. Database Smart Flash Cache can improve the performance of Oracle databases by reducing theamount of disk I/O at a lower cost than adding an equivalent amount of RAM. Database Smart Flash Cache is supported onOracle Linux and Solaris operating systems only.Hitachi Infrastructure Analytics AdvisorWith Hitachi Infrastructure Analytics Advisor, you can define and monitor storage service level objectives (SLOs) forresource performance. You can identify and analyze historical performance trends to optimize storage system performanceand plan for capacity growth.8

9Use Hitachi Infrastructure Analytics Advisor to register resources (storage systems, hosts, servers, and volumes), and setservice-level thresholds. You are alerted to threshold violations and possible performance problems (bottlenecks). Usinganalytics tools, you find which resource has a problem and analyze the cause to help solve the problem. The InfrastructureAnalytics Advisor software ensures the performance of your storage environment based on real-time SLOs.Hitachi Storage AdvisorHitachi Storage Advisor is an infrastructure management solution that unifies storage management solutions such asstorage provisioning, data protection, and storage management; simplifies the management of large scale data centers byproviding smarter software services; and is extensible to provide better programmability and better control.Oracle Enterprise ManagerOracle Enterprise Manager provides a“single pane of glass”that allows you to manage on-premises and cloud-based ITusing the same familiar interface you know and use every day. Oracle Enterprise Manager is the nerve center of IToperations among thousands of enterprises. Millions of assets in Oracle SaaS and PaaS public cloud operations aremanaged by Enterprise Manager around the clock.Enterprise Manager is the industry’s first complete cloud solution with Cloud Management. This includes self-serviceprovisioning balanced against centralized, policy-based resource management, integrated chargeback and capacityplanning, and complete visibility of the physical and virtual environments from applications to disk.This solution uses Oracle Enterprise Manager Cloud Control, version 13c release 2. This allows you to use these cloudmanagement features: Use the Database Cloud Self Service Portal Benefit from the Improved Service Catalog Perform Snap Cloning using ”Test Master Snapshot” Take advantage of the Chargeback and Consolidation Planner pluginsFor more information, see New Features in Oracle Enterprise Manager Cloud Control 13cHitachi Storage Adapter for Oracle Enterprise ManagerHitachi Storage Adapter for Oracle Enterprise Manager presents an integrated, detailed view of the Hitachi storagesupporting your Oracle databases. By gaining visibility into capacity, performance and configuration information,administrators can manage service levels more effectively, and ensure service level agreements (SLAs) are met to supportbusiness goals.Hitachi Server Adapter for Oracle Enterprise ManagerHitachi Server Adapter for Oracle Enterprise Manager is an Oracle Enterprise Manager plug-in that enables monitoring ofHitachi Advanced servers in Oracle Enterprise Manager.For Hitachi Advanced servers, it provides visibility into the components, including their status, health, and attributes. Inaddition, the adapter supplies information about any Oracle database instances running on the servers. Both RAC and nonRAC databases are supported.VMware ESXiVMware ESXi is the next-generation hypervisor, providing a new foundation for virtual infrastructure. This innovativearchitecture operates independently from any general-purpose operating system, offering improved security, increasedreliability, and simplified management. This refence architecture uses VMware ESXi for the management servers only.9

10VMware vCenter Server ApplianceThe VMware vCenter Server Appliance is a preconfigured Linux virtual machine, which is optimized for running VMwarevCenter Server and the associated services on Linux.vCenter Server Appliance comes as an Open Virtualization Format (OVF) template. The appliance is imported to an ESXihost and configured through the web-based interface. It comes pre-installed with all the components needed to run avCenter Server, including vCenter SSO (Single Sign-on), Inventory Service, vSphere Web Client and the vCenter Serveritself.This reference architecture uses VMware vCenter Server Appliance for the management server cluster only.Brocade SwitchesBrocade and Hitachi Vantara partner to deliver storage networking and data center solutions. These solutions reducecomplexity and cost, as well as enable virtualization and cloud computing to increase business agility.SAN switches are optional and direct connect is also possible under certain circumstances, but customers should checkthe support matrix to ensure support prior to implementation.The

Cisco Nexus 93180YC-EX 48 10/25 GbE network ports 6 40/100 Gbps Quad SFP (QSFP28) ports BIOS: version 07.61 NXOS: version 7.0(3)I4(7) 2 Cisco Nexus 3048TP 1 GbE 48-Port Gb Ethernet Switch BIOS: version 4.0.0 NXOS: version 7.0(3)I4(7) 1 TABLE 1. HARDWARE COMPONENT