WIXLIB

Oracle BackupsBenchmarked and Best PracticesRyan BehielSolution Architect

Agenda An Oracle Backup Performance Retrospective Key Oracle Backup Concepts – a Few Simple Things you Should know NetBackup Copilot for Oracle - Overview Testing Performance Limits (AKA Don’t Shoot the Messenger!) Oracle Database Performance Enhancing Configuration Suggestions Copilot For Oracle Performance Numbers Q&A2Copyright 2017 Veritas Technologies LLC

An Oracle Performance Backup Retrospective3Copyright 2017 Veritas Technologies LLC

April 15, 2002“eWeek Labs and engineers fromVeritas Software Corp.'s labs, inMountain View, Calif., recentlycompleted a high-end Oraclebackup performance test ”“eWeek Labs worked directlywith Veritas engineering on thisbenchmark.”4Copyright 2017 Veritas Technologies LLC

Veritas Oracle Backup Benchmark Hardware ConfigurationBenchmark Hardware le9iSTK L700 FibreBrocade 3800/32002 Gb Switches11L700 ControlPathBackup ServerSun Micro 680024 750MHz CPUs25GB of RAMDisk StorageSTK 9176B (x2)2.88 TB (each)Tape DrivesSTK 9840B (x24)Tape LibrariesSTK L700 (x2)2 Gb Fibre SwitchBrocade (x6)2 Gb Fibre HBAEmulex (x31)Est. HW Cost 1.9MM (2002 ’s)2311STK L700 Fibre4VERITASNetBackup 4.5Sun Fire6800512Zoned3FibreChannelCopyright 2017 Veritas Technologies LLC

A total of 31 Emulex 2 Gb Fibre cardswere required to connect all this hardwaretogether.In 2002, 2Gb Fibre was relatively new.The first batch of 2 Gb cards were fromJNI. We found a hardware defect thatonly occurred when restoring data. All 31cards had to be replaced and the zoningcompletely reconfigured due to newWWN’s.This defect was very costly to JNI.6Copyright 2017 Veritas Technologies LLC

6 Brocade 2 Gb Fibre switches wereneeded to connect the massive amountof equipment together.7Copyright 2017 Veritas Technologies LLC

StorageTek (now part of Oracle) loaned us 24STK 9176B tape drives for this benchmark.This was the only tape drive that offered acombination of two critical performance criteria:– 5 second tape load speed– 20 MB/sec backup performance (native)There were faster tape drives but most drives ofthe day took over a minute to load. Achieving a2 TB benchmark number would have beenimpossible if it took 24 minutes to simply load allof the tape drives.8Copyright 2017 Veritas Technologies LLC

Benchmark ResultsDrum roll, please “It took us 1 hour to back up 2 terabytes of data.”To put this in perspective, a few months before this benchmark, anOracle Corporation benchmark was only able to achieve 1TB/hour.Let’s use the lessons learned from this type of benchmark to help uscreate the fastest and most efficient Oracle backup environment.9Copyright 2017 Veritas Technologies LLC

The NetBackup Appliance – Performance PowerhouseOracle 2TB/hr Benchmark (2002)10NetBackup 5330 (2017)315U (7.5 Racks)Rack Units14U2TBBackup Storage687TB2TB/hrBackup Performance7TB/hr 1.9MMPricePricelessCopyright 2017 Veritas Technologies LLC

Key Oracle Backup ConceptsA Few Simple Things You Should Know11Copyright 2017 Veritas Technologies LLC

Key Oracle ConceptsRMAN: Recovery Manager. Oracle created backup API. Introduced in Oracle 8Online redo logs: Files that store changes to database as they occur.Archived redo logs: Online redo logs that are transitioned to an offline stateDatafile: The majority of Oracle data is stored inside of datafilesOracle Restore: Physical recovery (e.g. datafile) of databaseOracle Recovery: Logical recovery of database to point in time using redo logsBCT: Block Change Tracking. When enabled, makes Oracle incrementalbackups faster12Copyright 2017 Veritas Technologies LLC

NetBackup Copilot for OracleQuick Overview13Copyright 2017 Veritas Technologies LLC

Backup Processing Overview – Copilot for Oracle2NetBackup automaticallysweeps the Oracle backupdata into NetBackup shareOracleServerNetBackup DeduplicationApplianceOracle RMAN script isautomatically generated anddatabase backup is1scheduled and run byNetBackup14Copyright 2017 Veritas Technologies LLC3NetBackup knows exactlywhich DB the backup imagesbelongs to, how long to retainimages, and automaticallymanages the database diskbackup location (MSDP,Tape, etc.)

Testing Environmental Performance LimitsDon’t Shoot the Messenger15Copyright 2017 Veritas Technologies LLC

Baseline Performance Testing“The lesson here: Analyze the entire data path when designing backupsystems. You'll be wasting money on high-end tape and disk hardware ifthe systems and network between them are slow.” Baseline performance testing is arguably the most important step in theprocess of optimizing the backup performance of your environment Baseline testing determines performance characteristics of hardware externalto the NetBackup Appliance Focus on the network, SAN and Oracle environment– The performance of the NetBackup Appliance is thoroughly tested and documented16Copyright 2017 Veritas Technologies LLC

Baseline Performance Testing – Oracle Database ServerNetBackup DeduplicationAppliance Basic performance metricsOracleServer– 10 GbE 600 MB/sec (nominal)– 8 Gb Fibre 600 MB/sec (nominal)– Bonding increases throughput but not 100% linear I/O Capacity– So many variables associated with I/O– Only true way of determining I/O capabilities is through testing of every storage component17Copyright 2017 Veritas Technologies LLC

Baseline Performance Testing – Oracle Database Server1For standard Oracle clientand Copilot for Oraclebackups, the NetBackupOracle agent will be installedagent1NetBackup DeduplicationApplianceUse the NetBackup agent for basic perf testing1 1OracleServerBackup simple (non-Oracle) data from all storage connected to the Oracle server– This is a safe, non-destructive test– Configure a non-Oracle policy for single and multiple stream backups– Document backup performance achieved as streams are added– Likely to saturate performance with 2 or 3 streams– This test will give you an indication of achievable Oracle backup performance18Copyright 2017 Veritas Technologies LLC

Baseline Performance Testing – Oracle Database ServerAdditional baseline performancetesting utilities:RMAN BACKUP VALIDATE command––––Run this from the DB serverNon-destructiveDesigned to validate datafile blocksAlso provides a good indicator of I/O capacityassociated with datafiles– Have DBA run this if backup performance is in questionIOMETER– Open source I/O testing tool– Non-destructive– Provides detailed analysis of all I/O patterns19Copyright 2017 Veritas Technologies LLCOracleServer

Oracle Database Performance EnhancingConfiguration Suggestions“An effective backup strategy must be tailoredto the type of data being backed up.”20Copyright 2017 Veritas Technologies LLC

Oracle Database Performance Enhancements Enable Block Change Tracking (BCT)– Significantly reduces I/O impact of incremental backups– Without BCT, 100% of every datafile is scanned during every backup– Added impact on Oracle database is negligible Limit number of full backups– Use differential incremental backups– Oracle’s Incremental Merge can help (more on this later) More datafiles enables better backup parallelism– Typical to have 1 backup stream per datafile– This can impede backup performance if number of datafiles is small– You have little impact on this setting but you can discuss with DBA21Copyright 2017 Veritas Technologies LLCOracleServer

Copilot for Oracle Performance Enhancers Optimized Share Reserve provides optimal performance– I/O is load balanced across entire shelf (6 LUNS)– Minimize I/O contention by scheduling the “sweep” portion of the dump and sweep activityafter all backups have completed. NetBackup buffers– Default setting is generally good– For some environments, adjustments may be necessary– Might be worth looking at if performance is subpar Is the NetBackup Appliance oversubscribed?– The Appliance is extremely scalable– Any hardware has limits22Copyright 2017 Veritas Technologies LLCNetBackup DeduplicationAppliance

Copilot For Oracle Performance Numbers“Our primary concern for thistest was performance ”23Copyright 2017 Veritas Technologies LLC

NetBackup Oracle Agent vs. NetBackup Copilot for OracleThis test is a comparison of two backup methods– Standard Oracle agent– Copilot for OracleDatabase size 15TBBackup Method (Full)With Copilot, DBAs areempowered to perform adhoc backups, process allrestore requests and areprovided comprehensivereporting through the RMANcatalogThroughputCPU Load - DB ServerNetBackup Oracle Agent3.04 TB/hr8.01%NetBackup Copilot – Std Share2.84 TB/hr5.74%NetBackup Copilot - OSR4.12 TB/hr8.8%Recommendation: OSR offers optimal performance while offloadingcomplicated RMAN tasks to Oracle DBA24Copyright 2017 Veritas Technologies LLC

NetBackup Copilot for Oracle Concurrent ThroughputThis test measures concurrent throughput of multiple DB backupsto an Optimized Share ReserveMultiple Oracle databases configured on single serverBackup Method (Full)Aggregate ThroughputCPU Load - DB ServerSingle DB backup4.12 TB/hr8.8%2 Simultaneous DB Backups7.0 TB/hr8.06%3 Simultaneous DB Backups7.2 TB/hr9.12%Recommendation: Two simultaneous backups from single DB server deliversoptimal mix of aggregate performance and load on DB server25Copyright 2017 Veritas Technologies LLC

Copilot for Oracle Incremental Merge Incremental Merge technology was developed by Oracle– A.K.A. Incremental Update A logical full backup without the impact of a full backup– Significantly reduces backup impact on the Oracle server– Reduces network overhead & provides significant storage savings NetBackup engineers developed a method of combining Oracle’sIncremental Merge technology with the Copilot for Oracle feature The result is a unique “incremental forever” Oracle backup integrated with theCopilot for Oracle “backup to disk” method Implementation is simple– Entire process is handled by NetBackup Copilot for Oracle and completely automatic26Copyright 2017 Veritas Technologies LLC

NetBackup Copilot for Oracle Incremental MergeMeasure the performance advantages that incremental merge provideDatabase size – 10 TBBackup MethodBackup DurationCPU Load - DB ServerFull3:02 (hh:mm)8.83%Incremental Merge – OLAP (0%)4:00 (mm:ss)---Incremental Merge – OLTP (3.2%)7:01 (hh:mm)1.52%Recommendation: Effectiveness of Incremental Merge highly dependent onchange rate in Oracle DB. Oracle has documented this.27Copyright 2017 Veritas Technologies LLC

Copilot for Oracle Instant Recovery“No matter how fast, how big or how much is spent on abackup implementation, none of it matters if you can't restorethe data accurately and quickly.”28Copyright 2017 Veritas Technologies LLC

Copilot for Oracle Instant Recovery Process (NBA 3.1)1NetBackup DeduplicationAppliance321 Oracle “backup to disk” image is cloned1.––No changes to original backup image ever occurAny RMAN full/incr backup images located on backup disk can be used2 Cloned image is exported to Oracle server over NFS2.–Oracle redo log group is cleared3 Oracle database is mounted, open and fully accessible3.29Copyright 2017 Veritas Technologies LLC

Copilot for Oracle Instant Recovery - Backup To Disk and Sweep Instantly recover Oracle database– Full backup– Incremental backup– Incremental merge backupOracle Redo Log Cleared?Database provisioned,mounted and openNo31min 37secYes4min 46 secRecommendation: Understand the impact that the Oracle reset log processcan have on overall instant recovery performance30Copyright 2017 Veritas Technologies LLC