WIXLIB

Redundancy & I/O In the case of Normal redundancy, there will be two write callsfrom the host side (by database). This could potentially be an issue if you go from external tonormal redundancy. ASM tries to keep nearly same number of primary and secondaryextents in each disk(lun). This provides an uniform distribution of I/O activity in all luns. But, ASM does not know anything about striping & mirroring inthe SAN. Double SAME methodology in play, generally. OraInternals Riyaj Shamsudeen29

I/O Errors – Normal redundancy (DB)IS readerror?Read secondaryextentFailure.Signal ASM.Offline tablespace.SuccessCopy from secondaryto primary extentWrite alert and continueFailureSignal ASM to offlinedisk. OraInternals Riyaj ShamsudeenWrite alert and continue30

I/O Errors – Normal redundancy (ASM)Disk offlinemessage or writeerrors in ASMCopy the extent to new Success. Mark original AUAU in the same disk.invalid. ContinueFailureCheck if sufficientpartner disks alive.NoDisk group offlineYesDisk offline, drop (later)and then reblance. OraInternals Riyaj ShamsudeenWrite alert and continue31

Fast Mirror Resync Disk goes offline, if ASM encounters errors. But, in 11g, ASM doesn’t drop the disk for 3.6 hours. After 3.6hours, disks are dropped if it is not available. You can modify disk repair time from 3.6 hours.ALTER DISKGROUP DATA SET ATTRIBUTE 'DISK REPAIR TIME' ‘10H'; Idea here is that transient failures do not trigger massiveresilvering activities. Changes to the extents are tracked in a bitmap, and this bitmap isused to copy the extents once the disks are available. This is truly useful, say, if a controller fails, as the disks are fine. OraInternals Riyaj Shamsudeen32

Failures and corruption ASM also reads only a primary extent normally. This means thatthe corruption in the secondary extent will not be noticeduntil the primary extent is not accessible. But, writes will write to both extents and so, can detectcorruption. Hardware failures will be detected immediately though since eachdisk will have an approximately equal number of primary &secondary extents. OraInternals Riyaj Shamsudeen33

Diskgroup check If there are any disk errors, checking diskgroup might be a firststep to take. Returns with no errors if the disk group is good. Checks for ASM metadata consistency: Verifies file extent maps and allocation tables. Verifies the directories, files, and aliases are correct. Reads metadata and backup, and verifies them. OraInternals Riyaj Shamsudeen34

Use same size luns If a lun fails, then ASM will induce rebalance and will copy theextents from primary or secondary. Database will continue to read from the available mirror and willnot see any errors. For these reasons, it is important to have same size luns in a diskgroup. We will discuss rebalance operation later. OraInternals Riyaj Shamsudeen35

How many disk groups? 2 or 3 (DATA, FRA,CRS) One disk group for database files (say DATA) and another groupfor flash recovery area (say FRA) is the recommended approach. ASM follows SAME methodology. For example, If there are 5disks in a disk group (assuming external redundancy), file will bespread on all the available luns. OraInternals Riyaj Shamsudeen36

Redundant copies If there are two disk groups configured at DB creation time, acontrol file and a redo log file member will be placedautomatically in both disk groups. You could do this manually too, later. Even if you have many database instances using that ASM, still,just 2 or 3 ASM disk group is the recommended approach. There is an exception: If you have tier 1 and tier 2 storagearchitecture, then it makes sense to have more disk groups. OraInternals Riyaj Shamsudeen37

ASM files – Normal redundancy ASM files are allocated from mirrored extents between the failuregroups.ASM file 1v asm file, v asm alias, x kffxpDisk group DATA1Failure group D1v asm disk groupFailure group D2v asm diskDemo: asm file analysis.sql OraInternals Riyaj Shamsudeen38

ASM files – Normal redundancy - Exadata ASM files are allocated from mirrored extents between the failuregroups.v asm file, v asm alias, x kffxpDisk group DATA1cell01Demo: asm file analysis.sqlv asm disk groupcell02 OraInternals Riyaj Shamsudeencell03v asm disk39

Extents vs Files ASM files are allocated as series of extents. ASM extents are made up of one or more allocation units.DB fileASM fileASM extentsASM AU ASM extents are contained within an ASM disk though. OraInternals Riyaj Shamsudeen40

Extent vs AU 1 extent 1 AU up to 20000 extents. 1 extent 8 AUs after 20000 extents. This is one asm file and so extents are distributed between the devices (striping).0481215913 2000020004261014 371115 20001200022000520006 OraInternals Riyaj Shamsudeen 200032000741

Allocation unit (AU) Allocation unit defines a smallest size disk segment that can beallocated, at disk group level. Allocation unit defaults to 1MB. It can be increased in multiplesof 2 i.e. 2,4,8,16MB etc while creating a diskgroup. (11g). Once a disk group is created with an allocation unit it can not bealtered. In 10g, underscore parameters asm ausize can be used tomodify the allocation unit. Increased allocation unit is useful in VLDB daabases. OraInternals Riyaj Shamsudeen42

Striping File extents are striped. There are two types of striping: coarseand fine. With coarse striping, one allocation unit is the size of stripe. Thisis used for database files. With fine striping, 128KB is interleaved with 8 allocation units.This type of striping is used for online redo log files, controlfiles, and spfiles. Striping is controlled by templates. Template can be altered, but be careful of implications.Demo: asm templates.sql OraInternals Riyaj Shamsudeen43

ASM files You don’t need to specify complete file name while creating filefrom the database. ASM will generate a system defined unique file name if you don’tspecify complete path.create tablespace ts small datafile ' DATA' size 10M;select file name from dba data files where tablespace name 'TS SMALL’FILE --------------------------------- DATA/solrac/datafile/ts small.281.764615081Demo: cr ts small.sql OraInternals Riyaj Shamsudeen44

ASM Directory You can create directory structure in ASM and use that for filenames (ASM instance).SQL alter diskgroup data adddirectory ' DATA/app';SQL alter diskgroup data adddirectory ' DATA/app/oracle';A new file with user defined file name can be added to thedatabase.SQL alter tablespace ts small add datafile' DATA/app/oracle/ts small 02.dbf' size 10M; User defined files are simply alias: asmcmd ls -ltTypeRedund' DATA/app/oracle/ts small 02.dbf'StripedTimeSysNameNts small 02.dbf DATA/SOLRAC/DATAFILE/TS SMALL.280.764616401Demo:add directory, al ts small, drop directory, drop tablespace, OraInternals Riyaj Shamsudeen45

Rebalance Addition or deletion of asm disk from a disk group will trigger arebalance operation.Extents are moved fromexisting disks to newdisks, rebalancing thedisk usage. OraInternals Riyaj Shamsudeen46

Processing details RBAL is triggered when there is addition/deletion/resize ofdisks.RBAL acts as a co-ordinator process, updates metadata that ASMrebalance is underway. Determines the extent to move and the target disk. Hands offthe work to ARBx process. ARBx process moves the extent and replies back to RBAL afterthe successful completion. This goes on until RBAL completes the rebalance operation. OraInternals Riyaj Shamsudeen47

Asm power limit Asm power limit controls the speed of rebalance operation. This parameter controls number of ARBx process performingthe rebalance operation. Each ARBx process locks just one extent at a time and moves theextent to another disk. You can increase asm power limit parameter to improverebalance operation speed. It is not uncommon to disable the rebalance during busy hoursand increase the limit to higher value during off hours. OraInternals Riyaj Shamsudeen48

Rebalance miscellaneous Rebalance operation can be manually controlled using ‘alterdiskgroup . Rebalance’; Command will return immediately, rebalance will continueasynchronously. You could specify wait keyword to wait for the rebalance tocomplete. Only one disk group can participate in a rebalance activity at anypoint in time, in an ASM instance. Potentially, rebalance can be triggered in another disk group froma different node. OraInternals Riyaj Shamsudeen49

Adding disks If you are adding many disks, then rebalance only after adding alldisks. For example, to add 10 disks to an disk group, then:(i) Disable asm power limit by setting that to 0 with sid ‘*’.(ii) Add all the disks as you wish.(iii)Enable asm power limit to, say 5, during non-busy hoursand let the rebalance work. V asm operation can be used to monitor rebalance activity. In RAC, only one node will be performing the rebalancingactivity. Improved in 12c. OraInternals Riyaj Shamsudeen50

Migrating from one disk array to another. Same principle applies if you are migrating from one disk arrayto another. High level tasks in the case will be:(i) Set asm power limit to 0(ii) Add the disks from the new array.(iii) Drop the disks from the old array. (Dropping simplymarks them to drop and you can’t drop until rebalancingcompletes).(iv) At this point, set asm power limit to 5 or 10, let therebalance completes the move of extents.(v) After the rebalance you can remove the luns. OraInternals Riyaj Shamsudeen51

V views Access to v asm disk will do a discovery of disks and could becostly. Oracle provides v asm disk stat to do performancemeasurement. V asm disk stat does not do discovery and so much lighter toaccess. V asm diskgroup stat does not asm diskgroup discovery. OraInternals Riyaj Shamsudeen52

asmcmd Asmcmd provides an interface similar to a file system interface. Type ‘asmcmd help’ to see the syntax. Command ‘asmcmd –p’ tosee the current path in asmcmd. In a logical view, ASM Disk groups are the root directories: asmcmdASMCMD ls -lt data/Type ny flags in the ls are supported here too. Use ls –ls to see thesize of files.Demo: asmcmd commands, help etc OraInternals Riyaj Shamsudeen53

Asmcmd cd cd is also supported.ASMCMD cd dataASMCMD ls -ltTypeRedundASMCMD ASMCMD ls RFILE/YONLINELOG/YDATAFILE/YCONTROLFILE/N : asmcmd commands OraInternals Riyaj Shamsudeenspfilesolrac.ora 54

Asmcmd lsdg Other disk group levels commands are also available.ASMCMD lsdgStateTypeRebal Sector BlockOffline disks Voting files NameMOUNTED0ASMCMD EXTERNN512Y4096AUTotal MBFree MBReq mir free MBUsable file MB104857616000498904989DATA/lsofDB NameInstance NamePath ASM ASM2 data.255.4294967295solracsolrac1 csolrac1 csolrac1 lrac1 rac1 data/solrac/datafile/system.256.731448831Demo: asmcmd commands: lsct, lsdg, lsof, du, iostat etc OraInternals Riyaj Shamsudeen55

Asmcmd find Find is also available, very similar to UNIX find utility.ASMCMD find --type DATAFILE DATA * DATA/SOLRAC/DATAFILE/EXAMPLE.267.731449265 DATA/SOLRAC/DATAFILE/SYSAUX.257.731448845 DATA/SOLRAC/DATAFILE/SYSTEM.256.731448831 DATA/SOLRAC/DATAFILE/SYSTEM.275.732461065 DATA/SOLRAC/DATAFILE/TS LMT HW.277.758566281 DATA/SOLRAC/DATAFILE/UNDOTBS1.258.731448847 DATA/SOLRAC/DATAFILE/UNDOTBS2.268.731449831 ASMCMD find --type CONTROLFILE DATA * DATA/SOLRAC/CONTROLFILE/Current.260.731449169 D find --type DATAFILE DATA UNDO* DATA/SOLRAC/DATAFILE/UNDOTBS1.258.731448847 DATA/SOLRAC/DATAFILE/UNDOTBS2.268.731449831Demo: asmcmd commands: lsct, lsdg, lsof, du, iostat etc OraInternals Riyaj Shamsudeen56

Asmcmd cp Cp is another tool to copy from one asm- asm or asm- filesystem.ASMCMD cp SYSTEM.275.732461065 /tmp/copying DATA/SOLRAC/DATAFILE/SYSTEM.275.732461065 - /tmp//SYSTEM.275.732461065 You can also copy the file to a compressed pipe and transmit to adifferent server. ASM instance should be up and running in both sides for ASMto ASM copy to work. Asmcmd cp does not co-ordinate with the database. So, youshould alter the database /tablespace to backup mode before cpoperation.Demo: asmcmd cp OraInternals Riyaj Shamsudeen57

THANK YOU Email: rshamsud@orainternals.comBlog : orainternals.wordpress.com Web: www.orainternals.com OraInternals Riyaj Shamsudeen58

OraInternals Riyaj Shamsudeen 2 Me 23 years as DBA OakTable member Oracle ACE Director Specializes in RAC, performance tuning and Internals. Slowly in to BigData rshamsud@orainternals.com orainternals.wordpres