WIXLIB

Managing the Manager: Tips andTricks for Maintaining OracleEnterprise Manager 12SubtitleAaron RimelProduction Owner Systems ArchitectSeptember 23, 2013

AgendaAbout Enterprise Manager at CernerOur Enterprise Manager TeamMonitoring Enterprise ManagerPlanned Maintenance Activities 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

About Enterprise Manager atCerner

About MeDBA with over 10 years withOracle Database, 9i-11gExtensive usage of EnterpriseManager for the last what feelslike 150 years, but is more like 7.Worked with Oracle StrategicCustomer Programs, EM 12c EarlyAdopter and participate inCustomer Advisory BoardPast presentations at OpenWorldand recent IOUG Webcast 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Cerner at a GlanceLargest Strategic Global Health Care IT CompanyFounded in 1979, headquartered in Kansas City, MOSupports:10,000 Facilities Worldwide2,600 Hospitals3,500 Physician Practices (30,000 Physicians)500 Ambulatory Facilities800 Home-Health Facilities1,600 Retail PharmaciesR&D Commitment and Clinical CultureComprehensive Suite of Health Care Solutions & ServicesOver 1,000 CliniciansOver 2.5B of Cumulative R&DContemporary, Scalable Solution Architecture10,000 Associates Worldwide3,000 Employed in Professional & Managed Services2,000 Employed in Intellectual Property (IP) Organization 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

The Business Case - Beyond our Core OfferingWe want to change the way Ourclients think about IT in theirhealthcare environment.Reduce costs and maintenanceIncrease stability and capabilityRefocus on their core . . .HealthcareManaged Servicesalready has:Size and ScaleSecurityTechnology ExpertiseProven Quality DeliveryHealthcare FocusIt makes senseto go beyond 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Foundation for new opportunity: OEM as a ServiceIntegrated central repository for enricheddatabase management:Automated toolkitCustom compliance policiesSelf-healing actions capabilitiesReduced Capital ExpenseReduced hardware capital by 9.5MIncreased DBA Efficiency12% client base growth with no additional staffImproved Stability & Performance50% decrease in database incidents as aresult of preventative actions17% of problems are now identified withoutDBA omes 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Our Baby 22,000 Targets, 950 AgentsCerner hosted targets as wellas OEM as a Service100 Custom MetricExtensions50 Custom ComplianceStandards 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Host Metric Examples Diff check:CPU# of Filesystems onlineMemoryASM LunsHugepages CheckHost uptimeCompliance StandardsPatchInit Settings 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Database Metric Examples AWR StandardsAWR Monitoring: Buffer gets, Gets per exec, Disk Reads,Top 5Failed RMAN BackupsBlock Change Tracking StatusStat’s Collection StatusSQL Tuning Advisor Results is compounded and viewedGloballyAuto adding datafile and datafile auditing 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Enterprise Manager 12c Architecture 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Enterprise Manager Version History12.1.0.312.1.0.111.1.0.1 August 2013 April 2012 201110.2.0.5 200910.2.0.3 2006 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Our Enterprise Manager Team

If you really want to use EnterpriseManager to its full potential, this is not a1-man (OR WOMAN) job.

Cerner’s Enterprise Manager Staffing ModelNo longer just a DBA toolOur team now consists of 6 individuals who shareresponsibility and knowledgeSolution Support is Owned by the CernerWorks Emerging TechnologyServices group, a remote hosting division of CernerI am the Client Owner and Production Owner Drive development and manage day to day activitiesCombined knowledge of Oracle Database, WebLogic Server, OraclePatching Process, OS ExperienceNobody’s full-time job 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

EM Staffing Roles and ResponsibilitiesArchitectureSystem Architecture andIntegrationFuture direction and TestingInstallationOMS PatchingAdvanced Troubleshooting andAvailabilityDiscovery EngineerTypically India associatesPerform target discoveries andend user managementSystem EngineersBasic Agent and OMStroubleshootingOMS PatchingPlug-in maintenanceSelf-Updates (patches, plug-ins, etc.)Metric / Compliance StandardsDevelopmentTwo individuals script, DBA groupvalidatesIncludes custom metrics, reporting,and monitoring best practices 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

EM Virtual Roles and ResponsibilitiesRemedy knowledge for ticket integrationNetwork, firewall, VPN, Bluecoat administratorsSoftware load balancer configurationsEM Security for Groups, Roles, Administrator segregation tocontrol access for non-Cerner usersActive Directory administratorsSales associates for external customersDBA & Middleware teamsSystem AdminsAlarm Management Group for SMNP Trap Configuration 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Target Owner ResponsibilitiesNon-EM AssociatesAssigned specific “clients” for which they are responsible for theavailability, monitoring and configuration.Non-EM associates are responsible for the installation andconfiguration of the agent and related targets.EM AssociatesInitial discovery, upgrade, and patching of agents.Our India EM engineers are responsible for the creation andmaintenance of users. We leverage Active Directory for internalusers, our external users are not yet integrated. 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Monitoring Enterprise Manager

What I’ve Learned About Managing EM The more you rely on EM andthe more you use it Thelouder people scream whenit’s down or not performing!Following are mysuggestions to keep youinformed and aware 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Notifications About EM ItselfCreate an Incident Ruleset to monitor the EM components(OMS and Repository target)Your EM Support Team should get theseThresholds may need to be adjusted based on your site size and activityWe integrate with SNMP traps and email notificationsMOS Note 1553342.1 - Oracle EnterpriseManager 12c Configuration Best Practices 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Out-of-Bound NotificationsIf Enterprise Manager goes down, do you know before yourDBAs call you?Out-of-Bound Notifications via an email, script, and/or trapwhen fatal conditions occurSingle OMS environment, if the OMS is down, but the Agent is upMulti-OMS environment, if all OMS’s are down, but the Agent is upIf Repository database is unavailable (down, archive hung, listenerdown, etc)MOS Note 1472854.1 - How To Setup Out OfBound Email Notification In 12c 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Manage Cloud Control PagesHealth OverviewStatus and Console ActivityJob SystemNotification BacklogUpload Performance/BacklogBackoff RequestsRepositoryRepository details, session breakdownScheduler Status, Job queueRepository Collection backlogRepository Job StatusManagement AQ Status 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Manage Cloud Control PagesAgentsAgent status (pending, blocked, up)Agent version, countAgent control and configuration Start/stop agents Edit agent configuration propertiesManagement ServicesDashboard of incidentsLoader ThroughputCPU/Heap Usage 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Database PerformanceTop Activity (database application waits/cluster waits)CPU/Memory utilizationADDM/AWRSQL MonitoringStatistics 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Health ReportsPre-defined reports can be scheduled and checkeddaily/weekly to look for increasing trends 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.

Know Your DataOMS and Repository /Monitoring /All MetricsMetrics about EM collected andstored for your enjoymet!The more you know what youraverage values are, the better youcan adjust thresholds and knowwhen there’s a problem. 2013 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information which may not be reproduced or transmitted without the express written consent of Cerner.