Cisco Tidal Enterprise Scheduler

Transcription

Getting Started withCisco Tidal EnterpriseScheduler 6.2May 2014

Meeting Logistics Attendees will be placed on mute upon joining Please use the Chat panel if you have any technical difficulties withaudio or in viewing the presentation Please use the Q&A panel to ask questions Session will be recorded for offline reference Slides used in this session will be shared over email after the session 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential2

AgendaTES 6.2 UpdatesWhat’s new in TES 6.2TES 6.2 Early Field Trial UpdateArchitecture UpdateRecap on TES 5.3.1 to 6.1 Architecture ChangesTES 6.2 Architecture ChangesPlanning your TES 6.2 ImplementationTES 6.2 Infrastructure RequirementsUpgrade/migration scenariosBest Practices and RecommendationsReference documentsKey Takeaways 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential3

TES 6.2 Updates 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential4

TES 6.2 Release Drivers Expand on Private and Public Cloud Automation capabilities with a newadapter for UCS Manager Improve performance and user experience of the web application Address key enhancements to meet large organizational needs Provide a Java client to deliver desktop client performance 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential5

What is new in TES 6.2New Adapter and Updates UCS Manager Adapter – stop/start, restart Roll-up of support for latest versionsLightweight Web-deployable Java ClientWeb Client Perf & Usability ImprovementsTransporter ImprovementsSupport for multi-domain AD/LDAP authenticationRuntime users for TES workgroupsAbility to terminate user sessionsCM install process to utilize external cacheAPI Guide Update 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential6

TES 6.2 EFT Update TES 6.2 Early Field Trial – a program to solicit early customer feedback Select customers and partners were invited to participate Program ran for 6 weeks and close to 25 unique issues were reported Engineering scrubbed the reported items and critical items are getting fixed for GArelease Key Improvements based on EFT feedback: Java Client data reading has been optimized to reduce memory and initial load timeSolution for RPC error on the web client 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential7

AD/LDAPTES 6.2 Java ClientJavaClientPrimaryMaster Connects directly to the Master through the same APIs that CM uses The Java client also syncs data from the Master, but it is many times faster than the CM because alldata is stored into memory rather to an external database. Most DBs syncs within seconds Smooth scrolling with zero latency since data is rendered from memory (i.e. no paging delay) Faster search and filtering as well since data is queried from memory instead of remotely from anexternal database Can run as a standalone Java app or run through a browser, such as Firefox, IE Java client updates will be deployed through the Master Java client requires JRE 1.7 and above“I also want to mention that the Javaclient is working very good. It’s as fast asthe 5.3.1 thick client, if not faster.”- Feedback from EFT 2013-2014 Cisco and/or its affiliates. All rights reserved.“The agent list window is vastlyimproved. The FX client is much, much,faster than the web client”- Feedback from EFTCisco Confidential8

TES 6.2 Java Client BaselineJobDefinitionSub Tasks &Number ofRecordsExpand All 25K records50K records0-1 secCollapse All 25K records50K records0-1 sec0-1 sec0-1 secScrolling(with 25 K) Top to BottomBottom to TopInstantaneousScrolling(with 50 K) Top to BottomBottom to TopFilter Show GroupUncheckedShow GroupsChecked 2013-2014 Cisco and/or its affiliates. All rights reserved.5.3.1 (40Krecords)6.2 JavaClient(56Krecords)JobActivitiesSub Tasks &Number ofRecords5.3.1(40Krecords)6.2 JavaClient(50Krecords) 25K records50K records0-1 sec0-1 sec0-1 sec0-1 secCollapse All 25K records50K records0-1 sec0-1 sec0-1 sec0-1 secInstantaneousScrolling(with 25 K) Top to BottomBottom to neousScrolling(with 50 K) Top to BottomBottom to TopInstantaneousInstantaneous0-1 sec0-1 secFilter 2-3 sec0-1 sec0-1 sec0-1 secShow Jobs(Expanded)Show Jobs(Collapsed)Show GroupsShow Jobs &Show Groups1-2 sec0-1 sec1-2 sec1-2 sec0-1 sec0-1 sec0-1 sec0-1 secExpand All Cisco Confidential9

2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential10

Java Client Release PhasesPost TES 6.2 – July Drop (Phase 2)TES 6.2 GA (Phase 1) Administrative Screens Agent Jobs Business Views Oracle Apps Adapter Operations Screens JDBC Adapter Reports Oracle DB Adapter Web Services Adapter Amazon S3 Adapter Amazon EC2 Adapter Business Objects DSAdapter PeopleSoft Adapter VMWare AdapterUCS AdapterHive AdapterMS SQL DB Adapter SSH Adapter Business Objects BI Adapter Business Objects AdapterMap Reduce AdapterData Mover JDE AdapterRemote Job AdapterHP Open View Informatica Adapter SAP Adapter Cognos Adapter Email Adapter JMS Adapter Job Activities Events Alerts Logs Master Status SchedulesDefinitions Job Definitions Calendars Actions Events Job Classes Variables Agent Lists Queues Resources Fiscal Calendars 2013-2014 Cisco and/or its affiliates. All rights reserved.Sqoop AdapterCisco Confidential11

What does this mean? Users can continue to view and monitor the jobs for even the adapters that are notsupported in Phase 1. However, they won’t be able to edit the jobs as the Adapter specificTab in the Job Definition and Job Activity console will be missing Web Client will continue to have full support for all the adapters 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential12

Web Client Improvements Configurable option for Auto refresh intervalBulk operation improvements (operation tune up as well as UI feedback)JVM tuning on Client ManagerOptimized Client Manager data layer and UI components in JobDefinition and Job Activity screens for: Expand All (one-click expand)Collapse All (one-click collapse)Scrolling (optimized load on DB)Filtering (optimized query and fixed some known bugs)Note: These data layer optimizations are applicable only for external ClientManager database (Oracle/MS SQL Server)“The Web client is much betterthan previous versions”- Feedback from EFT 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential13

TES 6.2 Web Client Performance BaselineJob Definition – 30KRecords (in Seconds)Page LoadSelect AllExpand SelectedExpand allRefreshScroll from Top toBottomScroll half way downScroll Bottom to TopCollapse6.1Baseline13.77.616.958512.16.2% 1% 2013-2014 Cisco and/or its affiliates. All rights reserved.Job Run/JAC – 30K6.16.2% ImprovementRecords (in Seconds) Baseline BaselineExpand All5.72.997%Collapse AllScroll from Top toBottomScroll from Bottom toTop1701.98.74.27.94.38847%107%84%Cisco Confidential14

Transporter Improvements Parallel Read – read time optimized Optimized Data Validation Lot of usability enhancements and bug fixesTransaction (Time inSeconds)Reading data6.1 Baseline6.2 Baseline % ImprovementAchieved with parallelism and tuningpost read operations543236Transporting First (1) Job42641153608% Job has dependency depth level 184Transporting 15 jobs90326401311% Complex jobs (nested dependency) 2013-2014 Cisco and/or its affiliates. All rights reserved.130%RemarksCisco Confidential15

UCS Manager AdapterUCS Manager Adapter integrates with UCSM 2.0 through XML APIPower up UCS ServerPower up a UCS server using the service profile associated with itPower down UCS ServerPower down a UCS server using its service profile.Reset UCS ServerReboot a UCS server blade associated with a service profile. 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential16

Adapter for UCS ManagerUCS Manager User need to have the following privileges: Server EquipmentServer MaintenanceServer PolicyService Profile ConfigService Profile Config PolicyService Profile Ext AccessService Profile ServerService Profile Server OperService Profile Server Policy 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential17

Multi-domain AD/LDAP AuthenticationWith TES 6.2, a single instance of Client Manager and Master can be configured toauthenticate users across multiple AD/LDAP domainsExample: If ABC-Americas and ABC-EMEAare two geographic AD domains then with TES6.1 they need to have two instances of ClientManagers (say CM1 and CM2). CM1 isconfigured to authenticate users in ABCAmericas and CM2 is configured toauthenticate users in AMC-EMATES 6.2 addresses this through centralauthentication mechanism 2013-2014 Cisco and/or its affiliates. All rights reserved.TES 6.1CM1ABCAmericasTES 6.2CMABCAmericasTES 6.1CM2ABC-EMEATESMasterAD/LDAPABC-EMEACisco Confidential18

Runtime User Support for TES Workgroups A new “Runtime Users” tab is added tothe Workgroup definition dialog whereruntime users can be associated withthe workgroup Members’ runtime users can be used bythe workgroup When checked all the members of theWorkgroup can leverage the Runtime usersselected in this tab for job definitionsWhen unchecked only the runtime usersspecifically assigned to the members canbe used in jobs owned by the workgroupThe equivalent of the legacy behavior isto have “Members’ runtime users ”checked and to have no additionalruntime users selected“This is huge We will definitely assignruntime users only at the group level.”- Feedback from EFT 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential19

Architecture Update 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential20

TES 5.3.1TES 6.15Core Business Logic & HACore Business Logic & HARESTAPIFault MonitorBackupMasterPrimaryMasterTES DBFault MonitorBackupMasterTransporterPrimaryMasterCLITES DB41Client Manager2Cache6Native ThickClientsTransporter 2013-2014 Cisco and/or its affiliates. All rights reserved.3Browser BasedAD/LDAPCisco Confidential21

TES 6.1TES 6.2Core Business Logic & HARESTAPITransporterTES DBRESTAPIFault MonitorBackupMasterCore Business Logic & HAPrimaryMasterTES DBFault Client ManagerClient ManagerAD/LDAPJava ClientCacheCache1Browser Based 2013-2014 Cisco and/or its affiliates. All rights reserved.AD/LDAPBrowser BasedCisco Confidential22

Cisco Tidal Enterprise Scheduler 6.2AdaptersBI & DatabaseStandards Based InterfacesSSHREST/SOAPHistoric andPredictiveAnalyticsLight WeightJava ClientSMTPJMSDatabase(Oracle,MS-SQL)Core Business Logic & HAJDBC Oracle DB MS-SQL Informatica Cognos BusinessObjectsData Center ApplicationsLDAPFault MonitorJDESAP PeopleSoft OracleEBSBig Data (Hadoop)BackupMasterPrimaryMasterMapReduceAdapter HostFrameworkClient ManagerHiveSqoopHDFSPrivate & Public CloudClient ManagerVMWareEC2S3UCSManagerAgentsSelf 2013-2014ServicePortalCisco and/or its affiliates. All rights reserved.MobileAppUI – Browser Based(IE & FF)CLI/TransporterCisco Tru6423Windows

Planning Your TES 6.2Implementation 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential24

TES 6.2 Software Specifications All of the TES components require Java 1.7 and above Supported OS for Master, Fault Monitor and Client ManagerHPUX11.23, 11.31SolarisAIXWindowsRHELCent OSSUSE9, 106.1, 7 and 7.1Server 2008 (Standard Enterprise), Server 2012 (Standard)Server 2012 R2 (Standard)Redhat Enterprise Server v5, v6 (64 bit)v5, v6SUSE Enterprise Server v11, v11.3Oracle Enterprise Linuxv5, v6VMWareESX/ESXi 4.0, 5.0, 5.1 Supported Platform for Transporter (64 bit) - Windows 7, Windows 2008 Supported Browsers – IE 10, FF 24 (recommended FF 20) 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential25

TES 6.2 Hardware RecommendationsConfiguration geConfigurationMaster and Client Manager DBJobs DefinitionDB Size1 – 3000 4 GBConcurrentUsers5 - 103,000 – 20,000 16 GB10-2020,000 and lessthan 100 K 32GB50-100Master ConfigurationMaster ConfigurationMemoryCPU CoresSmall8GB4Medium16GB8Large24GB16Note: For each adapter instance use the following guidelines forRAM allocation:oo1 GB of extra RAM for Small and Medium configurations2GB of extra RAM for Large configuration 2013-2014 Cisco and/or its affiliates. All rights reserved.Minimum size required by Master and Client ManagerDatabases: MS SQL Server: 128 MB Data, 32 MB Log Oracle: 400 MB Data, 300 MB Index, 200 MBTempAs the number of jobs or logs increases, DBAshould tune the DB accordinglyClient Manager ConfigurationClient ManagerConfigurationSmallMediumLargeMemoryCPU Cores12 GB24 GB32 GB81624Cisco Confidential26

TES 6.2 Hardware Recommendations Contd.,Java Client ConfigurationJava ClientSmallMediumLargeMemory4 GB8 GB12 GBCPU Cores448Transporter 2GB16GBCPU Cores4812NOTE: All Memory & Disk requirements are for TES relatedcomponents and doesn’t include any OS requirements 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential27

Upgrade/Migration ScenariosSupported versions for Upgrade & Migration 5.3.1 to TES 6.2 (In line Upgrade if h/w specs are met / Migration) 6.0.3 to 6.2 (In line Upgrade / Migration) 6.1 to 6.2 (In line Upgrade / Migration)High level check list Run a readiness assessment to check if your environment meets the recommended hardware and software configurations Make backups of all systems/databases/etc Engage TAC/Services to learn of any hotfixes/patches of interest Do early testing on a non-production environment first Upgrade all TES agents – must be TES 3 agent for TES 6x Engage directory services team for AD/LDAP information Limit your history retention prior to the upgrade – best practice is 30 days of historical information onlineUse Cisco Services or one of our recommended 3rd party partners for 5.3.x to TES 6.2upgrade/migration 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential28

Cisco Advanced Services forTidal Enterprise Scheduler 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential29

TES Service OfferingsPre-Implementation Planning & Implementations Traditional Implementation Services Conversions and Migrations Customized Integrations and Solutions Business Process Analysis, Design & Re-engineerComprehensive Upgrade Services Enterprise Scheduler Solution Connectors to Adapters 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential30

TES Service OfferingsRemote Services and Support Production Go live & Post Production SupportServices Scoping & Delivery:Luis Bravo - Engagement Managerlubravo@cisco.comPatti Stephenson - AS Directorpattstep@cisco.comEnvironmental Assessments Software and Hardware Requirements Evaluate Current TES Infrastructure Complete Healthcheck Comprehensive ReportProduct Education & Training Formal Hands-on Product Training Intermediate, Advanced, Administrator, Operator, Transporter, Upgrades Customized Training & Workshops 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential31

Operational Best Practices andRecommendations 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential32

ItemAreaAffectedcomponentsRemarksSizing various components correctlyCapacityAllRefer to performance tuning guide. Loadtest the same.Having Multiple CMs Load balancedLoad DistributionMaster, CMsHaving multiple CMs help to distributeload and avoid downtimes duringmaintenanceUpgrading one CM at a time & offpeak time maintenanceMaintenanceCMsSee above noteCorrect parameter tuningJVM, ThreadsAllRefer to Perf Tuning GuideHaving higher auto-refresh intervale.g., 30 or 60 seconds for web clientsPerf TuningCM & Web ClientDefault is 30 seconds in 6.2 (can bechanged by using Web client View Configure Auto-refresh Interval)History PurgeMaintenanceMaster & CMsKeep at the most 30 days of data online.Database Index usage review (for CMonly and share the indexes withCisco)MaintenanceMaster & CMsDBA needs to monitor to make sure thatthere are no table scans (except 1-2known)Configure timeouts properly to avoidRPC 2013-2014errorsCisco and/or its affiliates. All rights reserved.System/RPCerrorsWeb clientsApplication note published.Cisco Confidential3333

ItemAreaAffected componentsRemarksInitial roll out of Java Client & CMSyncTapering/Perf TuningMaster, CM and JavaClientMake sure full CM sync is completedbefore Java Clients connect to Master.Run Master, CM and TP on differentservers (Mandatory for Productionoperations)PerformanceAllMaster and CM databases should alsobe on separate servers for productionsystems.Job Hierarchies (limit to 4 or less ifpossible)PerformanceAll1.2.Log file countsMaxLogFiles 1000dsp.propsclientmgr.propsservice.props for eachadapter,transporter.propsIn order to help troubleshoot, increase #of log files retained to say 30 daysworth (e.g., # of logs per day * 30).Review space requirements, per yourlog level.Software patching (upgrade to eachmonthly or regular rollups) –including agentsMaintenance/performanceAllKeep your system patched to thecurrent version as improvements arecontinually being made to the product.Admin & End User TrainingContinued learningAllKeep key resources working on TEStrained well on product and these bestpractices34Cisco Confidential 2013-2014 Cisco and/or its affiliates. All rights reserved.Avoid too-deep tree structureAvoid too many nesteddependencies34

References Cisco TES Home Pagewww.cisco.com/go/workloadautomation From TES Home Page you can Download Software View/Download Data Sheet and Whitepapers View/Download Installation and Configuration Guides View/Download User Guides Monthly Roll ups of the software are posted in tml?mdfid 283340354&flowid 16741 Subscribe to our Support Community to participate in TES ty/5711/tidal-enterprise-scheduler 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential35

2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential36

Key Takeaways TES 5.3.1 EOL timelines End of Engineering Maintenance: Sep 30, 2014 End of TAC Support: Jul 31, 2015 TES 6.2 will be made generally available by May end Ensure proper hardware sizing for successful implementation Follow the best practices to ensure smooth operation Engage services for TES 5.3.1 to TES 6.2 migration 2013-2014 Cisco and/or its affiliates. All rights reserved.Cisco Confidential37

Thank you.

Cisco Tidal Enterprise Scheduler 6.2 Backup Master Primary Master Fault Monitor Core Business Logic & HA Client Manager Client Manager UI – Browser Based (IE & FF) Adapter Host Framework Database (Oracle, MS-SQL) LDAP CLI/ Transporter Adapters Standards Based Interfaces SSH REST/SOAP SM