WIXLIB

Multiple Means of Scheduling andExecuting IT WorkloadsWhen businesses first started using IT much of the work was computationalin nature, with data input on punch cards and the processing run as a batchof cards that was available at the end of each day. Batch-oriented processingdominated early business computer use well into the 1980s. The source of thedata changed as CRTs, personal computers, and other machine-to-machinesources of data emerged. Networking added a significant way to share databetween systems and between companies. The idea of batch processingevolved as more and more systems became near-real time, but the need to runcertain processes on a schedule has never ceased to dominate the back officeof IT operations.Early scheduling was nothing more than a list of tasks to be manually executedat certain times of the day or when certain events had happened. As complexityrose, software to automate the execution of certain programs or multiplesteps in a longer process was created. This software became known as jobscheduling. Initially, the software was calendar- and time-based, but becamemore sophisticated to include event triggers, such as the availability or arrivalof a file or email, or the completion of another process.All major OS platforms have some form of a job scheduler built into the operatingsystem. Examples include Cron on Unix/Linux environments, Windows TaskScheduler, or IBM Advanced Job Scheduler for iSystems. The emergence ofpublic clouds generated similar environment-specific schedulers, such as AWSBatch and Azure Scheduler, as well as Azkaban and Apache Ozie for the Hadoopecosystem for big data processing. These schedulers can be sufficient forsome users, but this level of job scheduler comes with some disadvantages.Platform-specific schedulers are just that: specific to one platform. There is nocoordination between jobs or schedulers on other platforms. Modern IT cansee data originate on systems in a variety of environments including Linux,Windows on-premises, cloud environments, and mainframes. Job schedulingoften requires manual intervention and change management, which meansnew coding for new or modified jobs. With different schedulers in differentenvironments, staff need to be trained on different user interfaces and differentmeans of defining job flows.To rationalize the scheduling function across multiple platforms within anenterprise, the class of software known as workload automation (WLA) wascreated. WLA offers improvements to the scheduling tasks across an enterprisewith multiple environments by bringing consistency in job definitions, broadervisibility, and control to the entire enterprise. Processes that begin in oneenvironment and end in another can be executed in an orderly fashion andmonitored through completion of the whole process.4Organizations differ in size, technical sophistication, maturity, complexity,volume of processing, and many other characteristics that can affect ITworkloads and scheduling needs. As a result, different organizations havedifferent scheduling needs and priorities. Some have only one environmentand get by on the native OS scheduler. Others have multiple environmentsbut do not prioritize having an enterprise-class WLA solution, and instead livewith two or more native OS schedulers. Those with multiple environments,key processes that span multiple systems, SLAs with significant economicpenalties, or other sophisticated needs should adopt an enterprise-class WLAsolution. As needs continue to evolve, many companies look to migrate to evenmore powerful WLA solutions. Still, others remain content (or maybe struggle)with native OS schedulers. This study even uncovered some that continue tofunction completely with a manually run list. Even as modern enterprise-classWLA continues to advance, adoption is far from ubiquitous.Past EMA studies on WLA focused on users of this class of software, specifically.This study is different in that using an enterprise-class WLA product was nota requirement to complete the survey. Those using an environment-specificscheduler were included as well. Those using only manually run processeswere counted, but terminated from the rest of the survey. EMA found that 65%of respondents use both environment-specific schedulers (non-WLA) and oneor more enterprise-class WLA products, while 16% are using only non-WLAschedulers and 6% use a WLA product and no other schedulers. Just 13% haveno scheduler of any kind in use.Scheduler UseNo Scheduler,13%Non-WLASchedulers Only,16%WLA SchedulersOnly, 6%Both WLA andnon-WLASchedulers, 65%Sample Size 423 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comNon-WLA Schedulers OnlyWLA Schedulers OnlyBoth WLA and non-WLA SchedulersENTERPRISE MANAGEMENT ASSOCIATES

In 2018, EMA found that 37% of respondents had come to use a WLA productfor the first time in the previous four years. Now, just 16% remain on non-WLAschedulers and 13% seem to have no automated scheduling needs at all. Thebulk of the holdouts to enterprise-class WLA seem to have moved up to moresophisticated scheduling software within the past five years. During this sametime, EMA has been monitoring a trend for those already using one or moreenterprise-class WLA products to migrate to or standardize on a new WLAproduct. In this study, 54% said they were considering migrating to new WLAsoftware. This is closer to the numbers seen in EMA research in 2016 for NAand Europe only. In 2018, 44% in NA and Europe said they were considering achange. The downward trend from 2016 to 2018 in NA and Europe is a resultof many having already switched. The larger number in 2019 is a result ofincluding Asia in this study, where 73% said they are considering changingWLA software. NA and Europe seem to be leading in this trend by about fouryears, but Asia is showing a similar trend.Is your organization considering migratingto a different WLA software?Considering Migrating to a DifferentWLA Software by Region48%North 0%60%80%NoSample Size 4235 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comENTERPRISE MANAGEMENT ASSOCIATES

Non-WLA Scheduling SolutionsRespondents were asked which non-WLA schedulers they were using, hadpreviously used, or were evaluating. The blue column represents those using aspecific product, the red bar shows those who discontinued use of a product,and the green bar shows those evaluating a product. The chart is sorted frommost used to least used. Windows Task Scheduler is the most used non-WLAscheduler at 40%, followed by Azure Scheduler at 28% and AWS Batch at 24%.Cron was reported at just 16%, which was a surprise as a much bigger 10.00%Number of Non-WLASchedulers in UseNon-WLA Scheduling Solutions50.00%40.00%group was expected, and a further surprise is that more are using manually runscripts than Cron. Apache Airflow, Ozie for Hadoop, and Azkaban for Hadoopare all relatively new, but have sizable numbers evaluating them for deployment.Azure Scheduler and AWS Batch share first place for evaluations at 10%. Notsurprisingly, manually run scripts have the largest number of previous users at18%, followed by Windows Task Scheduler at dbaoon(fporHadoop)Previously Buaatchllyrunscripts-20.00%In Use40.0%20%20.0%11%10.0%5%3%0.0%12345 EvaluatingSample Size 423Those using only one non-WLA scheduler represent 62% of respondents, with 20% using two and 11% using three. Less than 8% are using four or more of thenon-WLA schedulers.6 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comENTERPRISE MANAGEMENT ASSOCIATES

Some organizations have defined significant numbers of jobs on non-WLAschedulers, but generally those on non-WLA schedulers have defined fewerjobs than those using enterprise-class WLA products. The most mentionednumber of defined jobs for non-WLA schedulers was a tie between less than1,000 and 1,000-5,000 jobs, each at 18%. Using the midpoint of each range(e.g., 1,000-5,000 treated as 3,000), the average number of jobs defined on eachnon-WLA product was calculated to create the chart on the right. While Cronis only used by 16% of respondents, they have defined the highest averagenumber of jobs at over 275,000. AWS Batch has the second-highest number ofjobs defined at just under 150,000. All the others average below 100,000 jobsdefined. While 22% are using manually run scripts, on average, they are definingabout 35,000 jobs, appropriately the lowest number and the least-automatedmethod.Number of Defined Jobs on Non-WLA Schedulers –All .00%4.00%2.00%0.00%300,00017.6% 17.6%13.2% c heinhe rdo ApduawchleserTaAirskfloAScAp z 11,0000thanLessAverage Number of Defined Jobs byNon-WLA SchedulersSample Size 4237 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comENTERPRISE MANAGEMENT ASSOCIATES

The number of jobs defined is growing for 68% of organizations, and stablefor almost 23%. For most, growth is in the less than 10% category with nearlyas many in the 10% to 25% category. Using the midpoint for each range, theaverage job growth by product is estimated. Azkaban has the highest growthin number of defined jobs at just over 23%, followed closely by AWS Batch andAzure Scheduler. While Cron adoption is at only 16%, those using it have over12% growth in the jobs defined. Appropriately, manually run scripts is growingthe least.How Number of Jobs is Changing for NonWLA Schedulers - All Products30.00%Average Growth in Number of JobDefinitions for Non-WLA 5.00%15.00%10.00%12.4%10.00%5.00% 2019 Enterprise Management Associates, Inc. All Rights Reserved. educhOtheronClereduchBatchp)AWSzkaban%75% 50%to7550%% 25%%10toto2510%e Staye sam%e10th- to-10% 50%to25%80.00%1.7%(Hadoo3.2%1.0%d0.0% 0.2%- -5750%%to-25% 75%0.00%4.9%A3.6%5.00%ENTERPRISE MANAGEMENT ASSOCIATES

Users of non-WLA schedulers were also asked to rate how well each productused meets their needs on a five-point scale in which 5 is very well and 1 isnot at all well. All of the non-WLA schedulers had a mean satisfaction of 4(moderately well or better) with the exception of manually run scripts. AzureAverage Satisfaction Rating for Non-WLAScheduler ProductsAzure SchedulerPrimary Reason for Discontinuing Non-WLASchedulers4.60AWS BatchScheduler scored the highest of all non-WLA schedulers at 4.6. Interestingly,Microsoft has decided to retire Azure Scheduler at the end of September, 2019.Azure Logic Apps will be the replacement. Online forums are not taking thischange kindly.Advanced needs of digital transformationWe had a better tool available4.434.33Apache Airflow4.26Complexity to Manage4.25Standardizing on a single toolCron4.15Other1.002.003.004.007.3%5.3%Poor root cause analysis5.0%5.0%Poor reliability4.6%Our merger or acquisition2.3%Vendor merger or acquisition0.0%Cost0.0%5.00Mean0.0%Sample Size 4235.0%10.0%15.0%20.0%25.0%Sample Size 155Even though satisfaction is high for most non-WLA schedulers, 155 of 423respondents have moved on from one or more non-WLA scheduler. Of thosediscontinuing use of non-WLA schedulers, 19% did so because of advancedneeds to support digital transformation, which is the most stated reason formaking such a change. Other leading reasons include 15% who said they99.9%Discontinued the functions being scheduled3.510.0013.0%Discontinued the underlying platforms4.00Manually run scripts13.4%Poor workload change management4.24Windows Task Scheduler15.3%Advanced needs of app modernizationAzkaban (for Hadoop)Apache Ozie for Hadoop19.1%had a better tool available, 13% who said they discontinued use because ofadvanced needs of application modernization, and 13% who said they had poorworkload change management. All of these concerns can be improved with anenterprise-class WLA product. 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comENTERPRISE MANAGEMENT ASSOCIATES

Enterprise-Class Workload Automation (WLA) Scheduling SolutionsRespondents were asked which enterprise-class WLA solutions they wereusing, had previously used, or were evaluating. The blue column representsthose using a specific product, the red bar shows those who discontinued useof a product, and the green bar shows those evaluating a product. The chart issorted from most used to least used. IBM Workload Scheduler is the most usedWLA scheduler at 26%, followed by IBM Z Scheduler at 22% and ASG Zeke at justunder 10%. All products have users that have moved on and prospective userswho are evaluating them. Note, this chart does not reflect market share. Surveyrespondents are anonymous and multiple persons from the same organizationcould have taken the survey, which would result in multiple responses for asingle installation of a given product.WLA 5.00%0.00%-5.00%IBMWIBSMZ/OASSGZCAekeBM WLACDCHEita o nc h troliJP1 MAJS3CATidaCA7lAu Edititom onicCA WLAWCA LAASAEWCLAICA AcEti v S PDeBollaatrcUni hSM verseAInOSpfinHtCoite elp onnSy ebDATrastnAeAu ms c hHeltoJApSmysat MSeNtemOsWASRo!GReboZdwentaoo SchdedRuunM leyJobs-10.00%In Use10Previously Used 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comEvaluatingENTERPRISE MANAGEMENT ASSOCIATES

The number of jobs defined in enterprise-class WLA solutions is greater thanthe number of non-WLA schedulers on average, with WLA solutions havingthe most organizations in the ranges of 25,001 to 50,000 (18%) and 50,001 to100,000 (16.5%) jobs defined.Number of Defined Jobs on WLA 000-1,0001.6%thanLess16.5%Sample Size 42311 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comENTERPRISE MANAGEMENT ASSOCIATES

Just as with the non-WLA schedulers, enterprise-class WLA solutions are alsoseeing growth in the number of job definitions, with very similar numbers.Growth in job definitions is the experience of 70% of WLA users, just over 20%have stayed the same, and a few organizations are seeing a decline.How Number of Jobs is Changing Across all WLA 0%5.8%5.0%0.0%0.0%0.5%1.8%2.7%2.9%1.8%Decreased Decreased Decreased Decreased Decreased Stayed Increased Increased Increased Increased Increasedmore than between between between less than the same less than between between between more than75%50% and 25% and 10% and10%10%10% and 25% and 50% and75%75%50%25%25%50%75%Sample Size 42312 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comENTERPRISE MANAGEMENT ASSOCIATES

Those using only one WLA solution represent almost 69% of respondents,slightly more than those using only one non-WLA scheduler. Two WLA productsare in use for 17%, with 8% using three. Less than 6% are using four or moreof the WLA solutions. The most common reason stated for using more thanone WLA solution is meeting the demands of digitalization and applicationNumber of WLA Products in UseWhy are you using multiple WLA products?80.0%70.0%modernization (38%). This is the first time EMA has seen digitalization as thenumber-one reason for multiple products. In the past, the most stated reasonwas that different groups required different scheduling tools, which was selectedby 35% of respondents and is the second-highest response in this study.To meet the demands of digitizati on andapplication modernizati on68.8%38%Different groups requi re different s cheduling tools60.0%35%Mainframe platform has one s cheduler, distributedplatform(s) has a differen t scheduler31%50.0%Int egration limitations40.0%Knowledge gap between different WLA s olutions30.0%Migrations c ons ume too many resources17.3%20.0%1.4%227%3426%Legacy systems25%In the middle of converting from one to another25%Inherited through acquisi tion25%4.3%0.0%129%Functionalit y limit at ions8.3%10.0%30%5Sample Size 4230%5%10%15%20%25%30%35%40%Sample Size 364, Valid Cases 364, Total Mentions 1,06013 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comENTERPRISE MANAGEMENT ASSOCIATES

For those with multiple WLA solutions, 29% say they do not have cross-platformdependencies between the several systems. For the 71% who do have crossplatform dependencies, 44% say the impact is minimal, while 11% have regularproblems and 16% have addressed cross-platform problems by integrating ananalytics solution to provide a consolidated view of all environments. Thoughonly 11% have ongoing issues and 16% have resolved the issues with analytics,80% say they are considering rationalizing to a single product.Do you have cross-platform dependencies betweendifferent schedulers?NoAre you considering rationalizing to a singlescheduler product?29%No20%Yes, with minimal impact44%Yes, and it regularly causesproblems11%Yes, but we have integrated ananalytics solution that provides aconsolidated viewYes80%16%0%10%20%30%40%50%Sample Size 36414Sample Size 364 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comENTERPRISE MANAGEMENT ASSOCIATES

There has been a trend for the past seven or eight years for organizations toconsider changing their WLA solution. In this study, 110 of 423 respondentshave discontinued one or more WLA products and 177 are actively evaluatingnew WLA software. Both these groups were asked about their reasons fordiscontinuing a WLA product or for considering a new WLA product. Of thosewho discontinued using a product, 26% said their primary reason was theadvanced needs of digital transformation, with the second-most commonresponse being the advanced needs of application modernization for almost19%. Similarly, those evaluating a new WLA product state digital transformationas the most common reason at 34%. Cloud-based jobs (28%) and improvingresource utilization (28%) tied for the second-most common reason for lookingfor new software.Reasons for Evaluating New WLA SoftwarePrimary Reason for Discontinuing WLAProductsAdvanced needs of digital transformation12.3%Poor root cause analysisApplication modernization21%21%4.8%Discontinued the underlying platforms2.7%1.6%Poor relationship with the vendorPoor reliability1.1%0.5%0.0%24%Improve root cause analysis6.4%Poor workload change management26%New directives from senior IT management11.8%Complexity to manage28%Scalability concerns13.9%Cost28%Improve resource utilization18.7%We had a better tool available34%Cloud-based jobs26.2%Advanced needs of app modernizationDiscontinued the functions beingscheduledDigital transformation5.0% 10.0% 15.0% 20.0% 25.0% 30.0%Workload change management20%Currrent solution is too complicated20%Major new application requirements19%Audit and compliance concerns18%DevOps18%ERP implementation17%Problems meeting SLAs16%Too many disruptions to processing16%Merger or acquisition12%0%Sample Size 110155%10% 15% 20% 25% 30% 35% 40%Sample Size 177, Valid Cases 177, Total Mentions 596 2019 Enterprise Management Associates, Inc. All Rights Reserved. www.enterprisemanagement.comENTERPRISE MANAGEMENT ASSOCIATES

Using Non-WLA Schedulers OnlyOf respondents using a non-WLA scheduler only (16% or 68 respondents), 49have no plans to add a full WLA product. The most mentioned reason for notusing a WLA product is that needs are met by the lesser scheduling software(51%), with staff experience (33%) and cost of WLA (31%) next-most mentioned.In EMA’s 2018 WLA research, 37% had not been using a WLA product untilsometime within the past four years (2014 to 2018). Clearly, a large numberof those needing the advanced features of an enterprise-class WLA

Scheduler, or IBM Advanced Job Scheduler for iSystems. The emergence of public clouds generated similar environment-specific schedulers, such as AWS Batch and Azure Scheduler, as well as Azkaban and Apache Ozie for the Hadoop ecosystem for big data processing. These schedulers can be sufficient for