Transcription
Co:Z SFTP – New features and Batch JobBest PracticesDecember 13, 2011Kirk WolfSteve t 2011 Dovetailed Technologies LLCSlide 1
Dovetailed TechnologiesWe provide z/OS customers world wide with innovativesolutions that enhance and transform traditionalmainframe workloads:Co:Z Co-Processing Toolkit for z/OSOpenSSH Accelerator for z/OST:Z Quickstart for Tomcat and z/OSJZOS - acquired by IBM in 2005 and now part of thez/OS Java SDKCopyright 2011 Dovetailed Technologies LLCSlide 2
AgendaCo:Z Toolkit release 2.0.0 release new features- currently in “beta”, expect to provide production support by1/31/2012.Best practices for Co:Z SFTP batch jobsUsing file / data set name patterns to set transferoptionsCopyright 2011 Dovetailed Technologies LLCSlide 3
New SMF Recording FeaturesNew SMF 119 Record Subtypes- Co:Z SFTP messages, informational level or above, that wereassociated with the previous transfer.- Subtype 192 - Co:Z SFTP server log messages- Subtype 193 - Co:Z SFTP client log messagesNew internal utility program “ssh-socket-info”- Invoked by Co:Z SFTP client to obtain accurate local and remotehost/port information from child SSH process- Uses IBM EZBNMIFR network management API- APF Authorized with fallback to default behavior if not authorizedCopyright 2011 Dovetailed Technologies LLCSlide 4
Extended Address Volumes (EAV) SupportNew support in Co:Z 2.0.0Actual I/O to DSORG PS-E datasets is supported inz/OS 1.12 by the IBM C libraryCo:Z adds EAV support for FMT8/9 DSCBs and dataset space calculation to Co:Z SFTP, SFTP datasetlistings, and the catsearch commandNew allocation options added to Co:Z SFTP dsntype library pds large extreq extpref basicCopyright 2011 Dovetailed Technologies LLCSlide 5
Co:Z SFTP Batch jobs usingthe original sample RUNSFTP JCL// EXEC PGM COZBATCH//STDIN DD * -- input to z/OS Unix shell# Customize these.coz bin "/opt/dovetail/coz/bin"ruser "uid"server "remote.host.name"servercp "ISO8859-1"remotefile "/path/to/file"export PASSWD DSN '//COZUSER.PASSWD(SITE1)'export SSH ASKPASS coz bin/read passwd dsn.shexport DISPLAY nonessh opts "-oBatchMode no"ssh opts " ssh opts -oConnectTimeout 60"ssh opts " ssh opts -oServerAliveInterval 60"Copyright 2011 Dovetailed Technologies LLCSlide 6
RUNSFTP JCL (continued). coz bin/cozsftp ssh opts -b- ruser@ server EOBlzopts mode text,servercp servercpget remotefile //DD:DOWNLOADEOB//DOWNLOAD DD DSN .,DISP (NEW,DELETE),//DCB (.),SPACE (.)//Copyright 2011 Dovetailed Technologies LLCSlide 7
RUNSFTP sample - shortcomingsCommon settings repeated in every JCL deckExposes low-level shell script code (to folks that maynot know Unix) Becomes a maintenance problem as the inventory ofjobs growsCopyright 2011 Dovetailed Technologies LLCSlide 8
New sample SFTPPROC and scripts//SFTPGET EXEC PROC SFTPPROC//SFTPIN DD *pwdsn "COZUSER.PASSWD(SITE1)"user myuserhost myhostlzopts "mode text"lfile //DD:MYDSNrfile /etc/profile. script dir/sftp get.sh//MYDSN DD DSN COZUSER.SFTPGET.DATA,DISP (MOD,KEEP),//DCB (LRECL 80,RECFM FB),SPACE (CYL,(3,1))Copyright 2011 Dovetailed Technologies LLCSlide 9
Sample SFTPPROC//EXSFTPPROC ARGS ,//LIBRARY 'COZUSER.COZ.LOADLIB',//SFTPIND 'COZUSER.COZ.SAMPJCL(SFTPIND)',//REGSIZE '64M',//LEPARM ''//RUNSFTP EXEC PGM COZBATCH,REGION ®SIZE,//PARM '&LEPARM/&ARGS'//STEPLIB DD DSN &LIBRARY,DISP SHR//STDINDD DSN &SFTPIND,DISP SHR Inst Defaults//DD DDNAME SFTPIN User input//SFTPINDD DUMMY// PENDCopyright 2011 Dovetailed Technologies LLCSlide 10
SFTPPROC – scripts and variablesSample scripts:sftp get.sh,sftp cat.shsftp put.sh,sftp connect.sh,Connection and authentication variables:user, host, port, pwdsn, certSFTP / SSH options variables:sftp optsConfiguration variables:cozbin dir, script dirTransfer variables:lfile, rfile, lzopts, rpatCopyright 2011 Dovetailed Technologies LLCSlide 11
Another example: using JCL variables.//SFTPPUT EXEC PROC SFTPPROC,// ARGS 'user &USER host &HOST rfile &RFILE'//SFTPIN DD *cert "MY-RING RSA-CERT"lzopts "mode text"lfile //DD:MYDSN. script dir/sftp put.sh//MYDSN DD DSN COZUSER.SFTPPUT.DATA,DISP SHRSee Co:Z Batch User's Guide for information onpassing in variables 100 charactersCopyright 2011 Dovetailed Technologies LLCSlide 12
New sample SFTPPROC - benefitsUnix shell script logic factored into separate script filesStandard set of variables available to controlconnection, authentication, options, filenamesSeparation of installation default options fromindividual JCL membersCan be customized and extended Simplifies maintenance and supportCopyright 2011 Dovetailed Technologies LLCSlide 13
File PatternsPreviously, options are set prior to a file transfer.Defaults may be set by user or installation, but applyto all files.Co:Z SFTP now allows options to be automatically setbased on matched file / data set name patternsPatterns use shell pattern matching syntax “fnmatch()”Pattern / option associations set in the Co:Z configuration filesCopyright 2011 Dovetailed Technologies LLCSlide 14
Config FilesSite-wide SFTP default options- /etc/ssh/cozsftp config- /etc/ssh/cozsftp server config(cozsftp client)(server)Three Sections:fixed:default:pattern:- site-wide options that cannot be overridden- site-wide defaults that can be overridden- options activated when transferring a file ordataset that matches a name patternUser specific config files are now allowed- Located in HOME/.ssh/- pattern: sections have priority over site config files- No fixed: or default: sections allowedCopyright 2011 Dovetailed Technologies LLCSlide 15
File Pattern Examples (cozsftp config)Ensure that JCL members are transferred in text modepattern: //*.JCL(*)mode textcozsftp put //MY.DEV.JCL(RUNJ) runj.txtAllocate space for specific datasets automaticallypattern: //*.TRANS*space cyl.3.2,recfm fb,lrecl 80cozsftp get remote.file //PARTNER.TRANS03Copyright 2011 Dovetailed Technologies LLCSlide 16
File Pattern Examples, Con't HOME/.ssh/cozsftp server configpattern: *.txtmode text,clientcp 1252,linerule crlf/etc/ssh/cozsftp server configpattern: *.zipmode binarypattern: *.paxmode binarypattern: *.txtmode text,linerule lfsftp get myarchive.pax local.tarsftp get mynotes.txt local.textCopyright 2011 Dovetailed Technologies LLCSlide 17
Client Session Options Priority OrderClient session settings are determined in the followingpriority order (from highest to lowest):1. The fixed: section of/etc/ssh/cozsftp config2. The first matching pattern (if any) from HOME/.ssh/cozsftp config3. The first matching pattern (if any) from/etc/ssh/cozsftp config4. The interactive command: lzopts5. The environment variable SFTP ZOS OPTIONS6. The default: section of/etc/ssh/cozsftp configCopyright 2011 Dovetailed Technologies LLCSlide 18
Server Session Options Priority OrderServer session settings are determined in the followingpriority order (from highest to lowest):1. The fixed: section of/etc/ssh/cozsftp server config2. The first matching pattern (if any) from HOME/.ssh/cozsftp server config3. The first matching pattern (if any) from/etc/ssh/cozsftp server config4. The interactive command: ls / 5. The environment variable SFTP ZOS OPTIONS6. The default: section of/etc/ssh/cozsftp server configCopyright 2011 Dovetailed Technologies LLCSlide 19
More informationIBM Ported Tools for z/OS: OpenSSH User's GuideCo:Z SFTP User's Guidehttp://dovetail.com/forum (public bulletin board)Open a support ticket with your questions, or call usOur webinar archives: http://dovetail.com/webinarsFuture webinar:Using z/OS SAF Keyrings with SSH and SFTPCopyright 2011 Dovetailed Technologies LLCSlide 20
- Co:Z SFTP messages, informational level or above, that were associated with the previous transfer.-Subtype 192 - Co:Z SFTP server log messages-Subtype 193 - Co:Z SFTP client log messages New internal utility program "ssh-socket-info" - Invoked by Co:Z SFTP client to obtain accurate local and remote host/port information from child SSH process
