WIXLIB

Input File GenerationInput file generation command: input generator -n base file name -t template file [-p] [-f] [-b] [-q] [-c]The file passed in as template file must be overwritten with the generated input file (therefore it isa copy of the EDML registered template)The base file name is used to generate the names of the other files needed as inputThe input file generator command must be invoked in the same directory where the other filesneeded as input resideThe other command line options are flags to indicate the existence of specific files:–––––“-p” Î base file name .param contains the details dialogs key/value pairs for both the theorybase file name .paramtheory and runtypedialogs“-f” Î base file name .frag contains the chemical system in the ECCE standard MVM formatbase file name .frag“-b” Î base file name .basis .basiscontains the basis set already converted to the format neededbase file nameneeded by the code“-q” Î base file name .esp contains ESP constraints set in the Calculation Editor Partial Charge Editorbase file name .esp“-c” Î base file name .conbase file name .con contains geometry constraints set in the Calculation EditorEditor GeometryConstraint EditorThe .param file also contains special ECCE values that are needed for input file generation suchas the calculation name, user annotation, theory name, runtype, number of electrons, charge, etc.Input file generation files live in ECCE HOME/scripts/parsers

Input File Generation (cont.)Basis set reformatting–Existing support is provided for the following codes:NWChemGaussian 92, 94, 98, 03GAMESSGAMESSGAMESS-UKACES Typically the reformatting scripts for these codes can be reworked to add support for newcodes by starting with the code closest in basis set format to the new code–Two Perl scripts are needed:The one specified in the EDML file as the BasisSetTranslationScript , by convention named BasisSetTranslationScript ,std2 code A script for writing out the basis set format needed by the code from the ECCE standard format, byconvention named wr code GBS.pmwr code GBS.pmThe std2 code script is invoked by the Calculation Editor duringduring input file generationThe std2 code script invokes the wr code GBS.pm scriptwr code GBS.pm–Check out the existing scripts such as std2NWChem and wrNWChemGBS.pm in ECCE HOME/scripts/parsers to learn more

Input File Generation (cont.)ECCE registered codes use a Perl script in combination with a template file to generate input filesThe developer may choose to do input file generation in another way (Python script, FORTRANexecutable, no template file, etc.) provided it adheres to the input file generation commandconventions listedTemplate file contains “##” delimited keywords that are placeholders for substituting in the inputspecified by the user for the given calculationThe “##” keywords correspond to either––Python details dialog user input field “name”name” parameters (or a unique substring match)Subroutine names in the input file generation scriptPython details dialog “name” parameter substitution works well for keyword driven input likeNWChem, poorly for more obtuse formats, e.g. GaussianTemplate file lines containing “##” keywords with neither a corresponding details dialog value(because it’s not applicable to the theory/runtype or because the user left the value defaulted) noran input file generator subroutine (or the subroutine returns an empty string) are removed from thegenerated input fileInput file generator subroutine name substitution is needed for more complex input directivesrather than a straight mapping of details dialog input fields to input file keywords (e.g., Gaussianroute card)The subroutine can then take a combination of user input field values and coerce them inunnatural ways to come up with the required input file syntaxThe input file generator is also responsible for composing the other input it is given (fragment file,basis set file, etc.) into a properly formatted code input file

Input File Generation (cont.)Template file nwch.tpl snippet:Title "##annotation##“"##annotation##“# ECCE user annotation or calculation name givengiven in .param.param fileStart ##title### ECCE calculation name given in .param.param fileechoMemory ##MemorySize## mw##MemorySize### Details dialog ES.Theory.SCF.MemorySize##Charge### Input generator scriptscript subroutine##chemsys####chemsys### Input generator script subroutinesubroutine to write the chemical systemecce print ##parseFile####parseFile### ECCE registered parse file name given in .param.param file##basis### Input generator script subroutine to write the basis setscf##SCFTheory### Input generator script subroutine to format theory category/name##SCFTheory####SCFDirect### Input generator script subroutine,##SCFDirect##subroutine, see next pagethresh ##SCF.ConvergenceGradient.Value### Details dialog rgenceGradient.Value##maxiter ##SCF.ConvergenceIterations### Details dialog ceIterations####SCFLevelSCFLevel##### Input generator script subroutine, see next page##end

Input File Generation (cont.) Input generator script ai.nwchem snippet:#!/usr/bin/env perlsub SCFLevel {sub SCFDirect {local( shift1, shift2, crossover, result);local( result); shift1 AbiDict{"LevelShiftSize"}; result AbiDict{"ES.Theory.SCF.Direct"}; shift2 AbiDict{"NewLevelShiftSize"};if ( result eq "" && AbiDict{"SCF.DiskSize"}) { result "semidirect filesize ". AbiDict{"SCF.DiskSize"}."000000";} elsif ( result eq "Direct") { result "direct";}return result;} crossover AbiDict{"NewLevelShiftCrossover"};if ( shift1) {if ( crossover && (! shift2 )) { shift2 0.0 } result "level pcg shift1 crossover shift2";} else { result "";} result;return result;}

Launch PreprocessorAugments the code input file with any directives not known beforethe job is launchedScript is invoked with “-p” option specifying the name of a kev/valueparameter fileParameter file contains Job Launcher GUI settings including rundirectory, scratch directory, number of nodes and processorsselected, etc.Currently, NWChem launch preprocessor script only adds thescratch directory to the input fileSome codes don’t properly divide chemistry calculation input fromjob submission input making the launch preprocessor criticalLives in ECCE HOME/scripts/parsers

Calculation Viewer

Code Registration: Output ParsingCompute Machine ÍÎ ECCE Application MachinePerlCode OutputJobJobMonitorMonitorParseDescriptorText Block 1Parse Script 1Text Block 2Parse Script 2.Text Block NParse Script nViewerViewer

Output Parsing: Parse DescriptorCompute Machine ÍÎ ECCE Application MachinePerlCode OutputJobJobMonitorMonitorParseDescriptorText Block 1Parse Script 1Text Block 2Parse Script 2.Text Block NParse Script nViewerViewer

Parse DescriptorDuring job execution, eccejobmonitor perl script scans theoutput file for matches on strings specified by parsedescriptor fileWhen a parse descriptor is matched, buffer data until theend marker of the descriptor is foundData block is shipped via ssh from the compute machine tothe eccejobstore process on the ECCE application machineParse descriptor file lives in ECCE HOME/scripts/parsersTo be continued

Parse Descriptor (cont.)Parse Descriptor nwchem.desc snippet:[TEVEC]Script nwchem.teBegin task gradient%begin%total energyFrequency allEnd task[END][DIPOLE]Script nwchem.dipoleBegin begin%total dipoleFrequency allLines 2[END][ESPCHARGE]Script nwchem.espFile ##CalcName##.qBegin ##CalcName##.q[END]

Properties FileMaps property names (keys) to data representation based ondimension and type of data per dimensionCalculation Viewer uses the data representation attribute todetermine the property panel GUI and whether/how to visualize,graph, or display raw data for the propertyDescriptive label and units are also properties file attributes used bythe Calculation Viewer property panel GUIsCurrent 250 properties are listed – best strategy is to find one closeto the new oneLives in ECCE HOME/data/client/config

Properties File (cont.)properties file edDerivedDerivedDeriveduser label##########prop ero Point EnergyTotal Energy VectorKinetic Energy VectorPotential Energy VectorSCF Energy VectorDFT Energy VectorMP2 Energy PropTSVector GeometryPropTSVector Geometry Step PropTSVector GeometryPropTSVector Geometry Step PropTSVector GeometryPropTSVector Geometry Step PropTSVector GeometryPropTSVector Geometry Step PropTSVector GeometryPropTSVector Geometry Step PropTSVector GeometryPropTSVector Geometry Step Mulliken ChargesMulliken Shell ChargesElectron Density at NucleiFermi ContactESP Charge hargeEFieldGradMHertzChargePropVector Atom PropVector Atom PropVector Unknown PropVector Unknown PropVector Atom PropVector Atom PropVector Atom PropVector Atom PropTable PropTable Atom,Charge Atom,Charge S**2Dipole MomentMP2 Dipole MomentQuadrupole upolePropValuePropVector Coordinate PropVector Coordinate PropVector Coordinate PropVector Coordinate PropVector Unknown PropVector Unknown

Output Parsing: Parse ScriptsCompute Machine ÍÎ ECCE Application MachinePerlCode OutputJobJobMonitorMonitorParseDescriptorText Block 1Parse Script 1Text Block 2Parse Script 2.Text Block NParse Script nViewerViewer

Parse Scripts Output parsing continuedeccejobstore process invokes the proper parse script with commandline arguments on the data blockParse script reformats the data based on its properties file datarepresentationeccejobstore uploads reformatted property data to ECCE dataservereccejobstore sends out JMS message notifying interestedapplications of the new propertyCalculation Viewer(s) in context of calculation process JMSmessage, retrieve, and display/visualize propertyParse scripts live in ECCE HOME/scripts/parsers

Parse Scripts (cont.)Parse script invocation command: parse script key runtype theory category theory name open shells parseOutFile parseInFileNote that the parseInFile and parseOutFile are specified with fileredirection and parse scripts read from stdin and write to stdoutBy convention, key is “.” as a placeholder and parse scripts knowor determine the key they need to use for the parseOutFile (it is leftfor “historical reasons”)Parse scripts typically load only the command line argumentsneeded and most need none.

Parse Scripts (cont.)The parse output file format is tied to the datarepresentation with four basic output formats:– Scalar datakey:size:values:END– Vector datakey:size:columnLabels:values:END– Table datakey:size:rowLabels:columnLabels:values:END– Vector of bels:values:END

Parse Scripts (cont.)Parse script nwchem.dipole:– Need to reformat this data block extracted using the parse descriptor file from theecce.out file:task optimize driver task gradient scf%begin%total dipole%3%double4.38894415454724e-01 3.91321140865017e-01 -4.38827528892410e-01– into this format specified by the properties file:key: DIPOLEsize:3rowLabels:xyzvalues:1.02648373484354 1.1658016598219 -1.07813172811085units:DebyeEND

Parse Scripts (cont.)Parse script nwchem.dipole:#!/usr/bin/env#!/usr/bin/env perl# Force output to be flushed 1; label STDIN ;chop( label);chop( label); line STDIN ;chomp( line);chomp( line);@values split(/ /, ########## convert atomic units to ######## size @values;for ( i 0; i size; i ) { values[ i]];values[ i] 2.541766* values[ i2.541766* values[ ######## for mp2, both the mp2 and scf dipoles are ######### label /(\/(\S )%begin/;S )%begin/; prebegin 1; # might be task tasktype or a theory typetask tasktype if ( prebegin( prebegin /mp2/) { key "MP2DIPOLE";} else { key "DIPOLE";}print "key: key\ key\n";print "size:\"size:\n3\n3\n";print "rowlabels:"rowlabels:\\n";print "x y z\n";print "values:\"values:\n";foreach i (@values) { print " i"; }print "\"\nunits:\nunits:\nDebye\nDebye\n";print "\"\nEND\nEND\n";

Parse Scripts (cont.)Parse script nwchem.esp:–Need to reformat this CalcName .q file specified by the parse descriptor file:OHH3 30.000000-0.0782990.078299–into this format specified by the properties file:0.000000 -0.012125 -0.824086 -0.823655 -1.6386480.000000 0.048499 0.412455 0.412240 0.0000000.0000000.000000 0.048499 0.411631 0.411415 -1.618905key: s:columnLabels:ESP RESP 0e-01 -8.236550000000000e8.236550000000000e-01 -1.638648000000000e 00 000000e4.122400000000000e-01 0.000000000000000e 00 4.116310000000000e4.116310000000000e-01 4.114150000000000e4.114150000000000e-01 1.618905000000000e 00units:eEND

Parse Scripts (cont.)Parse script nwchem.esp:#!/usr/bin/env#!/usr/bin/env perl# Force output to be flushed 1;# read the data from stdin:stdin:# line STDIN ; line s/ \s/ \s*//; line s/\s/\s* //;( natom, ncol( natom, ncol)) split(/ /, line); icnt 0;@charges ();while ( STDIN ) {last if ( icnt);( icnt natom natom); line ; line s/ \s/ \s*//; line s/\s/\s* //;@values ();@values split(/ /, line);for ( i 4; i 4 ncol; i ) {push(@charges,]);push(@charges, values[ i values[ i]);} icnt ;icnt ;}## Print out the data in standard format.#print "key: ESPCHARGE\ESPCHARGE\n";print "size:\"size:\n";print natom natom . " ncol ncol\\n";print "rowlabels:"rowlabels:\\n";for ( i 1; i natom; i ) {( i 1; i natom; i )print " i ";if ( i % 12 0 && i natom natom - 1) { print "\"\n";}}if ( ncol( ncol 5) {print "\"\ncolumnlabels:\ncolumnlabels:\nESP CESP RESP CRESP CRESP2\CRESP2\n";} elsif ( ncol( ncol 4) {print "\"\ncolumnlabels:\ncolumnlabels:\nESP CESP RESP CRESP\CRESP\n";} elsif ( ncol( ncol 3) {print "\"\ncolumnlabels:\ncolumnlabels:\nESP RESP RESP2\RESP2\n";} elsif ( ncol( ncol 2) {print "\"\ncolumnlabels:\ncolumnlabels:\nESP CESP\CESP\n";} elsif ( ncol( ncol 1) {print int "values:"; icnt 0;for ( i 0; i #charges; i ) {if ( icnt( icnt % ncol ncol 0) {print "\"\n";}printf("%.15e ", charges[ icnt]);", charges[ icnt]); icnt ;icnt ;}print "\"\nunits:\nunits:\ne\ne\n";print "END\"END\n";

ECCE Code Registration Gary Black Pacific Northwest National Lab gary.black@pnl.gov May 15, 2008May 15, 2008