Transcription
OpenCalc - An Open Source Programming Framework for EngineeringStephen Hollister, New Wave Systems Inc., Jamestown, Rhode Island US, shollist@newavesys.comAbstractOpenCalc is an open source framework that allows users and programmers in engineering to createcross-industry interactive and automated design solutions based on common batch calculationinterfaces and XML data files. While OpenCalc is open source and free, developers can add in theirown proprietary components that allow users, software developers, students, and researchers toindependently work together in complex markets not properly supported by commercial programs. Thispaper explains the basic parts with application to interactive CAD programs and multi-disciplineoptimization (MDO) processing of single and multiple wrapped calculation components. OpenCalc isa flexible and agile framework that can adapt to any design optimization, lifecycle management, ordigital twin methodology.1. IntroductionThe interactive application (“app”) model of computer programming, where independent developerscreate and link together user interfaces, calculations, and proprietary data definition and file formats, isnot adequate for the evolving demands of users and subject matter experts (SMEs) who want automated,cross-industry calculation and optimization tools. Traditional interactive app control preventsautomated connection and processing of calculations from different sources - a requirement for multidiscipline engineering and optimization (MDE/MDO). The evolution of the internet increases thedemand for collaboration between all engineers and designers, especially in the conceptual andpreliminary design phases. This is hindered by independent and proprietary app development withincompatible data definitions and two-step neutral file conversions.OpenCalc offers an open source framework that solves these problems. Apps are split into threeseparately developed and tested external programming objects: batch calculations or Calc Engines(CEs), User Interface Frameworks (UIFs), and layered and cross-industry open source data filedefinitions (XML) with input/output (I/O) code that can store any program data or structure in acommon file. This split organization better fits the needs of all stakeholders in the software developmentworld: subject matter experts (SMEs), computer scientists, and industries. OpenCalc allows thesegroups to work independently on calculations and data definitions that can be later combined by usersto create completely new tools with no extra programming. New solutions can no longer be the domainof competing software companies that write applications or suites for “users.” Engineers will becomecreators of new solutions and programming will become scripting at a much higher level using commoncalculation components and data definitions.OpenCalc components are independent objects that can be combined to meet any analysis approach,whether it’s multi-objective programming, set-based design, critical path methods, digital twin oranything else. These driving solution systems (UIFs) can be either interactive hands-on or automatedand batch. The low level calc engines and independent XML data files can be used for any higher-levelprocessing methodology with no reprogramming or custom containerization. In general, calc enginesand XML data definitions are created once with a common and open source programming template thatallows many user and UIF combinations with no additional coding.
2. Historical DevelopmentOpenCalc evolved from the author’s work in computer-aided ship design software development overthe last 45 years, from card programming on mainframes to PCs and DOS, then Windows and now theinternet. Many calculations were lost at each change in technology, especially when DOS evolved intoWindows. Calculations might stay the same, but the cost of rebuilding new applications with new userinterfaces was and is too expensive, thereby losing much good software. The author created TheNautilus System in DOS as a spiral ship design system with a modular calculation approach for acommon user interface with graphical output and a common design database. However, only theNURB-based hull design and fairing CAD software (ProSurf) justified conversion to Windows. Usefulcode, such as longitudinal strength calculations, Wageningen propeller optimization and a VelocityPrediction Program (VPP) for sailboats, never got updated to Windows-based user interfaces. Maritimeand other industries can’t support these technological changes with the traditional and costly all-in-oneapplication approach to computer software development. This is a market and cost issue as much as itis a technology issue.In 1996, the author presented a Society of Naval Architects and Marine Engineers (SNAME) paperHollister (1996) that combined modified Lackenby hull variation, hydrostatics, and Holtrop resistancecalculations into one DOS program to search for an optimum hull shape starting from a parent hull.This required all the source code and a custom user interface front end that needed a lot of developmentand testing. That this could be done was of no real benefit since the market would not support the cost.Also, that was when DOS was replaced by Windows and the recoding cost was no longer viable for thenew technology and size of market. Now, twenty years later, the internet is causing further disruptionsin software development. Some now believe that the solution to ship design is to build customcalculations inside of a general interactive CAD program, but the goal is not to hope for one CADprogram to win the market battle. A different approach was sought.In 2014, Hollister (2014) described tests that showed how one can launch large reusable calculationsfrom a spreadsheet user interface in a general way using comma separated value (CSV) files. Thisturned into SNAME Project 114 and culminated in the first release of the software at the WorldMaritime Technology Conference (WMTC) - Hollister (2015). The second release of the softwareoccurred at the SNAME Maritime Convention (SMC 2016) in Seattle where multiple wrapped calcengines (recreating the author’s work in 1996 on hull variation and optimization) were shown using amore general XML database format for all program variables and structures - Hollister (2016). A statuspaper was given of a new revision at a SNAME section meeting in New York City - Hollister (2017).A third revision was presented along with a SNAME short course at SMC 2018 - Hollister (2018). Itgeneralized the basic components to create an open source framework that can be applied to all areasof programming. Three underlying objects were formalized, documented, and separated from Project114. These programming components became the foundation of the Tri-Split ProgrammingArchitecture (TSPA) (www.trisplit.com). The maritime components were built on top of TSPA andbecame the newly named OpenCalc System. This hierarchical structure allows for compatibility andautomation of calculations from many different industries.3. Splitting the AppThe traditional programming model in computer science is the interactive app where one softwaredeveloper or team creates the user interface, calculations, and the data definition and file format.Separate calculations from multiple apps cannot be automated and the data must go through a two-stagefilter process to and from a neutral file format (Fig. 1).
Fig. 1: Traditional programming architectureTSPA splits this structure into three separate “external” programming objects: User InterfaceFrameworks (UIFs), Calculation “Engines” (CEs), and open source hierarchical levels of crossindustry and specific-industry defined variables and data structures (XML) that can be definedseparately by computer scientists, subject matter experts, and industries.These new external (as opposed to internal class-based) programming objects can be mixed andmatched in many new and creative ways by users and software developers, and each component can belocated anywhere on the web. This will be explained in more detail later in the paper.Fig. 2: New Tri-Split external programming objects
As illustrated in Fig. 2, not only can one calc engine have more than one user interface, but one userinterface can launch many different calc engines. The glue holding these pieces together are commondata and structures built using open source XML files. Note that the UIFs can be either interactiveprograms or batch command files. They can also be open source or proprietary Process Integration andDesign Optimization (PIDO) systems that meet specific needs.The fundamental change defined by this work involves splitting traditional interactive “apps” into threeseparately developed and tested parts. Calculations and data processing tools can be written separatelyby subject matter experts (SMEs), tested once, have long lives, and be used and reused for manyinteractive and automated applications.OpenCalc currently offers a UIF using an Excel spreadsheet with open source VBA code that can launchany external calc engine. The spreadsheet prompts for all input, launches the external CE or sequencedCEs, reads the results back in, and then displays, prints, and graphs the results. This UIF works withany CE without any changes to the spreadsheet or the calc engine, i.e., no extra programming orcontainerization process.The spreadsheet launches the CEs in the background and the user never knows that the calculations aredone externally from the spreadsheet. There is no delay. Results are displayed immediately even for aCE as large as a full damaged stability calculation. The benefit is that the user can now add in additionalcustom calculations, say for a specific ABS or USCG stability rule, and save the changes to a newspreadsheet. Better yet, each rule could be a separate calc engine that a user wraps with a stability calcengine to search for a hull that meets specifically-selected rules. That is how separate calculation CEscan be mixed and matched by users to solve custom problems without writing additional code. A keyelement in this process is the use of common data and data structure formats using XML. Industriesneed to exert influence over their data to enable compatible and low-cost tools.The next sections discuss each of these three external TSPA objects in more detail.4. Calc Engines (CE)The first TSPA object type is a "Calc Engine" (CE) executable batch program that has no user interface.It's a stand-alone program that reads an XML text file of input, processes it, and writes the results outto the same or different text file.Calc engines can be written in any computer language, validated separately by subject experts, used formany purposes, located anywhere on the web, and have a long life not affected by computer technologychanges. It's just a simple EXE (or other type of executable program) batch file that reads and writes atext file and acts like a subroutine - one that is tested and validated and can be used automatically inmany ways. The following UIF section will describe various classes of user interface programframeworks (UIFs) that can be constructed to launch any calc engine automatically.A calc engine is like well-known Unix filters such as "grep" (A batch program to search for a string ina text file.) except that the calculations can be as complex as computational fluid dynamics (CFD)analyses and support any program variables and data structures - not just strings in text files. Also, calcengines do not just filter out data from one text file to another. They treat the text files as random-accessdatabases of any program variable or data structure. This is described in the section on the layered XMLdefinition of data.To turn separate calculations into reusable stand-alone tools for any XML data file, TSPA requires thatCEs include an XML text file that defines all input and output data (like a subroutine argument list)using the TSPA XML schema for variables and data structures. Fig. 3 shows a TSPA/XML subroutineargument list definition for a simple calc engine (Add2.exe) that adds variable ‘A’ to variable ‘B’ toproduce variable ‘C’. (This is like a “Hello World” example for TSPA.) The UIF spreadsheet and macro
code that comes with this system can read this definition file to prompt for all input and show all outputwithout knowing any more about the external calc engine. The CE definition file allows any UIFprogram to use that calc engine without any custom programming or containerization.Fig. 3: XML CE/subroutine definition fileThese variable (VAR) definitions will be discussed in the XML section.Like Unix filters, batch CE programs can be launched from the command line, a script file, or frominside other programs and passed “arguments” just like a subroutine. For this Add2.exe calc engine, thecommand line string that would start the batch program looks like this:Add2 (A,B,C) [io MyDataFile.xml]Note that all batch programs (like Unix filters) can include string data after the name of the executablefile. That string is passed to the “main” routine of the program for processing. For calc engines, thearguments are put in parentheses and the options are put in brackets.The open source I/O code of TSPA/OpenCalc reads and parses the Add2 argument string to know whatvariables to get from the XML file. Also, like standard subroutines, arguments are passed by position,so one could launch Add2 with “(X,Y,Z)” and the code would read the input variables (X,Y) from the“io” file (MyDataFile.xml) and assign them to “A” and “B” in the calc engine. Writing out the “C”value would go into the “Z” variable in the file, creating one if it didn’t already exist. If the argumentlist becomes long, which is not uncommon, it can be put into a text file and referred to like this:Add2 [args add2args.txt io MyDataFile.xml]If no argument list is given, the CE will use its own variable names to access data in the XML data file.That mig
OpenCalc currently offers a UIF using an Excel spreadsheet with open source VBA code that can launch any external calc engine. The spreadsheet prompts for all input, launches the external CE or sequenced CEs, reads the results back in, and then displays, prints, and graphs the results. This UIF works with
