USING JYTHON TO PROTOTYPE AND EXTEND JAVA-BASED

Transcription

USING JYTHON TO PROTOTYPE AND EXTEND JAVA-BASED SYSTEMSDale Parson, Dylan Schwesinger and Thea SteeleDepartment of Computer Science, Kutztown Universityparson@kutztown.edu, on is an implementation of the interpretiveprogramming language Python written in Java anddesigned to generate Java Virtual Machine byte codes. Itprovides easy access to compiled Java classes includingsubstantial Java libraries. It also provides many of thelibraries that are popular with programmers of C-basedPython. Python‟s object-oriented language constructssupport creation of executable specifications for objectoriented Java systems. Python‟s high-level sourcelanguage mechanisms include generic container types,meta-classes, first-class functions, closures, generators,list comprehensions, source-level reflection and run-timeinterpretation of generated source code. Consequently,Jython can serve as a powerful rapid prototyping andextension environment for Java applications andframeworks. Its integration into the Java Virtual Machineallows construction of layered systems composed ofJython and Java layers. Example systems discussed in thepaper include a prototype computer game that uses Java‟sgraphical user interface libraries, a computer musicperformance system that allows users to write instrumentcontrol code as part of a performance, a Jython-Javalayered system for analyzing audio data streams, andmultiprocessor performance benchmarks written in Jythonand Java.KEY WORDSExtension language, object-oriented software, Python,Java, Jython, rapid prototype.1. IntroductionThis report on using the Jython [1,2] implementation ofthe Python programming language [3,4] to prototype andextend object-oriented Java systems [5] grows out of fourprojects that the authors have undertaken recently. Thefirst project is a graphical game for a course in advancedobject-oriented system design that illustrates the power ofusing Jython to prototype Java applications while1Portions of this work were made possible by a PASSHE (PA StateSystem of Higher Education) Faculty Professional Development Grantfor the summer of 2010, and by an Instructional Materials Grant fromthe Kutztown University Professional Development Committee.leveraging Java‟s graphical user interface (GUI) classesand other classes from Java‟s substantial class library [6].The second project is a graphical computer musicperformance application that includes support for livecoding, an established means of electronic instrumentcontrol, that in this application relies on Jython‟s ability tocompile and run extension code at execution time. Thethird project is a benchmark program that comparesperformance of a CPU intensive task on twomultiprocessor server systems in several multithreadedprogramming languages including Java and Jython. Thefourth project is an investigation into a Java streamingaudio library that uses a Java application layer forperformance-critical audio analysis tasks and a Jythonlayer for user interface construction as well as commandline exploration of the audio library. Jython, Java and Cbased Python are open source languages with freelyavailable implementations of compilers, run-timesystems, code libraries, documentation and supportcommunities.2. Related WorkPython, Tcl and Perl are representative scriptinglanguages [7] originating in the late 1980‟s that remainpopular. Interpreted languages such as LISP, BASIC andthe assorted UNIX Shell languages exerted stronginfluences on the creation of these languageenvironments. Emphases in using these languages includeinterpretation and interactive coding, dynamic typing,high-level container types such as associative arrays, andavailability of an eval function that can interpretgenerated code at run time. They provide mechanisms foreasily invoking and coordinating (a.k.a. “gluingtogether”) compiled executable programs written insystem programming languages such as C, C and Java.Python and Tcl go further in serving as extensionlanguages for application frameworks. While all scriptinglanguages provide means for dispatching and coordinatingother programs, an extension language provides softwareinterfaces whereby a developer can load compiledlibraries into the extension language interpreter at runtime and create application-specific commands for theoperations of those plug-in libraries. Scripting languagessupport program-level granularity in integrating system

programs and other scripts into aggregate programs;extension languages go further in supporting commandlevel extension of the language itself via an extensibleinterpreter and dynamic loading of compiled procedures.An extension language caters to the incremental creationof a domain-specific language via the loading of domainspecific command libraries into the language interpreter.Extensionlanguagescript plicationlibraries ncallbacks (3)Figure 1: Extension language extension mechanismsFigure 1 shows three general types of extension supportedby extension languages; arrows show direction ofprocedure invocation. With category 1 the programmercreates extension language procedures that invoke built-inlanguage mechanisms. With category 2 the programmerextends the language‟s set of built-in commands viadynamic loading of compiled, plug-in extensions.Category 3 consists of callbacks from built-in or plug-inprimitives to code written in the extension language. Forexample, handlers for events triggered in the compiledsystem programming language can be written in theextension language. An example usage of a category 3mechanism is the coding of event handlers for graphicaluser interface (GUI) events in the extension language.3. Jython in Java-based systems3.1 The Jython language and librariesJython is an implementation of the Python languagespecification written in Java. Python is a dynamicallytyped, interpreted language. The Jython interpretercompiles Python source code into Java byte code atruntime. The Java Virtual Machine (JVM) sees the sameresult from compilation of either Java or Jython – JVMbyte code. However, the code generated from the Jythoninterpreter will in most cases contain more instructionsthan a semantically equivalent block of code compiledfrom Java. This is because dynamically typed languagesput off the type decisions until runtime. Statically typedlanguages handle type checking at compile time,eliminating some instructions that are otherwise necessaryto handle type checking at runtime.The Jython distribution includes the core Python libraries.This inclusion allows for most existing Python code torun on the Jython interpreter with no modification. Thereal power of Jython over the official C basedimplementation is the ability not only to leverage existingPython libraries, but also existing Java libraries as well.The mechanism that achieves this integration of Pythonand Java code is the import statement. Unlike Java'simport statement, which is a compiler directive, Jython'simport statement is an expression that imports at runtime.Any compiled Java class is available to Jython. Duringthe import process Java reflection accesses classinformation. Reflection is the examination of objects atruntime to determine all pertinent information, such asdata members and methods. As Jython imports Javaclasses, it converts Java data types into their equivalentJython types. Writing custom Java code for a Jythonframework is far less painful than writing C or C codefor C-based Python. While the latter activity requireswriting low level adapter functions in the systemprogramming language, integrating custom Java code intoa Jython framework is accomplished via a simple importstatement.Moreover, the integration infrastructure between Java andJython also enables utilizing Python modules from withinJava code. The most common method to accomplish thisis to create a factory method in the Java code that createsan instance of the Jython interpreter, which is itself a Javaclass, to load the Python module. The module can then beconverted from a Python object to a Java object. Thisintegration requires a small amount of effort, but again itis not nearly as troublesome as writing the convolutedwrapper code that is necessary for other extensionlanguage implementations in accomplishing similar tasks.3.2 Rapid prototyping for Java-based systemsThe key for building a prototype rapidly is the ability totranslate high level abstractions into code as effortlesslyas possible. The denser the code can be, i.e. the fewerlines of code required to express a concept, the easier it isto create a prototype quickly. A language that does notrequire much boiler plate code and has high levellanguage abstractions is an ideal candidate for rapidprototyping.Python is such a language. The syntax of Python issimilar to pseudo code, and it is a fairly concise sourcelanguage. Python is an object-oriented language in thesense that every data type is an object and all of thestandard object-oriented abstractions are present,including inheritance and polymorphism. In the case ofJython, these mechanisms offer an easy way to mirrorJava class hierarchies. This makes it fairly easy to create aprototype in Jython, and once the high level designdecisions are ironed out, the classes can easily betranslated into Java equivalents.

Python offers more than just the object-orientatedabstractions. It also has implicitly polymorphic built-incomplex data types, primarily container objects that cancontain heterogeneous data, primitive and aggregate alike.Container types include sequences, associative arrays, andsets. Having these aggregate types built into the languagewith a compact syntax increases the density of the sourcecode. For example, the following line of Python code setsthe variable a to refer to a list that contains a string, aninteger, and an associative array that maps a string to astring.a ["Don't panic!", 42, {'key' : 'value'}]While the preceding code could be duplicated in Java, itwould require much more effort than the single line ofPython code.Python also supports powerful functional programmingfeatures that increase the conciseness of the sourcelanguage and ability to abstract at a high level. The corefeature that supports functional programming is thatfunctions and closures are first-class objects. This meansthat a function can be passed as an argument to anotherfunction, a function can be returned from a function, and afunction can be stored in a variable. A closure is a firstclass function that has one or more free variables boundwithin its lexical environment. Closures can be used asstateful objects. A full discussion of closures is beyondthe scope of this paper. The language also supportslambda expressions and has higher-order functionsincluding map, filter, and reduce, for building andapplying composite functions, which are typical higherorder functions in functional programming languages.Another advantage of using Jython is the fact that thecode base is compact and readable, making it easy forsomeone new to the project to use. Had we asked foroutside advice on some aspect of the game play, anyonewith the ability to read pseudocode could have understoodwhat was going on. Likewise, there is very little overheadin bringing someone new to the project up to speed.Many of the benefits above would have been apparent inpure Python as well. One advantage to using Jython isthat we were able to add a GUI by leveraging the JavaSwing library without rewriting any of the code in Java.In that sense, our prototype was not just a proof-ofconcept – we were able to use it to complete a finishedlooking game. The syntax for using Java objects in Jythonis actually more concise than it is in Java. Thus, the GUIcode looks a lot cleaner in Jython.Normally, the trade-off for using a dynamically typedlanguage is a hit to performance due to runtime typechecking. In this particular application performance wasreally not an issue because most of time the game is justwaiting for player input. There is another trade-off in thetesting phase as well. A statically typed language willcatch most type errors at compile time. Programs writtenin dynamically typed languages need to be morethoroughly tested to ensure that all expressions areproviding the correct types. The game is a relatively smallamount of code, so in this case the additional testing wasalso small.Most interpreted languages have the ability to invoke theinterpreter during runtime. Python is no exception. APython program can call the interpreter via eval and exec.When eval is called on a string that is a valid Pythonexpression, the code is evaluated and the result of theevaluation is returned. The exec statement is similarexcept any arbitrary string of Python code is executed asif it actually appeared in place of the exec statement. Execcan compile Python class and function definitions for laterexecution as byte code. Eval and exec allow for powerfulmetaprogramming capabilities.3.3 Example prototype: successes and pitfallsIn designing a graphical game, we considered a number ofpossibilities for game play that we could not effectivelyevaluate without playing the game. Rapid prototyping inJython let us quickly get to the point of testing the gameto make sure that we were on the right track. Had we notused Jython, it is possible that we would have needed tomock up a physical game first. But with Jython, we wereable to develop the code and the game play mechanicssimultaneously.Figure 2: A Java-based graphical, interactive game in JythonThe main pitfall when designing the game wasremembering that certain Java types are converted intoJython types. In most cases a given Java object is usedlike any Jython object, for example using dot notation tocall methods. But in the case of strings, the conversion

results in a Python string. So, instead of callingstring.length() as is done if the object behaved like a Javaobject, it must be called as len(string) as in the Pythonsyntax.3.4 Jython as an application-specific languageThis section outlines a project that includes support forlive coding, an established means of electronic musicalinstrument control, that in this application relies onthe previous game example, this application uses JavaGUI library classes to create the display. It also providesevent-handling classes written in Jython that extendJava‟s event-handling interfaces. These Jython subclassesof Java interfaces react to user actions and periodic timerevents.Each row in the GUI configures performance data for oneof 16 MIDI channels in the sound synthesizers. Eachchannel controls one instrument voice. Graphical controlsFigure 3: Graphical User Interface for a Jython / Java Just-Intonation KeyboardJython‟s ability to compile and run extension code atexecution time. The need for this application grew out ofa desire to perform synthesized computer music using justintonation, an approach to musical scale construction thatdiffers from the twelve-note equal-tempered scale used inmost modern Western music [8]. The primary nongraphical mechanisms of this program are construction oftables of per-note frequency information from scaleparameters, followed by translation of these frequencytable entries into ordered pairs of note number and pitchbend values for the Musical Instrument Digital Interface(MIDI) protocol [9] used to control music synthesizers.Table construction and translation occur at start-up timeusing the functional programming mechanisms of Pythonto convert custom scale parameters to frequency values,and then to convert these frequencies to MIDI values.Java‟s extensive code library contains packages for thecontrol of MIDI-based synthesizers, including a largearray of built-in instrument voice generators. Java‟s MIDIlibrary can also control software synthesizers from thirdparty vendors as well as external hardware synthesizers.Figure 3 is a screenshot of the application‟s graphical userinterface, constructed after data table completion. As withinclude buttons that play notes when pressed, spinnersthat adjust the octave, volume and instrument voice of arow‟s notes, and combo boxes for several parametersincluding change of key and target MIDI synthesizer.MIDI design is based on the Western equal-temperedscale, so that each channel can play only one note at atime in a non-equal-tempered scale, doing so by adjustingthe pitch of an equal-tempered note using MIDI pitchbend messages. This note-at-a-time constraint, known asmonophonic synthesis, makes it impossible to play chordsand multi-voice parts using standard musical keyboardtechniques. Also, a computer mouse supports pressingonly one button at a time. Partial solutions to thismonophonic constraint include the checkboxes to the leftof the note buttons for latching key presses, therebysustaining notes, along with a 16 x 4 crossbar ofcheckboxes that allow a button press on a single row to beused in all other rows connected to that row via a crossbarcolumn. The crossbar enables the simultaneous soundingof chords by multiple synthesized instruments, i.e.,polyphony.

The most important aspect of this musical interface forthe current discussion is the use of live coding, wherebyperformers can write and modify snippets of stylizedPython code at performance time to control a row of GUIcontrols in Figure 3 in a way that is reminiscent of aplayer piano.def genfunc(s): # „player piano‟ for MIDI channelimport randomdef r(lower, upper):return random.randint(lower,upper)dur 1.0de 1s.p 80s.o -1while True:s.b2 duryield deif r(0,3) 0: # do this 25% of the times.b3 duryield des.b1 duryield des.b0 duryield deperiodically without losing the bindings of its localvariables. The program also supports regular Pythonfunctions that return the delay until next invocation as areturn value at the end of each invocation. The GUI eventthread invokes execution of any channel‟s activated codeat its scheduled time under the scheduling control of aJava GUI Timer object. Buttons at the bottom of theeditor window of Figure 3 allow a performer to compileand run code after an edit, to cancel an edit or todeactivate the performance code on a channel. The GUIcontrols on a row remain active for performer interactioneven when live code is scheduled for that row.class PJCheckBox(JCheckBox):def getValue(self):return self.isSelected()def setValue(self, value):issel self.isSelected()if value and not issel:self.doClick()elif issel and not value:self.doClick()v property(getValue, setValue, None)Listing 2: Java checkbox class adapted in a Jython subclassListing 1: Live Python code for controlling the GUI keyboardListing 1 shows the code for one MIDI channel thatappears in the pop-up code editor window of Figure 3when a performer clicks that row‟s “Code” button. Thiscode is pure Python, which the application interpretsperiodically under the scheduling control of the code itselfwithin the run-time context of the GUI. Listing 1illustrates that live coding can import any Python librarysuch as the random module used here. Mnemonic namesthat appear across the top row of Figure 3, such as “b2”for note button 2 or “o” for the octave spinner on the GUI,provide the symbols through which a performer controlsthe GUI via live coding. While a name such as “b2” doesnot appear to be mnemonic, the use of terse names allowsfor minimal typing when using the pop-up text editorwindow during a performance. The appearance of theseterse names above their respective columns in the mainGUI window makes memorizing them unnecessary.The code in Listing 1 sets values for local variables durfor duration and de for delay in temporal units scaledaccording to a tempo initialization parameter. It sets thechannel‟s instrument voice spinner to MIDI “patch”number 80 via “s.p 80” and sets the octave spinnerusing the “s.o -1” assignment. The “s” symbolrepresents the scope of the row, and the properties such as“.p” and “.o” represent controls in that row.After initializing its variables and these two GUI controls,the code of Listing 1 goes into an unbounded loopwherein it presses one of the row‟s buttons for a durationof dur via an assignment statement (e.g., “s.b2 dur”),then yields control back to the GUI event thread for deunits of time. The code of Listing 1 comprises a Pythongenerator, that acts as a coroutine by yielding controlFour Python programming constructs make the livecoding shown in Figure 3 and Listing 1 possible. First isthe creation of Jython subclasses for the Java GUI controlclasses that add the ability to read and write their valuesas simple integers and strings via getValue and setValuemethods. Listing 2 shows an example subclass of Java‟sJCheckBox class. These subclasses allow the GUIcontrols to provide an identical pair of getValue /setValue methods, regardless of the GUI control class.Subclasses serve as adapters that make reading andwriting controls homogeneous with respect to methodnames, number of parameters and return values. Python‟sdynamic typing makes this task easier to achieve than it iswith Java, because the methods parameter types of JavaGUI controls vary according to control class, although itcould be done via java.lang.Object parameter and returntypes and explicit type checking in Java.Jython‟s subclass of Java‟s JButton control class was themost complicated control class to write because of theneed to emulate timed button presses and releases acrossrows tied together with a crossbar, but after completionthe button presses could be emulated in software ortriggered by user actions.The line “v property(getValue, setValue, None)” at thebottom of Listing 2 shows the second of the four pertinentPython constructs, which is the ability to create classproperties whose getValue / setValue methods areinvoked when the property is read / written respectivelywithin a Python expression. In the object representing arow in Figure 3, all of the GUI controls are accessible viamnemonic property names appearing across the top row,such as “s.b2” which is the b2 property of the row, this

property being tied to getValue / setValue method pairsfor the button object labelled “b2.” Tying GUI controlobjects to mnemonic property names makes it possible forGUI values to appear as rvalues (implicitly invokinggetValue) in expressions and as lvalues (implicitlyinvoking setValue) in assignment statements such asthose appearing in Listing 1. The result is minimalistsyntax for a musician transforming live code duringperformance, avoiding the more complex, alternativefunction invocation syntax.The third pertinent Python construct is support for addingnew fields, methods and properties to individual Pythonobjects as well as their defining classes at run time. Thisprogram has a class called environ with sixteen instances,one per MIDI channel. The “s” object referencemanipulated by statements such as “s.b2 dur” in Listing1 is an environ object reference. The environ classprovides a method for adding properties such as PJButton“b2” at run time as the GUI is constructed. Thismechanism simplifies tying together GUI objectconstruction with extension code access to GUI objectsvia incremental addition of these objects as symbolicproperties to the environ objects that are manipulated bythe live code.The fourth and most powerful construct is application ofPython‟s exec function for compiling live coding sourcefunctions, such as genfunc of Listing 1, at execution timewithin the scope of class environ and the MIDI channel‟sscope object. A musician‟s live code is compiled to bytecode via exec and then placed on a scheduling queue thatis maintained by Java‟s GUI Timer class. Compilationand scheduling occur when a performer clicks theCompile button at the bottom of the code editor windowof Figure 3. In this way user scripts can access GUIcontrols as symbolic names within algebraic expressionsas they appear in Listing 1 (s.x0, s.b0, etc.), and theunderlying machinery outlined above takes care of thework of converting appearance of these symbols-inexpressions into method invocations.memory multiprocessors running multithreaded Jython,Java and C code. The second is an overview of a twotiered approach to using an extension language, whereJython serves to configure and direct time-critical signalprocessing activities occurring in a Java thread.The first program to discuss is one of a series ofbenchmarks exploring the performance of severalmultithreaded algorithms using several programminglanguages on two shared-memory multiprocessor servers,a 64-processor Sun Sparc server with limited per-corecache [10] and a 16-processor Advanced Micro Devicesserver with copious per-core cache [11]. Table 1 showsexecution results in seconds for running a multithreadedsolution to the N Queens Problem that finds all solutionson a 15 x 15 chess board using Jython, Java and C onthese two multiprocessors. N Queens represents a searchproblem with a high number of search states, but withlimited demands on memory. The algorithm spawnsparallel search threads when advancing to adjacentcolumns in the board, with the number of threads being afunction of number of concurrent columns, configured asa run-time parameter. Table 1 shows execution seconds asa function of the number of software threads for eachmachine-programming language pair.threadsSparcC SparcJavaSparcPythonSparcJythonAMDC 178.1085807.91111283.7327065.999025.306Table 1: Search time seconds as a function of thread countIn a Java program without extension language support itwould be necessary to design a custom live codinglanguage, capture its grammar, build a scanner and parserusing custom code or a complex parser generatorpackage, compile live code functions, then schedule andinterpret their execution. Thanks to Jython‟s support forrun-time compilation and execution of Python sourcecode, this project has avoided the creation of a customlanguage, along with the compiler and run-time supportfor its execution. This live coding facility grows directlyout of Python, extending it into being a domain-specificlanguage for live coding.The most obvious result is that Jython is the slowest ofthe languages on both hardware platforms, despite the factthat Jython generates and executes Java byte code. Jythonis slower even than C-based Python. This fact isinteresting because C-based Python does not supportconcurrent execution of multiple threads within thePython interpreter. It is necessary to spawn multiplePython processes in order to achieve concurrent use of amultiprocessor. Jython, which supports multithreadingwithin a single process, is nevertheless much slower.Also, its multithreading performance is worse than itssingle threaded performance on the AMD machine.3.5 Layered systems and performanceThis section looks at performance issues from twoperspectives. The first is processing speed on two shared-Perhaps a more important result is the fact that Javaexecution speed is on par with C execution speed onboth platforms, in fact beating C performance in

multithreaded AMD runs. This result appears in otherbenchmarks as well. In many cases multithreaded Javasurpasses optimized, multithreaded C in speed on theSparc machine. This result appears to be attributed largelyto advances in run-time, profiling-based optimization bythe Java just-in-time compiler that translates Java bytecodes to machine code during execution.The conclusion, that Java execution has become fastenough to deal with some classes of real-time processingincluding audio signal processing, led to the architectureof the final project of this study. Diagrammed in Figure 4,this two-tiered software architecture uses Jython toconfigure a signal processing thread, coded in Java, thatreads input audio streams, transforms them, for exampleby mixing, adding delays and other effects, and sendsthem to output streams. Jython‟s first use in thisarchitecture has been as a means to explore thejavax.sound.sampled Java library interactively. Thislibrary includes reflective data access methods thatdescribe the audio input and output streams in symbolicterms. A programmer can use Jython interaction withreflective audio classes, in conjunction with the librarydocumentation, to learn how to use library components.After that the programmer can write Jython GUI andconfiguration code that interacts with Java components inconfiguring a Java signal processing thread byinterconnecting Java signal processing objects used by theJava thread. Jython can access locks and other Javaobjects from java.lang.concurrent in synchronizingconstruction of an audio signal processing flow. This twotiered architecture is an ideal match for the relativestrengths and weaknesses of Jython and Java. Jythonprovides interactive access to reflective (self-describing)classes and fast coding for constructing a GUI andconfiguration classes, and Java provides fast execution ofa signal processing thread.audioinputsconfigureaudiosignalthreadJava signalprocessing threadPython as a source language supplies object-orientedconstructs that make it a good fit for specifying andprototyping object-oriented Java systems. Python alsosupplies powerful functional programming constructs andgeneric container types built into the language. Thesecapabilities make Python a powerful vehicle for rapidprototyping by enabling concise, specification-likeconstruction of prototype code.Jython‟s support for interactive, run-time compilation ofPython source code procedures into Java byte code meansthat Jython can serve as a domain-specific language for anapplication written in Java. Interpretation, compilationand extension mechanisms built into Python and Jythoneliminate

the Python programming language [3,4] to prototype and extend object-oriented Java systems [5] grows out of four projects that the authors have undertaken recently. The first project is a graphical game for a course in advanced object-oriented system design that illustrates the power of using Jyt