WIXLIB

Journal of Advances in Information Technology Vol. 12, No. 4, November 2021implements a fundamental data structure oralgorithm. Element Fill-in-Blank Problem (EFP): EFPrequests to fill in the blank or missing elementswith the proper words in the given source code.The locations of the blank elements are explicitlyshown in the source code. Code Completion Problem (CCP): CCP requeststo fill in the blank or missing elements with theproper words in the given source code, like EFP.However, their locations are not shown in thecode. Students need to discover the locations, andcomplete the whole statements. Code Amendment Problem (CAP): CAP requeststo amend the incorrect elements in the sourcecode. The incorrect elements are either missing orwrong. Students need to find out the locations ofthe incorrect elements, and complete the wholestatements. Code Correction Problem (CRP): CRP requeststo correct the incorrect source code so that it canpass the given test code on JUnit. The sourcecode has several errors that cannot be passed bythe test code. Statement Element Fill-in-Blank Problem (SFP):SFP requests to fill in the blank statements in thegiven source code so that it can pass the giventest code on JUnit. Code Writing Problem (CWP): CWP requests towrite a source code that passes the given test codeon JUnit. To help a student, the detailedinformation for the source code implementationis usually described in the test code.In JPLAS, the answer to each problem will be markedautomatically on the server using the program. For VTP,EFP, CCP, and CAP, the answer is marked by the stringmatching with the correct one that is stored in thedatabase. For CRP, SFP, and CWP, the answer is markedby the software testing using the test code on JUnit.A. Definition of Grammar-Concept UnderstandingProblemA GUP instance consists of a Java source code, a set ofquestions, and the correct answers to the questions. Eachquestion describes a basic grammar concept in Javaprograming that appears in the source code, and requeststo pick up the corresponding element or keyword in thesource code. The student answer is marked by the stringmatching with the corresponding correct answer.B. Example of GUP InstanceHere, we show an example of the GUP instance.source code1 shows the source code.source code11 import java.util.Scanner;2 public class UserIntegerInput {3 public static void main(String[] args) {4Scanner scanner new Scanner(System.in);5int num scanner.nextInt();6 }7}Then, the set of questions and the correspondingcorrect answers are given as follows:1) Which keyword is used to refer to the classes andinterfaces in other packages? (import)2) Which library needs to run the Scanner class?(java.util.Scanner)3) Which keyword allows from other class in Line2?(public)4) What is class name? (UserIntegerInput)5) Which keyword allows the method to run withoutcreating an object? (static)6) Which keyword describes no returning data inLine 3? (void)7) Which keyword represents the entry point fromwhich the JVM can run this program? (main)8) Which data type is used in Line 3? (String)9) Which keyword represents the parameterspassed to the main method? (args)10) What is the object name of Scanner class?(scanner)11) Which keyword is used to create a new object orinstance? (new)12) Which keyword represents the standard inputstream that passes the predefined object forcreatinganobjectofScanner class? (System.in)13) Which data type is used in Line5? (int)14) Which method is used to scan the next token ofthe integer input? (nextInt)It is noted that the correct answer is indicated insidethe blankets. Fig. 2 illustrates the user interface for thisGUP instance.C. LimitationThese exercise problems assume that the students havealready acquired the basic grammar concepts andkeywords of Java programming in the lectures withtextbooks. To avoid a huge dropout from the course dueto insufficient knowledge, teachers should confirm theunderstanding levels of students in basic grammarconcepts and keywords of Java programming, and helpthe students out who may not catch up with them.Therefore, JPLAS should provide a new type ofprogramming exercises that directly ask the grammarconcepts or keywords that appear in a source code. In thenext section, we will present the Grammar-ConceptUnderstanding Problem (GUP) for further studies.III.PROPOSAL OF GRAMMAR-CONCEPTUNDERSTANDING PROBLEMIV.In this section, we introduce the GUP instancegeneration algorithm to assist a teacher to generate a newGUP instance among the selected source code.In this section, we present the definition of theGrammar Concept Understanding Problem (GUP) andthe algorithm to automatically generate a GUP instance. 2021 J. Adv. Inf. Technol.GUP INSTANCE GENERATION ALGORITHM344

Journal of Advances in Information Technology Vol. 12, No. 4, November 2021Figure 2. GUP user interface in JPLAS.The proposed algorithm uses the keyword list in TableI to list every possible keyword to represent the basicgrammar concepts to be studied through solving GUPinstances. The keywords are categorized into the fourtypes, depending on the uniqueness of the selectedquestion and correct answer.1) For the type-1 keyword, both the question and thecorrect answer are unique for any source code. Thekeyword itself becomes the correct answer.2) For the type-2 keyword, the question is unique forany source code. But, the correct answer must be foundfrom the source code using the keyword.3) For the type-3 keyword, the question contains theline number information in the source code to specify thekeyword, since otherwise, the question can be related toother keywords in the code. The line number must befound from the source code to complete the question. Thecorrect answer is unique for any source code, where thekeyword itself is the correct one.4) For the type-4 keyword, the question has multiplechoices, depending on the concept that the teacher wantsto ask to students. One question contains the line numberinformation in the source code to specify the keyword,which must be identified from the source code tocomplete the question. The correct answer is unique forany question, where the keyword itself is the correct one.A. Input FilesTo use the algorithm, a teacher needs to prepare thefile of the source code that covers the grammar conceptsto be studied by students through solving the GUPinstance. Then, the algorithm will read the source codefile and generate the GUP instance file through theprocedure in Section IV-D. The files for the keyword listand the question list must be prepared beforehand for thisalgorithm.B. Keyword ListTABLE I. KEYWORD LISTTypeQuestionAnswer1UniqueUnique2Uniquein code3in codeUnique4MultipleUnique 2021 J. Adv. Inf. Technol.Keywordsfor, while, do, try, catch,ArithmeticException,NullPointerException, finally,throw, throws,read, IOException, close, void,static, main, args, java.util.Scanner,new, System.in, nextInt, nextLine,extends, this, implements, return,abstract, instanceof, valueOfclass, interface, package, Scannerint, long, short, byte,double, float, Stringpublic, private, protected345

Journal of Advances in Information Technology Vol. 12, No. 4, November 2021TABLE II. QUESTION endsnextIntclasspackagescannervoidint, long, short, byte,double, float, String34public4private4ProtectedQuestionsWhich keyword represents Looping?Which keyword represents Looping?Which keyword always have to execute the loop at least once?Which keyword indicates the following lines may cause errors?Which keyword checks the error message when the exceptions occurred in the try block?Which exception is thrown when an exceptional condition has occurred in an arithmeticoperation?Which exception is thrown when referring to the members of an object is nothing?Which keyword represents the block that is always executed whether exception is occurredin the try block or not?Which keyword is used in method body to declare the exceptions that can occur in thestatements present of the method?Which keyword is used in method signature to declare the exceptions that can occur in the tryblock?Which method reads a byte of data from this input stream?Which exception is thrown when an input-output operation failed or interrupted?Which method is used to terminate this file input stream and releases any systemresources associated with the stream?Which keyword allows the method to run without creating an object?Which keyword represents the entry point from which JVM can run this program?Which keyword represents the parameters passed to the main method?Which keyword is necessary to inherit from the super class in the sub class?Which method is used to scan the next token of the integer input?What is class name?What is the package name?What is the object name of Scanner class?Which keyword describes no returning data at Line#?Which data type is used in Line#?What is the access modifier at Line #?Which keyword allows from any other class in Line#?What is the access modifier at Line #?Which keyword prohibits the access to this code from any other class?What is the access modifier at Line#?Which keyword allows the access to this code from other class only in the same package?5) If the same pair of the question and the correctanswer is selected, discard them as the duplicatedquestion.6) Output the GUP instance file of the source code,the questions, and the correct answers.For example, for source code1 in Section III-B, thefollowing keywords are extracted: Import, java.util.Scanner, public, class, UserIntegerInput, public, static, void, main, String, args, Scanner, scanner, new, Scanner, System.in,int, num, scanner, nextInt.Then, the 14 questions and the correct answers inSection III-B are selected from these keywords.C. Question ListThe question list in Table II is used to list the questionsfor each keyword. It is noted that Table II shows the partof the questions due to the limited space. For the type-1or type-2 keyword, the corresponding question is uniquefor any source code. For the type-3 or type-4 keyword,the question can be completed after locating the linenumber of the source code where the keyword appears. Inthe question list, the line number is described by # thatmust be replaced by the line number.D. GUP Generation ProcedureA GUP instance file is generated through the followingprocedure:1) Read a Java source code file.2) Extract he keywords in the keyword list from thesource code.3) Select the question in the question list thatcorresponds to each extracted keyword.3-1) If multiple questions are registered in thequestion list for the keyword, one of them israndomly selected.3-2) If the question needs to find the line numberof the source code for the keyword, it isfound and included in the question.4) Find the element as the correct answer from thesource code. 2021 J. Adv. Inf. Technol.V.EVALUATIONIn this section, we evaluate our proposal throughapplications to undergraduate students in two universitiesin Indonesia and Myanmar respectively.A. Application to Students in Indonesia UniversityFirst, we apply the proposal to students in a universityin Indonesia.1) Application Overview: We generated 20 GUPinstances with 99 questions from different source codesthat cover the topics of the basic Java grammar. Here, the13keywords,for,while,do,try,catch,346

Journal of Advances in Information Technology Vol. 12, No. 4, November 2021ArithmeticException, NullPointerException, finally,throw, throws, read, IOException, close, are notincluded in these source codes. It is confirmed that thegenerated questions are suitable for the level of novicestudents. Then, we required 100 undergraduate Indonesiastudents to solve the problems using the offlineanswering function [11].The results show that 87 students among them havelearned the fundamentals of Java grammar and shouldpursue an advanced level. However, the remaining 13students fail to achieve the required level. Thus, theteacher needs to take care of these students and provideadditional instructions and assignments.2) Correct Answer Results: Based on the result, first,we analyzed the performance by the number of correctlysolved questions, then divided the 100 students into fivegroups. Table III shows the range of the number ofcorrectly solved questions, the number of students, therange of the number of instances attempted to be solved,and the average number of answer submission times perstudent with its standard deviation for each group.well understand the questions and submitted theiranswers randomly. Furthermore, in group VIII, 7 studentsdid not reach even 20 submissions. The teacher needs tocare these 13 students.TABLE IV. SUBMISSION TIMES RESULTS# of solvedquestionsrange# ofstudentsABCDE999890-9789-5128-051102298# ofattemptedinstancesrange20201919-144-0ave. # ofsubmissions(SD)86.0 (58.6)99.9 (43.8)90.1 (83.1)63.7 (49.2)17.7 (14.0)The table indicates that in group A, 51 students among100 solved all the questions correctly. In group B, 10students did not solve only one question where theyattempted to solve all the 20 GUP instances. In group C,22 students solved 90 or more questions correctly wherethey did not attempt to solve one GUP instance. In groupD, 9 students solved less than 90 questions correctlywhere they did not try to solve several GUP instances. Ingroup E, 8 students only solved less than 28 questionscorrectly where they attempted to solve a few GUPinstances. Hence, the teacher may spend more time ontaking care of the students in group E.B. Submission Times ResultsNext, we analyzed the performance according to thenumber of times of answer submission. JPLAS allows thestudents to submit their answers to the server at any time,because it is the tool for self-studies. Table IV shows therange of the answer submission and the correspondingnumber of students.The table suggests that in group I, 29 students among100 submitted their answers 50 or less times to solve 20GUP instances, which indicates less than 2.5 submissionsfor each instance on average. These students havethoroughly understood the questions, and carefullyprepared the answers before submissions. In groups V, VIand VII, six students submitted answers 200 or moretimes, which indicates more than 10 submissions for eachinstance on average. It seems that these students did not 2021 J. Adv. Inf. Technol.Submissiontimes range# 200-250250-30015632VII300- 3501VIII0- 197C. Application to Students in Myanmar UniversityNext, we apply the proposal to students in oneuniversity in Myanmar.1) Application Overview: In this application, weadditionally generated 8 GUP instances with 30 questionsfrom source codes that cover the 13 keywords that werenot included in the previous 20 source codes. Then, weasked 29 undergraduate students to solve the instancesusing the offline answering function, where among them,only 15 students solved both the previous 20 instancesand the additional 8 instances.The results confirm that all the students have acquiredthe fundamentals of Java grammar and may continuestudying Java programming. This difference from theIndonesia students may come from the difference in themotivations of the participated students between the twouniversities. In the Indonesia university, the teacherrequested all the students in the class to answer the GUPinstances. On the other hand, in the Myanmar university,the teacher allowed the students to do so voluntarily.Thus, only the self-motivated students might answer theinstances.2) Correct Answer Results: Tables V and VI show therange of the number of correctly solved questions, thenumber of students, the range of the number of instancesattempted to be solved, and the average number ofanswer submission times per student with its standarddeviation for each group, for the previous instances andthe additional instances, respectively. Table V indicatesthat all the students solved 90 or more questions amongthe 99 correctly where they did not attempt to solve oneor two instances. Also, Table VI signifies that all thestudents solved 28 or more questions among the 30correctly where they tried to solve all the instances.3) Submission Times Results: Tables VII and VIIIdemonstrate the range of the times of answer submissionand the corresponding number of students, for theprevious instances and the additional instances,respectively. Table VII suggests that every studentsubmitted answers less than 8 times on average for eachof the previous 20 instances, and Table VIII does thatTABLE III. CORRECT ANSWER RESULTSGroupGroup347

Journal of Advances in Information Technology Vol. 12, No. 4, November 2021every student submitted answers less than 7 times onaverage for each of the additional 8 instances.there were five categories. Four of them were symbolrelated or keyword-related errors (invalid symbols orkeywords, mismatched symbols, missing symbols, andexcessive symbols) and the last was naming-related error(inappropriate naming error).In [15], Okimoto et al. developed a learning supportsystem for C programming that will automaticallygenerate a source code to facilitate the programminginstruction through code reading, which is effective forimproving basic skills by tracing and debugging,supporting novice learners who feel difficult inprogramming concept. The system proposes a questionthat requires learners to answer the proper value of avariable after the execution of the code. The authorsutilized the system in a programming course with 108first year students majoring in informatics, and clarifiedthat the program reading comprehension is challengingfor novices.In [16], Jegede et al. analyzed error types and patternsby undergraduate students in Java programming based onfundamental concepts of methods and classes, decisionmaking, object concepts, and looping. The resultsrevealed that similar error types were found across abilitylevels where students should be instructed based onachievement levels, and learning Java programmingshould be accomplished with an unintelligent editor.TABLE V. CORRECT ANSWER RESULTS FOR PREVIOUS INSTANCESGroup# of solvedquestionsrange# ofstudents# ofattemptedinstancesrangeave. # ofsubmissions(SD)ABC999897- 901838202019 - 1871.4 (40.3)66.7 (17.4)48.9 (20.2)TABLE VI. CORRECT ANSWER RESULTS FOR ADDITIONAL INSTANCESGroup# of solvedquestionsrange# ofstudents# ofattemptedinstancesrangeave. # ofsubmissions(SD)ABC302928120388819.4 (12.7)0.0 (0.0)11.7 (2.9)TABLE VII. SUBMISSION TIMES RESULTS FOR PREVIOUS INSTANCESsubmissiontimes range20 – 5050 – 100100 – 150GroupIIIIII# ofstudents12125TABLE VIII. SUBMISSION TIMES RESULTS FOR ADDITIONALINSTANCESGroupsubmissiontimes rangeIIIIII8 – 3030 – 4040 – 50VI.VII.This paper proposed the Grammar-ConceptUnderstanding Problem (GUP) as a new type exerciseproblem in JPLAS. A GUP instance gives questions ongrammar concepts or behaviors in the code. Each answermay be a number, a word, or a short sentence, whosecorrectness is marked through string matching with thecorrect answer.For evaluations, 28 GUP instances with a total of 129questions from simple source codes on fundamental Javagrammar were generated and assigned to 100 students inone university in Indonesia and to 29 students in oneuniversity in Myanmar respectively. The results showthat the proposal is effective in identifying the studentswho do not understand Java programming well and needmore instruction from the teacher.For the limitations, this algorithm involves severallimitations. When the same keyword appears in thesource code again, the algorithm will generate the samequestion for the keyword. A large number of Java sourcecodes may have common keywords. For those commonkeywords, the corresponding same questions areduplicated even for one problem. To avoid it, the teacherneeds to remove the duplicate or redundant questionsbefore presenting the GUP instance to students.In future works, we will generate a variety of questionsfor advanced Java programming topics using variouscodes and apply them to students in Java programmingcourses.# ofstudents1221RELATED WORKSIn this section, we discuss related work in literature. In[12], McIver et al. discussed seven undesirable featuresin programming language

Abstract—Nowadays, Java has been extensively adopted in practical IT systems as a reliable and portable object-oriented programming language. To encourage self-studies of Java programming, we have developed a Web-based Java Programming Learning Assistant system (JPLAS). JPLAS provides several types of exercises to cover different levels.