WIXLIB

SBC – Proceedings of SBGames 2013Art & Design Track – Full PapersAs for the present work, the content-based recommendationsystems, which employ similarity analysis techniques, are veryusefully to the suggestion of the most related GDP of a gamebased on the analysis of their content. The patterns wikicontains the full descriptions of the patterns and we can obtainthe game descriptions from many sources from the web. Theexplanation of the techniques and the process employed aredescribed in the following section.III.THE EXPERIMENTThe experiment consisted of selecting a game, obtaining itsdescription and matching it with all the GDP descriptions inorder to find the patterns which have the most similar content,thus suggesting them to the user. To achieve this, a softwareapplication was built in order to automate the entire process.A. Methods and ToolsA software application (Fig. 2) to execute the experimentwas built using the Java platform and Apache Lucene [11], alibrary for text processing. The software was a key element inthe experiment once it provides functionalities that allowed usto agilely extract and check the necessary data. By using theapp, we were able to get the patterns suggestion for the gameby inserting the game’s name in the search field. As can beseen in Fig. 2, we could also configure the parameters of thesimilarity analysis process. Those parameters are: the numberof terms of each document that will be used in the similaritymatrix, the number of reviews for the selected game that willbe processed during analysis and the size of the patterns list tobe shown in the result box.Beyond helping us to collect the necessary data andexecuting the similarity algorithms, the tool allowed us toreduce the load of checking the correspondence between eachsuggested pattern and the game by providing the patterndescription (this process is described in a later section).Another feature implemented in the software is the ability toplot a relationship map of a game design pattern for quicklyvisualization (Fig. 3). By looking into the map we can checkthe suggested patterns and the patterns to which these relate.While not specifically valuable to the experiment of this paper,it will serve in future works.Fig. 3. Relationship map of a game design pattern. The related patterns arepositioned around the selected pattern, which is at the center.The software application downloads all the necessary datafor the experiment from specific websites. The patternsdescriptions are obtained from the GDP project wiki. Thetextual data is then cleaned (removes formatting) in order toextract only the desired information to the experiment, whichare the patterns description. For the game description, therewere two options as source of data: game manuals or gamereviews. Game reviews were considered more appropriated tothis experiment as they contain game design analysis, whereasgame manuals are more concerned in teaching the user on howto play the game. Thus, the website GameFaqs2 was used as thesource of game reviews as it has many reviews for each game.The entire process performed by the software application torecommend patterns can be summarized in the following steps(see Fig. 4):1.When first used, the application will download, extractand clean the descriptions of all the patterns from theGDP project wiki. (The information regarding thepatterns are indexed with Lucene and stored locally forbetter performance).2.The user enters a game name;3.The software obtains and cleans the game reviewsfrom GameFaqs website;4.The data is indexed with the Lucene library and storedin a proper data structure.5.The game description data is compared with thedescription data of each pattern using a similarityalgorithm, which indicates how close the two textualcontents are.6.At the end, the suggestions of GDPs for the game arelisted in a descending order of similarity.The data structure, the indexing process as well as themethods for similarity analysis are described in the nextsubsections.Fig. 2. Screenshot of the suggestion tool built with Java.2http://www.gamefaqs.com/ (visited on 2013/06/24)75

SBC – Proceedings of SBGames 2013Art & Design Track – Full Papers[10] between them can be calculated. Singhal [15] states thatthe cosine similarity technique gives a useful measure of howsimilar two documents are likely to be in terms of their subjectmatter. This is especially useful for the experiment described inthis paper. We are interested in finding documents with similarmeaning and not character-level similarity, which wouldn’t saymuch about how close are a pattern and a game. With thisobjective in mind, we later combined another techniquesuggested by [10] that is taken as useful for measuring thesimilarity of meaning between documents: the TF.IDF score.Fig. 4. Operation method of the suggestion tool.B. Similarity analysis method employedThere are some techniques that can be used to denote howsimilar two documents are. In our case, we want the algorithmto tell us how similar a game description and a patterndescription are. After discovering the similarity between eachpattern and the game description, we can sort the resulting listin a descending order of similarity: from the most to the leastsimilar.The first step of the similarity algorithm between twodocuments employed consists of extracting all the terms (wordsor other strings of characters other than white space [10]) fromthe document content. This is done with the Apache Lucenelibrary, which removes the stop words and stems the resultingterms.According to [10], stop words are the least important termsin the processing of natural language, once they help to buildideas but do not carry any significance by themselves.Examples of stop words are articles and pronouns. In theexperiment, we used the same stop words dictionary fromMySQL [12], Onix [13] and Ranks [14] systems.After removing stop words from the text, the Lucene stemsthe remaining terms. The stemming is the process of reducinginflected words to their root form [10]. As an example, thewords “stemmer”, “stemming” and “stemmed” are reduced tothe root form “stem”.After the processing is completed by Lucene, the (root)terms of a document are stored into an associated onedimensional matrix, which is used to store the terms frequencywithin the document: the number of times the term appears inthe document. Each document has its own frequency matrix.The matrix length is equal to the total number of terms of alldocuments. Each position in the matrix corresponds to a termfrequency inside the document (replaced later in thisexperiment by the TF.IDF score [10] for better results, asdiscussed bellow).When the frequency matrices of the two documents – gamereview and pattern description – are done, the cosine similarityAfter the first run of experiments, we soon noticed that thepattern suggestions tool was overvaluing common terms like“game” or “player” in the frequency matrix, as they really werethe most common terms in the analyzed documents. However,the importance of these terms was very low to our analysis.Thus, we had to consider the importance of each word within adocument to discover its most meaningful terms. For this, wereplaced the frequency matrix by a matrix of TF.IDF scores ofthe terms. The TF.IDF (Term Frequency times InverseDocument Frequency) gives a score based on the importance ofa word in a document [10], which gave us better results in theexperiment (as demonstrated in the next section). The TF.IDFobtainment process is explained hereafter.C. The TF.IDF ScoreAccording to [10], the TF.IDF score can be used to find theterms that best describe the document they belong to. To obtainthe score of the importance of a term i in a document j, we usethe equation (1).(1)score(i, j) TFij x IDFiThe TFij in (1) represents the Term Frequency of the term iin the document j. The IDFi in (1) represents the InverseDocument Frequency of the term i over the entire collection ofdocuments under analysis. They are calculated by the equations(2) and (3), respectively.(2)TFij fij / NtermsjThe equation (2) shows us how the term frequency iscalculated. In (2), the frequency of the term i in the document j(fij), which is the number of times the term occurs in thedocument, is normalized by the total number of terms of thedocument (Ntermsj). If a term count is equal to the totalnumber of terms in the document, then the TF for this term willbe 1. Otherwise, the TF will have a value between 0 and 1.Themore the term appears in a document, the higher will be its TFvalue.(3)IDFi log2(Ndocs/Ni)In equation (3) we have the obtaining of the inversedocument frequency (IDF), which gives us the presence of theterm i in the entire collection of documents. Ni represents thenumber of documents in which the term i appears and Ndocs,the total number of documents in the collection.76

SBC – Proceedings of SBGames 2013Art & Design Track – Full PapersThe games chosen to be tested were classic games from thethird generation of videogame consoles. A total of three gameswere used to run the experiment: Super Mario Bros. [16], NinjaGaiden [17] and Tetris [18]. Two of these games were selectbecause they were the best selling games of their platforms[19] [20]: Super Mario Bros. for the NES [21] and Tetris forthe Game Boy [22]. The third game, Ninja Gaiden, also for theNES platform, was chosen by a personal preference of theauthors.Correct patternsThe experiments were done comprising various trialstowards better suggestions of patterns (the results are shown inthe next section). By better suggestions we understand havingmore patterns related to a game in the list showed by thesoftware tool. Instead of using just one game review, weprogressively tested the patterns suggestions against all theavailable reviews for each game. In all the tests, weexperimented the two similarity algorithms previously pointed,cosine similarity with frequency matrix and cosine similaritywith TF.IDF score matrix. The former was the firstimplemented in the software and the later, as a measure toimprove the suggestion mechanism.1816141210864201234525Correct patternsD. Running the ExperimentThe results of the experiment discussed in this paperrepresent a work in progress. The software tool built is notcomplete and mature enough to be used in real world scenarios,but the achievements done so far are very positive andencouraging. We chose to show the gradual results of theexperiment instead of only the final (and better) results as away to report the complete process and justify the e reviewsCosine with Frequency MatrixCosine with TF.IDF MatrixFig. 7. Progression of the correct suggestions of patterns over the number ofreviews for the game Super Mario Bros (NES).E. Results and discussionsThe results for the experiments are synthesized in thegraphs shown in the Fig. 5, 6 and 7. We ran the software toolfor each one of the chosen games and collected the list ofpatterns suggested. We then checked each of the patterns to seewhich were accepted for the games, which means that, wemanually verified if the suggested patterns were reallycontained in the games. This was comprised of a manual taskof reading each one of the patterns description and verifying itsacceptance to the game’s concepts. Then, we counted howmany accepted patterns the tool suggested and used this as ameasure of evaluation for the experiment. It is important notethat, as a recommendation system, the tool doesn’t “know”when a pattern is accepted or not, it simply recommend it inorder of similarity to the game description, in terms of theirsubject matter.Fig. 5, 6 and 7 show the graphs that contain the tests resultsfor the three games selected for the experiment. As can beobserved in the graphs, using the cosine similarity mixed withthe TF.IDF score matrix gave notably better results. Theseresults help to corroborate with the suspicion that, whencombined, the cosine similarity and the TF.IDF score form auseful tool to measure the similarity between two documents interms of their subject matter.Game reviewsCosine with Frequency MatrixCosine with TF.IDF MatrixCorrect patternsFig. 5. Progression of the correct suggestions of patterns over the number ofreviews for the game Tetris (Game Boy).2018161412108642012345678910Game reviewsCosine with Frequency MatrixCosine with TF.IDF MatrixFig. 6. Progression of the correct suggestions of patterns over the number ofreviews for the game Ninja Gaiden (NES).The analysis of different games with an unequal number ofreviews helped us to note another fact: as can be seen in thethree presented graphs, the number of reviews has stronglyinfluenced the results. More than that, it was directly related tothe number of accepted suggestions of patterns. In most cases,the more game reviews we used in the analysis process, themore patterns were appropriately suggested by the tool.The suggestion lists obtained for the three games are shownin the Tables I, II and III. The accepted patterns in each tableare presented in bold. In these tables, we limited the resultinglist of suggested patterns to the first thirty positions (from atotal of four hundred). Above this, we couldn’t find anyaccepted patterns in the list, although we don’t believe that“thirty” is a rule for all games, which will be subject of studyon future experiments. While the perfect result would behaving all the correct patterns grouped in the beginning of thelist, and not scattered in the first thirty positions, the currentresult is very positive, if we consider that the suggested77

SBC – Proceedings of SBGames 2013patterns were all positioned between the first thirty positions ofa four hundred list. This suggests that, although not perfect, theemployed technique is going towards the desired results.TABLE I.LIST OF SUGGESTED PATTERNS FOR TETRISNº. PatternNº. PatternArt & Design Track – Full Papers8Dice23 High Score Lists9Speed Runs24 Multiplayer Games10 Challenging Gameplay25 Rescue11 Temporary Abilities26 Big Dumb Objects12 Quick Travel27 MovementLine of Sight16 Crossmedia Gameplay13 Privileged Movement28 Diegetically Tangible Game Items2High Score Lists17 Meta Games14 Ghosts29 Backtracking Levels3Puzzle Solving18 Difficulty Levels15 Traverse30 Obstacles4Challenging Gameplay19 Levels5Time Limits20 Zero-Player Games6Casual Gameplay21 Time Limited Game Instances7Geometric Progression22 AI Players8Development Time23 Obstacles9Free Game Element Manipulation24 Speed Runs110 Game Worlds25 Score Tracks11 Non-Player Help26 Back-to-Back Game Sessions12 Single-Player Games27 Gameplay Statistics13 Split-Screen Views28 Ghosts14 Extra-Game Consequences29 Dice15 Tiles30 Diegetically Tangible Game ItemsTABLE II.LIST OF SUGGESTED PATTERNS FOR NINJA GAIDENNº. PatternNº. Pattern1Back-to-Back Game Sessions16 Cutscenes2Levels17 Feigned Die Rolls3Backtracking Levels18 Drop-In Drop-Out4Challenging Gameplay19 Avatars5Crossmedia Gameplay20 Loot6Difficulty Levels21 Trick Taking7Quick Returns22 God Fingers8Death Consequences23 Combat9Lives24 TraverseExaggerated Perception ofInfluence10 Enemies2511 Invisible Walls26 Game Worlds12 Weapons27 Freedom of Choice13 Boss Monsters28 Non-Player Help14 Split-Screen Views29 Damage15 Dynamic Difficulty Adjustment30 Illusion of Open SpaceTABLE III.LIST OF SUGGESTED PATTERNS FOR SUPER MARIO BROS.Nº. PatternNº. Pattern1Levels16 Parallel Lives2Warp Zones17 Enemies3Power-Ups18 Thematic Consistency4Crossmedia Gameplay19 Lives5Boss Monsters20 Difficulty Levels6Friendly Fire21 Diegetic Consistency7Back-to-Back Game Sessions22 AvatarsIV.FINAL THOUGHTS AND FUTURE WORKRecommendation systems have been used successfully inon-line services for consumer’s content suggestion, likeretailers, movie services and news articles. Even outside thisscope, these systems can be applied to measure the similaritybetween groups of textual contents, thus recommending thecontents with higher similarity. This is particularly valuable tothe question that driven this work: would be feasible to usethese techniques to help designers to identify the patterns usedin a game? The results of the experiment described in thispaper shows that it is possible, but improvements in thealgorithm used may lead to a better outcome.The similarity method employed in the software toolspecifically built for the experiment presented throughout thispaper, has showed encouraging results. It helped us to confirmthat both the cosine similarity and the TF.IDF score are reallyuseful methods to analyze the similarity between twodocuments in terms of their subject matter. The cosinesimilarity method alone was able to achieve some results, butthe mixing with the TF.IDF score matrix gave a notably betteroutcome. While it didn’t have achieved the most reliableresults, all the appropriately suggested patterns were grouped,with some scattering, at the first positions of the list, whichbrings a promising initial result.Another important fact that deserves to be noted is that thereviews used in the similarity analysis during the experimentwere mostly written by end users. Although some of themcover important aspects of the game being reviewed, they maylack some more technical language, like the one usuallyemployed by professional game designers or even thespecialized media writers. Clearly, the problem is where to finda good source of game reviews with technical quality.During the evaluation phase of the experiment, when wehad to confront each of the suggested patterns with the gameby reading the patterns definition, we noticed that somepatterns that should have been suggested for the game hadrelations with the suggested ones. We observed this fact whenvisualizing the patterns relations through the relationship mapsgenerated by the software tool. In this sense, it’s possible thatpatterns that have some relations may translate them into thegames that use these patterns, but this was clearly a suspicionand not an observed fact. However, this would be a probablesubject of study and experimentation in the next step of thepresent work.78

SBC – Proceedings of SBGames 2013Art & Design Track – Full Papers[5]As future work, we intent to implement other similarityalgorithms in order to compare the results with the onespresented in this paper. Rajaraman, Leskovec and Ullman [10]as well as Singhal [15] present other techniques that may betriable.Another possibility of a further study is the employment ofcollaborative filtering techniques in an attempt to enhance thesuggestion algorithm. We can create a collaborative tool wheredesigners and users may choose the most probably correctpatterns for a specific set of games and, based on the similarityof these games with the ones under analysis, we may suggestthe patterns. An experiment of this kind may allow us tocombine collaborative filtering with automatic content analysisin order to get more adequate suggestions of game designpatterns.With more solid results, we may have the tool available topublic usage and experimentation, allowing a broadercollecting and processing of usage data, thus improving therecommendation process via the combined techniques ofcontent similarity and collaborative filtering. When we reachthis mark, we will have a fully functional tool that may providean easy way for big studio designers, independent developersand game specialists to analyze and create games using thegame design patterns framework.REFERENCES[1][2][3][4]Neil, K. Game Design Tools: time to evaluate. Proceedings ofDiGRA Nordic 2012.S. Björk, S. Lundgren, and J. Holopainen. Game Design Patterns. InDIGRA Conf., 2003.Kreimeier, B. Game Design Methods: a 2003 survey. feature/2892/game design methods a 2003 survey.php.Costikyan, G. I have no words & I must design. Interactive Fantasy,Eng., n2, 18][19][20]

Among these works, the Game Design Patterns (GDP) approach [2] seeks to extract and document recurring patterns of game design in order to provide a framework for analysis and design of games. The identification of which patterns are used in a game is a long term process that must be entirely done by designers.