Rosetta Stone Version 3 Falls Short Of Manufacturer’s Claims

Transcription

UNIVERSITY OF MARYLAND CENTER FOR ADVANCED STUDY OF LANGUAGERosetta Stone Version 3 Falls Shortof Manufacturer’s ClaimsKatharine Nielson, MASuzanne Freynik, MAPurpose—To determine whether CASL’s review of Rosetta Stone Version 2is still accurate and whether Version 3 may have affected the outcomes of ourprevious empirical study.Executive SummaryConclusions—Some features of Rosetta Stone V3 have changed; however,these changes would not have affected the outcomes of our empirical study.PurposeRelevance—As a stand-alone package, Rosetta Stone is unlikely to be thesolution to the U.S. Government’s language learning needs.CASL review and research in 2007revealed shortcomings with RosettaStone Version 2 (V2) software. Rosetta Stone recently released Version3 (V3), claiming significant improvements over V2 of its product. To determine whether our V2 review is stillaccurate and whether V3 may haveaffected the outcomes of our previousempirical study, we compared the twoversions.Conclusions1. As in V2, the images in V3 are notculturally relevant. In fact, 90 percent of the photos in the Spanish andArabic programs are identical.TTO 2101 E.4.1 CDRL A017 DID DI-MISC 80508B Contract No. H98230-07-D-0175attempts to mimic conversationalpractice through multiple-choicedrills. However, these drills fall shortof the “real-life simulations” that thewebsite claims have been includedin V3.4. Although the V3 lessons are organized in more or less the sameway as the V2 lessons, V3 includesgrammatical concepts that V2 didnot include. The new conceptsinclude features of Arabic or Spanishthat would be especially difficult forsecond-language learners.2. As in V2, information about howa language’s writing system worksis not provided in V3. Learners areshown examples of the language’sscript and are expected to figure outhow the writing system works ontheir own.5. Although some features of the V3software provide more dynamicelements, the program is not trulyinteractive. For example,V3 tellslearners when they should try anexercise again but does not givethem a chance to participate inconversations.3. Unlike V2, V3 includes exercisesthat might help learners with basicgreetings and other real-life conversations. The new Milestone feature6. While some features of RosettaStone V3 have changed, thesechanges would not have affected theoutcomes of our empirical study. 2008 University of Maryland. All rights reserved.Learners in our study might havehad more familiarity with conversational vocabulary, but they wouldnot have had more conversationalpractice. Further, those who dislikedthe lack of explicit writing instruction or the tedium of the drillswould not have had a substantivelydifferent experience with V3.RelevanceRosetta Stone’s claims about theinnovativeness of the product, as wellas the language learning outcomespossible after its use, are generallyoverstated. The software does notprovide the dynamic environmentrequired to practice using thelanguage in context. Rosetta Stonemight be a useful tool to supplementvocabulary acquisition in a morewell-rounded language course, butas a stand-alone package RosettaStone is unlikely to be the solutionto the U.S. Government’s languagelearning needs.1

ROSETTA STONE VERSION 3 FALLS SHORTExecutive ReportPurposeTo help meet its language learninggoals, the U.S. Government has invested millions of dollars in the onlineforeign-language training program Rosetta Stone . In 2007 the University ofMaryland Center for Advanced Studyof Language (CASL) empirically examined the effectiveness of Rosetta StoneVersion 2 (V2) in Arabic, Chinese, andSpanish. Results of that study and aprevious CASL evaluation describingthe program’s potential effectiveness1revealed shortcomings with the software, both with respect to the deliveryof claims made by the company and inthe usefulness of Rosetta Stone for selfstudy by beginning language learners.For example, the software did not provide resources for learners to practiceusing the language in a communicativecontext, and the courses lacked anyconsideration of culture.After CASL published its reviewof V2 and while the empirical studywas underway, Rosetta Stone releasedVersion 3 (V3) in several differentlanguages,2 including Spanish andArabic. The company has claimed that“version 3 significantly improves ourproduct,” and should compensate formany of the problems identified by theCASL review and empirical study.What this study investigatedBecause V3 was released after ourreview and during our empirical studyof V2, our current evaluation3 aimedto determine (1) whether our expertreview of the software is still accurateand (2) whether V3 has any potentialimplications for the results of our empirical study.4In this report we first revisit the issueswith V2 raised by CASL researchers,identify the new V3 claims made by thesoftware manufacturers, and review V3in both Spanish and Arabic. We thenconsider whether using V3 during ourempirical study might have changed theresults. 2008 University of Maryland. All rights reserved.ConclusionsClaim by Rosetta Stone about V3:Version 3 uses Contextual Formation to ask the learner to writenew language in response toconversational prompts.”8 In addition, “Version 3 immerses youimmediately in the new language,providing just the right context toprompt you to speak, pronounce,read and write in the very first lesson! Intuitive, sequential learningbuilds progressively and makesevery lesson count.Our review of Rosetta Stone V3reveals that while some features havebeen updated and improved, manyhave remained the same. In this section, we revisit a number of problemsencountered with Rosetta Stone V2,present the claim made by RosettaStone addressing each problem (ifavailable), and review the content ofthe new course with respect to theproblem identified.1As in V2, the images in V3 are notculturally relevant.Problem with V25: The images usedthroughout the course are the same regardless of the language being taught,so they are not culturally relevant.Claim by Rosetta Stone about V36:Our new global tapestry ofpeople images reflects the diversity of people and cultures fromwhich our language productsderive. We hope you’ll find thesefascinating.Review: While the images have beenupdated for V3, 90 percent of the photos are the same for both Spanish andArabic.7 The photos are not languagespecific and are not culturally relevant.In other words, they do not immerselanguage learners in the target culturebecause they depict images of peoplein all cultures.2As in V2, information about how alanguage’s writing system worksis not provided in V3.Problem with V2: Rosetta Stone V2does not explain how the writing system works. Learners are expected tofigure out the writing system of eachlanguage on their own. The learnerlog comments from the CASL 2007empirical study indicated that the participants were particularly frustratedwith the non-Roman scripts, whichcaused users to either seek outsideresources in order to learn the writing system or abandon the programaltogether.Review: The prompts for the writingdrills in the newest version of RosettaStone are more conversational than theprompts used in V2. That is, learnersare sometimes asked to respond inwriting to conversational exchanges.However, V3 continues to take aninductive approach to teaching thewritten language. It does not explicitlyexplain how the writing system for alanguage works, and it assumes thatlearners will be able to map the written language onto the sounds they arehearing (while they are simultaneouslylearning to map those sounds to themeaning pictured).For example, the very first writinglesson in Lesson 1 shows four differentimages with the noun describing eachimage written in the target language.Students are asked to match the writingto the image. The next screen shows aletter or syllable from one of the previously displayed words written out, andstudents are expected to match it tothe appropriate sound. The softwareprovides no explanation with readingor writing activities, and learners areexpected to learn these skills throughtrial and error.3Unlike V2, V3 includes exercisesthat might help learners withbasic greetings and other real-lifeconversation.Problem with V2: At the end of theentire course, learners have not acquired fundamental material, such asbasic greetings or how to introducethemselves. In addition, there is nodialogue or interaction, but instead2

ROSETTA STONE VERSION 3 FALLS SHORTshort, unrelated sentences that have noreal-life counterpart. Finally, the singleparticipant in our empirical study whocompleted the 200-hour V2 course hadthe following comment: “While Rosetta Stone does teach a lot of words, theyare not always the words you need tohave an actual conversation.”Claim by Rosetta Stone about V3:New Contextual Formation feature uses real-world simulations to give you the benefits youneed to succeed. New Milestonefeature lets you try out your newlanguage knowledge in real-lifesimulations.Review: Rosetta Stone V3 appears tobe more relevant in terms of communicative language use. Nearly all ofthe grammar drills in V2 encouragedlearners to use the third person. V3, onthe other hand, includes a variety ofgrammar drills that encourage learnersto use the first person, which is morelikely to be needed in conversation.In addition, while V2 overlookedvery basic items like greetings, V3includes these from the start. However, the greetings are not necessarilypresented in a communicative context. A number of drills begin with agreeting, go on to address unrelatedgrammar and vocabulary, and then endwith an equally unrelated farewell.For example, the Core lesson in Unit1 begins with two images: a man anda woman. The word hello is writtenabove each picture and is heard aseach image is highlighted. Learners donothing on this screen, but they do seethe words and hear them. Then the actual lesson begins and learners matchwords (which have nothing to do withgreetings) to images and completesimple grammar drills. The last screenof the Core lesson is of two differentpeople leaving. The word goodbye iswritten above each picture and learnershear it as the images are highlighted.The hello and goodbye screens areunrelated to the material presented inthe rest of the lesson. While presentinggreetings and salutations in this fash- 2008 University of Maryland. All rights reserved.ion does expose learners to the words,it does not show them how they areused in an actual conversation. Rather,the words simply mark the beginningand the end of the lesson.Some of the picture-concept mapping exercises present images thatare related to one another in the formof a conversational exchange (e.g.,“My house is big.” “My house isbigger than your house.”). Again, thisexchange is more communicative innature than the examples used in theexercises of V2, but it hardly providesthe “immersion” environment claimedby the manufacturers.The most significant gesture V3makes toward communicative language use is the Milestone feature thatends each of the four units, where thelearner participates in a dialogue withpictured interlocutors by choosing thewritten utterance that fits each scenario. The interlocutors are presented inthe style of a cartoon strip, with photosscrolling across the scene and speechbubbles indicating who is speaking towhom. The dialogue presented in thisfeature is identical in every language,and while conversation-like, the exchanges are stilted and inauthentic, asin the following example:Hello.Hello.Are all the flowers the same color?No, the flowers are not all the samecolor. What color flowers do youwant?I would like red flowers.How many would you like?I would like 35. Thank you.You’re welcome.Good morning.Good morning.Do you sell eggs?Yes, I sell eggs. How many eggs doyou need?I need 48 eggs.Thank you.You’re welcome.The above dialogue is unrealisticand, more importantly, does not givelearners enough time for this type ofcommunicative drill. Each of the fourmilestones is estimated to take 10minutes. Therefore, of the 200 or sohours necessary for this course, a totalof 40 minutes is designed for practicein communication. These drills arenot the “real-life simulations” that thewebsite says have been included inV3.4Although the V3 lessons areorganized in more or less thesame way as the V2 lessons, V3includes grammatical conceptsthat V2 did not include.Problem with V2: The grammar les-sons appear to be sequenced by thenotion of moving from simple to complex grammar. This simple to complex sequencing is known as a covertgrammatical syllabus. It is commonlyfound in mass-market language materials, despite publishers’ claims to beadopting a more innovative approach.In addition, V2 provides no information on cultural-grammatical issuessuch as the difference between formaland familiar verb forms in Spanish orthe difference between dual and pluralnoun marking in Arabic.Claim by Rosetta Stone about V3:Improved intuitive, sequentiallearning makes every lessoncount and build progressively.Review: Lessons in V3 do not appearto be organized differently than in V2.The organization of the materials isstill based on the shift from simple tomore complex grammar, though thefocus in V3 is on different grammatical concepts. V3 does incorporatesome of the important grammaticalconcepts that the second versionoverlooked.Specifically, the drills in V3 providemore practice on some points of Arabic or Spanish grammar that mightbe difficult for speakers of otherlanguages. For example, V3 Arabicincludes additional drills that illustratethe difference between dual and pluralmarking on nouns, case-markingon nouns, and the conjugations forverbs according to gender as well as3

ROSETTA STONE VERSION 3 FALLS SHORTnumber, all of which were introducedonly sporadically in V2. V3 Spanish illustrates the difference betweenformal and informal second personand no longer focuses extensively onthe infrequently used present progressive tense. In both Spanish and Arabic,V3 introduces first, second, and thirdperson agreement all within the firstlesson, which is an improvement overV2, where first person agreement isnot introduced until Lesson 5 and second person agreement is ignored untilthe last lesson.In order to accommodate this morediverse array of grammatical forms,V3 dispenses with some of the unusualvocabulary that permeated V2; V3does not have as many drills aboutducks and horses and people crouching under tables. Rather, V3 introduces a sparser (and arguably moreuseful) core vocabulary and then drillsthe same items repeatedly with grammatical manipulations. For instance, aset of drills in V2 might depict peoplejumping off of many different things,while a comparable set of drills inV3 depicts different combinationsof people writing. The V2 manipulation illustrates the names of all thethings people can jump off of, whereasthe V3 manipulation illustrates howthe verb write agrees with differentsubjects.Unfortunately, in addition to paringdown vocabulary items, V3 also doesaway with some necessary grammatical concepts. While V2 had lamentably few exercises involving the pastand future tenses, V3 has even fewer,adhering almost exclusively to thepresent tense. In addition, the attemptsto include language-specific grammatical information are inconsistent.While it is standard conversationalpractice in both Spanish and Arabicto drop pronouns before conjugatedverbs, only the Spanish version showsexamples of this throughout the drills.The Arabic version of the softwareremains a translation of the Englishversion and retains pronouns in everyexample, which does not reflect howthe language is actually used. 2008 University of Maryland. All rights reserved.5Although some features of the V3software provide more dynamicelements, the program is not trulyinteractive.Problem with V2: The Rosetta StoneDynamic Immersion Method9 environment is quite impoverished incomparison with the rich immersionenvironments of naturalistic language acquisition. Rosetta Stone V2is restricted to a limited vocabularysurrounding the set of images. Theonly dynamic and interactive elementsare the distinctive tones and symbolsintended to indicate that a selectionamong four choices is correct or incorrect and the comparison of a visualrepresentation of the learner’s pronunciation to a native speaker’s graph.Claim by Rosetta Stone about V3:Rosetta Stone does not claim to havechanged their Dynamic ImmersionMethod environment; however, thecompany describes a new feature thatattempts to provide another form ofcustomization.New Adaptive Recall Language feature tracks progressto reinforce your strengths andrevisit needs. New proprietaryspeech recognition technologygets you speaking from the startand new speech analysis toolsperfect your pronunciation.Review: In terms of the dynamic andinteractive components, the coursecontinues to provide feedback via anegative tone. In other words, learners are not given any hints on how toselect the correct answer, even afterchoosing the incorrect answer repeatedly. Rather than receive the correctanswer, learners receive feedbackabout when they should attempt thelesson again. The V3 Adaptive Recallfeature records user performance,determines how many items the learneranswered correctly, and then makessuggestions for future study. When accessing the program on the suggesteddate, the learner is prompted to complete the old lesson before continuingwith new work.Similarly, although the technologybehind the speech tools appears to beimproved, learners still do not receiveany feedback other than the chance tocompare the graph of their utterancesto that of a native speaker. Unlikein V2, learners have more time topractice speech in V3. The lessons thatrely on the speech recognition softwareallow learners to link to a separatescreen where they can listen and recordeach statement multiple times beforecontinuing to the next question. Theyare able to spend more time workingon pronunciation, but they have nochance to actually interact in the targetlanguage.6While some features of RosettaStone V3 have changed, thesechanges would not have affectedthe outcomes of our empiricalstudy.The updated “global tapestry of images,” while more visually appealing,does not change the format of theprogram and would have had no effecton our empirical study. And while theaddition of the Contextual Formationand Milestone features is a gesturetoward more communicative language,the features do not provide practice forauthentic conversations with nativespeakers. Therefore, while learners inour study might have had more familiarity with conversational vocabulary,they would not have had any morepractice with completing conversations. It doubtful that the singleparticipant of the study who completedthe Oral Proficiency Interview wouldhave been better equipped to conversein Arabic.The inductive method of instruction has remained the same, so thosestudents who disliked learning throughguessing would not have had a different experience with the new product.In addition, the technology required torun the program is the same, so using anew version would probably not havechanged any of the technical issues,such as learners being unable to usethe software from secure facilities orwhile using dial-up Internet access.4

ROSETTA STONE VERSION 3 FALLS SHORTWhile the overall look and feel ofthe course is more modern, it remainsvery much the same program. Learnersstill do little more than match picturesto words and sounds. The language issomewhat more communicative, butthe program does not provide the opportunity for communicative practice.The “simulations” involve selectingwords from multiple-choice dropdown menus, so they do not preparelearners to actually speak in the targetlanguage. Finally, the one-size-fits-alllanguages approach has not changed,and while there have been someattempts to adapt the courses to fit language-specific needs, these have beenhaphazard. Since the structure of thecourse remains ess

ined the effectiveness of Rosetta Stone Version 2 (V2) in Arabic, Chinese, and Spanish. Results of that study and a previous CASL evaluation describing the program’s potential effectiveness1 revealed shortcomings with the soft-ware, both with respect to the delivery of claims made by the company and in t