WIXLIB

CMU-ITC-86-048Paper presentedat the AmericanEducationalResearchAssociationannual meeting, San Francisco, April, 1986. This work was supportedby a grant from the CMU Program in Technologyand Society, Divisionof Social Sciences, Carnegie-MellonUniversity,and by the InformationTechnologyCenter at CMU.TechnologyDo Computersin the Classroom:Change the Way We Write?ChristinaEnglishDepartmentHaasand tyEnthusiasmfor using computers to write and to teach writing has beenstrong among those of us who have a special interest in both computersand writing.We believe--usuallybased on our own experiencesaswriters--thatthe computerlessensthe tediousnessOf writing,thatcomputers free us from the mundane aspects of writing, that computersallow writers to focus on content and meaning,and that computersencouragerevision.We assume that the computer'sstrengths--speedand vast memory capacity--somehowhelp the writer, and that the resultis more and better writing.Unfortunately,thereis littleempiricalevidenceto supportourassumptions.In fact, what little there is seems to contradictthem: forinstance, John Gould in his important1980 study, found that expertwriters using text editors required 50% more time to compose on texteditors than on hard copy.But this extra time did not lead to "better"writing; there was little differencein quality between letters produced ontext-editorsand those produced on paper. Although computersmay aidwriters in some ways, these effects did not show up in Gould's study aslonger or better writing.Of course,since 1980 technologicaladvanceshave improvedtheusability,power,and attractivenessof text-editorsand computerdisplays.Possibly these design advanceshave made writingon thecomputermore efficientandGould'stroublingresultsare simply

-2-outdated.The purpose of this study was two-fold:first, it attempts to partiallyreplicateGould'sexperiment,usingmore advancedmachinesandeditors.Second,it tests severalhypothesesabout writingwith thecomputer.Specifically,that computershelp writers produce more text,produce text faster, and that computershelp writers write better.These questionsare importantfor educatorsto ask--we need to knowhow different hardware and software features affect writers and writing.Only then can we make informed choices about when and how to utilizecomputersin the classroom--andwhat kind of computers to use.Subjects:The inistratorsand system designersfrom Carnegie-MellonUniversity.The 11 men and four women ranged in ages from 23 to 42. All of theparticipantshad had at least several years of computer experience,andall worked daily or almost daily on the machines on which they weretested.Tasks: Subjects wrote persuasiveletters in four conditions.Two wereto a known audienceand two were to an unknownaudience.Theparticipantsproducedthe letters in two sessions--twoletters the recounterbalancedacross conditions.Conditions:The participantswrote letters in a hard copy condition andin three computer conditions.In two of the computer conditionswriterswrote using "Andrew,"an advancedcomputingsystem and relatedsoftwarebeing developedat the InformationTechnologyCenter as ajoint ventureof Carnegie-MellonUniversityand IBM(Morris,et al,1986).The advancedworkstationhas a large, bit-mapped,high-resolutiondisplay, and a mouse.The text-editorwas developedat the ITC andbased on EMACS.Placement of the cursor for editing the text is doneby pointing the mouse and clicking the mouse button.Writers movethrough the documentby pointing the mouse and clicking in a scroll barregion.For cutting text, writersused either a delete key or a menuoption;pasting was also done viaamenu.In one condition the windowused for writing was 9 1/2 inches wide and 10 1/2 inches long.Theother advancedworkstationconditiondifferedonly in the size of thescreen:it was 9 1/2 inches wide and 6 1/2 inches long, the size of astandard pc or terminal display.In the othercomputercondition,writersused an IBM-PCand the text-

-3-editor MINCE, also based on EMACS. The pc had a 9 1/2 inch by 6 1/2inch screen and a CRTdisplay.Moving the cursor, moving through thedocument,and editingare done with control-keys.In the pen-andpaper conditionwritersused 8 1/2 by 11 inch ruled paper and theirchoice of a felt-tip or ball-pointpen. When writingin the computerconditions,writers were not allowed to use pen and paper notes, not toget a printout of their texts.The salient differencestable below.betweenthe computersworkstationare summarizedin thepcMachinespeed16 kilo-baud9600 baudresolutionbit-mappedCRTblack on gray green on blackscreen size9 1/2 by 10 1/2 9 1/2 by 6 1/250 lines by24 lines by90 characters 88 charactersEditor (Note: all computer conditions used display editors.)text advancementmethodscroll bar and control-keys mousecursor movementmouse/pointer control-keysediting--deleting, cut/pstmouse/menu;delete keycontrol-keysAnalysesI looked at three quantitativemeasures:time to compose the letter,total words produced,and words producedper minute.Because therewas no significantdifference in any measure between the workstationlarge and small conditions,these data were collapsedinto a singleworkstationconditionand scores were normalizedfor topic and ordereffect.Time to compose was longest in the advancedworkstationcondition,and shortestin the hard copy condition.These differenceswere

-4-significantat the .02 level by analysisof variance.Neuman-Keulsanalysisshows this varianceaccountedfor by a significant( .05)differencebetweenthe workstationand paper conditions.TimeCond.Cond.Cond.1, workstation2, pc3, pen-and-paper17.40 minutes15.12 minutes13.43 minutesThe secondmeasure--numberof th the highest mean number of words in theadvancedworkstationcondition,and the smallestmean numberofwordsin the hard copy condition.These differencesare significantatthe .01level by analysisof variance.Neuman-Keulsshows thevarianceto be accountedfor by a significantdifferencebetweenconditions1 and 3. Condition2 was not significantlydifferentfromeither of the other conditions.Total WordsCond.Cond.Cond.1, workstation2, pc3, pen-and-paperThe words-per-minuteconditions--workstation20.94 wpm.352.79 words291.62 words264.05 wordsmeasure shows no significantdifferencebetween20.17 wpm;pc, 21.00 wpm; pen and paper,So in terms of quantity--bothtime on task and total words--the advancedcomputerseems to outperformpen and paper althoughthe pc, oftstandardcomputerconditiondoes not. But of course quantity is onlyhalf the story.So, I also collectedtwo measuresof writingquality:Independentreaders(English teachers with at least five years experienceteachingwriting)rated each subject'sletters using a forced quartile split.Theletters received scores for content--qualityof ideas--and for mechanics-sentence- and word level-correctness.Agreementbetween two raterswas 80%. When raters scores were more than one quartile different,athird reader read and rated the letters. Again, because there were nodiscernabledifferencebetween the two workstationconditions,the datawas collapsedinto a single workstationcondition.With possible scoresranging from 2 (low) to 8 (high), mean scores for each condition were:

-5ContentCond. 1, workstationCond. 2, pcCond. 3, pen-and-paper5.964.05.13MechanicsCond. 1, workstationCond. 2, pcCond. 3, pen-and-paper5.234.265.67The content measure differences are significant at the .02 level byanalysis of variance.Neuman Keuls analysis shows that the textsproduced in the pc condition were significantly poorer ( .05)thanthose produced in the advanced workstation condition. The differencesbetween the pc condition and the hard copy condition were just short ofsignificant.Although the differences in mechanics scores are notsignificant, the mean scores for the PC Condition are again the lowest.In addition, the difference in total quality scores was significant at the.02 level by analysis of variance.Mean total scores--sum of contentand mechanics scores--were:TotalCond. 1, workstationCond. 2, pcCond. 3, pen-and-paper.11.208.2610.89Neuman Keuls analysis shows again, that the pc condition wassignificantly different ( .05) from the advanced workstation conditionand from the hard copy condition.To summarize, writers on the whole produced significantly more textwith the advanced workstation than they did with pen and paper, ANDthey produced signifcantly better texts with pen and paper and theadvanced workstationthan they did with a standardcomputer.Unfortunately,I can't say much yet about WHY these differencesoccurred, beyond noting that screen size doesn't seem to account forthe difference.The improvement in performance of subjects in thisstudy over Gould's probably has to do with the differences between lineeditors and screen editors. Beyond that, these results may be due tomachine speed, resolution, method of moving through the text, methodof editing--ie, mouse vs. ctrl keys.

-6ConclusionLet's return to the questions with which this paper began: we noted twodifferent sets of claims--one that computers will help people write moreand better, and the other--supportedby Gould's early work--thatmachines do not lead to more or better writing.These seeminglycontradictoryclaims can be reconciled if we elaborate them: somecomputers can help people write more and better, but some machinesmay also make them write more poorly.Advances in machine design are encouraging--the poorer performanceGould found with computers seems to have turned, with more advancedmachines, to a slight advantage for computers.However, thisencouraging finding should be qualified in two important ways:thepositive results come from a very advanced--andneedless to say,expensive--machine,the kind of computer that most of us don't haveaccess to. They are not even widely available at CMU, or "ComputerU," as its sometimes called.We cannot assume--and indeed theseresults prohibit us from doing so--that the kind of computer typicallyused in education today will aid writers in the way that this workstationdid in this study. (Lest I sound to negative, I acknowledge that manyparts of the writing process and many kinds of writing may be greatlyaided by the computer. In addition, many excellent word-processingprograms and CAI programs may help writers and help teachers teachwriting.) What I am advocating is restraint in assuming that because ofthe computers strengths, it be adopted whole and with little criticalevaluation by teachers of writing and writers.Second, the writers in this study are experienced, expert writers andcomputer users. All wrote regularly on the job; most had been usingcomputers for at least 10 years; many had advanced degrees; and allwere significantly older than most of our students.We should not betoo hasty and extrapolate these findings to students--usuallynovicecomputer users, and often poor writers as well. Overall, the conclusionof these studies advocates CARE--there remains much to be done todetermine when, where, with whom, and how computers can best aidwriting.Our own research will pursueexaminationof the processes ofexperienced computer users and writers, to determine first, if there areprocess differences which led to the quality differences demonstratedhere. Second, these results, together with earlier reports of computerwriters' conscious adaptation of their writing process into a "system"for computer writing, (see Haas and Hayes, 1986), lead to thehypothesis that these experienced computer writers have "learned" towrite well on the machine. We plan to continue to explore just howcomputer writers utilize the strengths and adapt to the constraints of

-7the computer when writing.