WIXLIB

3. Visionin the real world, becoming fluent in a new languagein a classroom setting is difficult. In this 3D game,players navigate a virtual environment that simulatesbeing immersed in a real target language environment.Players collaborate through language quests thatrequire them to find words in the environment neededto accomplish objectives. Both contextual information onuse of language and collaboration have been shown todramatically improve learning outcomes.Researchers from different disciplines havedemonstrated the benefits of personalized educationtools in specific courses. We can build on this momentumand bring together researchers with different expertisein large-scale projects aimed at transformative changesin education technology. We envision that progressin personalized education technology can benefit thesociety in following ways:The role visualization can play in learning is evident fromthe success of Python Tutor. It uses visual interactionsto help people overcome a fundamental barrier inlearning programming, namely, understanding whathappens as the program executes different lines of code,and is being used by millions of people [Guo13] (see alsowww.pythontutor.com). Our goal is to train students in advanced topicswithout having them sacrifice quality of life. This canbe achieved by improving effectiveness of educationthrough technology by maximizing learning at realistictime investments by teachers as well as students. Oneconcrete measure of success could be that a futuresophomore computer science student will know what acurrent senior student knows.Learning Science Current techniques for assessment are focusedon short-term learning. We envision a future ofpersonalized learning apps that stimulate andincentivize people to be lifelong learners with a focuson long-term learning and knowledge retrieval ondemand.Cognitive science aims to understand some of theprinciples and processes involved in learning. The naturalquestion then is, how can we use computational tools tosupport these processes? Recent years have witnessedincreasing research in constructing mental models ofstudents based on their interaction with educationaltools, data mining past history of interactions tosuggest next steps, experimental analysis of howlearning outcomes are impacted by interventions, andunderstanding of the role of social factors in learning. A key challenge to personalized education is to foster arobust pipeline of a diverse group of students to STEMand related disciplines. Personalization can meet thedemands of heterogeneous backgrounds and differentlearning styles, and ensure engagement and retention.As an example of how technology can help learningoutcomes, consider the problem of detecting whetheror not a student is attentive while either sitting in alecture, reading a book, or interacting with an onlinetool. Sensor technology and smart cameras can nowdetect wandering minds with high fidelity [KDM15]. Suchtechnologies are leading to interactive books with a hugepotential of impacting education.4. Research RoadmapTo realize our vision of how computer-aided personalizededucation technology can impact society, we need tomake progress on the following research goals. We firstlist some long-term projects that will require sustainedcollaboration among computational and learningscientists.Learning science tells us that students learn best whenthey have an opportunity to collaborate, discuss, andform communities. This has been already put in practicein supportive collaborative learning environments suchchat rooms in MOOCs and similar platforms [AD 14,CL 15]. Current personal tutors are invariably focused onspecific concepts in individual courses.3

COMPUTER-AIDED PERSONALIZED EDUCATIONshould be based on some modeling of the state ofthe student, such as learning style, past knowledge,and understanding of certain concepts. While suchmodels have been studied in cognitive science, theirincorporation in computational feedback tools is apressing and challenging problem. A ten-year goal is to develop an expert teacher percomputer science student. Such a personalizedassistant can track an individual student’s progressthroughout the curriculum by actively providingfeedback and help. A comprehensive theory of learning is an achievableten-year target. Such a theory can in turn impactpersonalized teachers by constructing a mentalmodel of the student, adapting to how a student isresponding to interventions, and accounting for socialfactors in learning such as collaboration. Beyond STEM courses: Current tools for problemgeneration, automatic grading, and feedback generationfocus on mathematical problems in STEM subjects.Such problems are amenable to computationalformalization (a notable exception is grading of essaysfor grammar and style). Developing techniques that aremore broadly applicable will require novel integrationof many approaches, and offers a promising researchopportunity. A key to progress will be the availability of sharedlarge-scale data repositories and experimental testbedsto evaluate research ideas. Building such open-sourceand shared infrastructure is itself a challenging, longterm, and worthy research goal. Multi-modal interfaces: Rapid advances in sensortechnology are leading to new ways in which ahuman can interact with a computer, such as bytext, by speech, and by touch. Such natural modesof interaction are particularly relevant for studentengagement. At the same time, the specific goalsof such tools can help alleviate challenges incomputationally difficult tasks like translating naturallanguage to a formal language. Thus, developingeffective multi-modal interfaces for personalizededucation tools is an opportunity for creative researchat the intersection of many disciplines.To conclude this report, we list promising topics thatcan be explored by research teams. Progress on thesetopics in the next few years can provide the buildingblocks necessary to achieve the long-term potential ofpersonalized education. Scalability: Many educational tasks such as feedbackgeneration can be cast as search problems in a largespace of candidate artifacts. On one hand, there hasbeen significant research and engineering investmentsin generic search technologies such as SAT and SMTsolvers. While these techniques work fairly well forcertain domains and small problem instances, theydo not constitute a universal scalable solution. Onthe other hand, domain-specific search techniquesthat leverage knowledge of the underlying domainscale well in the target domain, but require significanttime, research expertise, and engineering effort. Animportant future research direction is to enable easyconstruction of search techniques that scale well tovarious domains by integrating generic with domainspecific components. Collaborative learning environments: There is plenty ofempirical evidence that students learn by collaboratingwith one another. Tools such as chat rooms and peergrading have been incorporated in current onlinelearning environments. However, to fully achievethe promise of collaboration, we need research forbetter understanding of principles of both the role ofcollaboration in learning and how to add collaborationto learning. Predictive models: One important goal for apersonalized education tool is to help strugglingstudents meet their educational goals. Predictivemodels based on modern data analytics can detectpotential problems in advance. For example, grades inquizzes early in a course can reliably predict the finalcourse grade. An interesting research question is, Mental models for feedback: Tools today are designedto give feedback based only on the student submission,but not so much based on the mental model of astudent. Various forms of feedback are possible,and the one that should be presented to a student4

how to design intervention strategies based onsuch predications, integrate them in learningenvironments, and ensure tangible improvements inlearning outcomes?right from the beginning. Finding the balance betweeninformation access and privacy and enforcementmechanisms are challenging technical problems, andresearch is needed to find solutions appropriate for thedomain of education. Adaptive syllabi and curricula: In a typical course,whether in a classroom or online, the content of thecourse is fixed in advance. Adaptive learning technologyoffers exciting opportunities to make the contentdynamic. At the micro level, there is already somesuccess in using computational tools for problemgeneration for specific concepts to suggest the nextproblem based on the student’s past interactions.New research though is needed for adaptive contentgeneration to dynamically develop the sequence ofconcepts resulting in a course that meets the desiredlearning outcomes and the sequence of coursesleading to a curriculum that meets the desired breadthand depth requirements.If we follow these suggestions as a community, wewill make significant progress towards not only bettereducating STEM students with diverse backgroundsbut also great strides in creating educational tools thatwill impact all students as we realize the benefits ofpersonalized education.References[AD 14] D. Adamson, G. Dyke, H.J. Jang, and C.P. Rose.Towards an agile approach to adapting dynamiccollaboration support to student needs. Intl. Journal of AIin Education 24(1): 91–121, 2014. Virtual Labs: A central component of engineeringeducation is learning by building artifacts in a lab. Thisraises the question: can we create online labs with alearning experience close to the physical lab. Virtualsimulation environments integrated with learningtechnology can offer a solution, and this leads to anumber of research questions.[BBJV14] M. Brooks, S. Basu, C. Jacobs, and L.Vanderwende. Divide and correct: using clusters to gradeshort answers at scale. ACM Conf. on Learning at Scale,pp. 89–98, 2014.[BBV12] L. Becker, S. Basu, and L. Vanderwende. Mind thegap: Learning to choose gaps for question generation.NAACL Human Language Technologies, pp. 742–751, 2012. Long-term learning outcomes: Traditionally testing isused to assess how much the student has learnedduring a course. A more meaningful assessment wouldbe to measure how much knowledge a student retainsover a long period and whether this knowledge canbe retrieved as needed to solve problems. Cognitivescience helps us understand how humans store andretrieve knowledge. A fruitful research direction is tointegrate this understanding in personalized educationtools to improve long-term learning outcomes.[CA 16] G. Culbertson, E. Andersen, W. White, D. Zhang, andM. Jung. Crystallize: An immersive, collaborative game forsecond language learning. 19th ACM Conf. on ComputerSupported Cooperative Work and Social Computing, 2016.[CL 15] D. Coetzee, S. Lim, A. Fox, B. Hartmann, andM.A. Hearst. Structuring Interactions for Large-ScaleSynchronous Peer Learning, Proceedings of the 18th ACMConference on Computer Supported Cooperative Work &Social Computing, pp. 1139–1152, 2015. Privacy: Tools for personalized education base theirdecisions on mining data from students’ solutionsand students’ history of interactions. These decisionscannot be made without access to sensitiveinformation, but naturally lead to concerns aboutpreserving privacy of individual students. Sincepersonalized education is a nascent technology, itwould be prudent to bake privacy concerns into tools[DK 15] L. D’Antoni, D. Kini, R. Alur, S. Gulwani, M.Viswanathan, and B. Hartmann. How can automaticfeedback help students construct automata? ACM Trans.Computer-Human Interaction, 22(2): 9:1–9:24, 2015.5

COMPUTER-AIDED PERSONALIZED EDUCATION[Gul14] S. Gulwani. Example-based learning in computeraided STEM education. Communications of the ACM 57(8):70–80, 2014.Participants[Guo13] P. Guo. Online Python Tutor: Embeddable webbased program visualization for CS education. ACMTechnical Symp. on Computer Science Education, 2013. Nina Amla, National Science Foundation Rajeev Alur, University of Pennsylvania Erik Andersen, Cornell University Anindya Banerjee, National Science Foundation[JDJS14] G. Juniwal, A. Donze, J.C. Jensen and S.A. Seshia.CPSGrader: Synthesizing temporal logic testers for autograding an embedded systems laboratory. Proc. 14th ACMConf. on Embedded Software, pp. 1–10, 2014. Richard Baraniuk, Rice University Lida Beninson, National Science Foundation Gautam Biswas, Vanderbilt University[KDM15] K. Kopp, S.K. D’Mello, and C. Mills. Influencingthe occurrence of mind wandering while reading.Consciousness and Cognition 34(1): 52–62, 2015. Rastislav Bodik, University of Washington Emma Brunskill, Carnegie Mellon University[LVWB15] A.S. Lan, D. Vats, A.E. Waters, and R.G. Baraniuk.Mathematical language processing: Automatic gradingand feedback for open response mathematical questions.ACM Conf. on Learning at Scale, 2015. Andy Butler, University of Texas, Austin Isaac Chuang, Massachussets Institute of Technology Sandra Corbett, Computing Research Association[LWSB14] A.S. Lan, A.E. Waters, C. Studer, and R.G.Baraniuk. Sparse factor analysis for learning and content Loris D'Antoni, University of Wisconsin, Madisonanalytics. Journal of Machine Learning Research 15:1959–2008, 2014. Lucas de Alfaro, University of California, Santa Cruz Sidney D’Mello, Notre Dame University[MXAS16] Y. Meier, J. Xu, O. Atan, and M. van der Schaar.Predicting Grades. IEEE Transactions on SignalProcessing 64(4): 959–972, 2016. Khari Douglas, Computing Community Consortium Ann Drobnis, Computing Community Consortium[NPHG14] A. Nguyen, C. Piech, J. Huang, and L.J. Guibas.Codewebs: Scalable homework search for massive openonline programming courses. Proc. Intl. World Wide WebConference, pp. 491–502, 2014. Barbara Ericson, Georgia Institute of Technology Kathi Fisler, Worcester Polytechnic Institute Michael Gleicher, University of Wisconsin, Madison[RV05] C.P. Rose and K. VanLehn. An evaluation of a hybridlanguage understanding approach for robust selection of Phillip Grimaldi, Rice Universitytutoring goals. Intl. Journal of AI in Education 15(4), 2005. Jonathan Grudin, Microsoft Research[SGS13] R. Singh, S. Gulwani, and A. Solar-Lezama.Automated feedback generation for introductoryprogramming assignments. ACM SIGPLAN Conf.Programming Language Design and Implementation, pp.15–26, 2013. Sumit Gulwani, Microsoft Philip Guo, University of Rochester Greg Hager, Johns Hopkins, CCC D. Fox Harrell, Massachussets Institute of Technology[TBS15] C.Tekin, J. Braun, and M. van der Schaar, eTutor:Online Learning for Personalized Education. IEEE Intl.Conf. on Acoustics, Speech and Signal Processing, 2015.6

Peter Harsha, Computing Research Association Kevin Wilson, Knewton Marti Hearst, University of California, Berkeley Helen Wright, Computing Community Consortium Sabrina Jacob, Computing Research Association Jerry Zhu, University of Wisconsin, Madison Mike Jones, Indiana University, Bloomington Ben Zorn, Microsoft Research Yasmin Kafai, University of Pennsylvania Jeffrey Karpicke, Purdue University Caitlin Kelleher, Washington University in St. Louis Anthony Kelly, National Science Foundation Ken Koedinger, Carnegie Mellon University Andrew Lan, Rice University Mark Liberman, University of Pennsylvania Mimi McClure, CSR/CNS/NSF Danielle McNamara, Arizona State University John Mitchell, Stanford University Mike Mozer, University of Colorado, Boulder Zoran Popovi, University of Washington Debra Richardson, University of California, Irvine/CCC Carolyn Rose, Carnegie Mellon University Beth Russell, AAAS Majd Sakr, Carnegie Mellon University Mihaela van der Schaar, University of California, LosAngeles Sanjit Seshia, University of California, Berkeley Beth Simon, Coursera Rishabh Singh, Microsoft Armando Solar-Lezama, Massachussets Institute ofTechnology Candace Thille, Stanford University7

1828 L Street, NW, Suite 800Washington, DC 20036P: 202 234 2111 F: 202 667 1066www.cra.org cccinfo@cra.org

courses has grown three-fold in the past decade. Recently, massive open online courses (MOOCs) have been promoted as a way to ease this strain. This at best provides access to education. The bigger challenge though is coping with heterogeneous backgrounds of different students, retention, providing feedback, and assessment.