Transcription
CROWD CONTROL: ORGANIZING THE CROWD AT YELPbyDavid Andrew AskayA dissertation submitted to the faculty ofThe University of North Carolina at Charlottein partial fulfillment of the requirementsfor the degree of Doctor of Philosophy inOrganizational ScienceCharlotte2013Approved by:Dr. Loril GossettDr. Anita BlanchardDr. Cliff ScottDr. Min Jiang
ii 2013David Andrew AskayALL RIGHTS RESERVED
iiiABSTRACTDAVID ANDREW ASKAY. Crowd control: organizing the crowd at yelp.(Under the direction of DR. LORIL GOSSETT)This dissertation investigates how businesses are able to align the collectiveactions of a disconnected crowd with the strategic goals of the organization. I examinedthis questions within the context of the business review website Yelp through aquantitative analysis of nearly 60,000 business reviews, 17 in-depth qualitative interviewswith reviewers, and a two-year ethnography. Interpreting the results of this data withinthe framework of the collective action space (Bimber, Flanagin, & Stohl, 2012) indicatesthat Yelp is able to manage the contributions of a relatively small subset of reviewersthrough the Yelp Elite Squad. Rather than simply motivating more reviews, the EliteSquad encouraged reviewers to interact more personally with other reviewers and acceptincreased institutional engagement with Yelp. In encouraging members of the crowd toproduce online reviews within this context, Yelp was able to use organizational culture asa control strategy for encouraging Elite reviewers to adopt a pre-mediated reviewingapproach to their reviews. This increased the frequency of moderate reviews anddecreased the frequency of extreme reviews. This behavior ultimately furthers theorganizational goals of Yelp, as moderate reviews are considered to be more helpful forreviews of businesses. Finally, implications for crowdsourcing, big data analysis, andtheory are discussed.
ivACKNOWLEDGEMENTSThis dissertation is culmination of five years in the interdisciplinaryOrganizational Science program at the University of North Carolina at Charlotte. I amgrateful to faculty across the departments of Communication Studies, Psychology, andSociology and the Belk College of Business for patiently helping me to understandorganizations and organizing from the perspective of these disciplines. I want to alsothank the students of the Organizational Science program, who have frequently acted assounding boards for my ideas and as sources of inspiration. Also, this study would nothave been possible without the interview participants, who graciously provided me withtheir time and trust so that I was able to pursue this research project.Without the guidance and support for my committee members, this dissertationwould not have been possible. Dr. Loril Gossett has tirelessly served as my advisor sincethe first year of my graduate education, introduced me to the field of communicationstudies, and chaired both my master’s thesis and this dissertation. Over five years, she hasconsistently challenged me to critically think about my research and somehow alwaysknew precisely the right questions to ask to get me there. I am thankful for her years ofenthusiastic mentoring, without which I would not have been here. Dr. Anita Blanchardhas also provided me with invaluable guidance in this dissertation and in my graduatestudies, developing my writing abilities and challenging me to situate my findings inother domains. Dr. Min Jiang opened up another world of references that I would nothave otherwise been exposed to, introducing me even to the theoretical framework usedin this dissertation. Dr. Cliff Scott developed this dissertation by asking insightful
vquestions that challenged me to think about the larger theoretical issues of participationand membership.Also helping me to formulate this dissertation were the immeasurablecontributions from students and scholars from several doctoral developmental initiatives.I want to acknowledge the students, scholars, and organizers of the Internet Research12.0 Doctoral Colloquium, the 2012 Academy of Management OCIS Division DoctorialConsortium, the 2012 Consortium for the Science of Sociotechnical Systems SummerResearch Institute, and the 2011 National Communication Association Doctoral HonorsSeminar. These individuals provided insight and direction during the development of thisdissertation.Finally, I want to acknowledge my friends and family who have supported methroughout graduate school. Despite the distance from moving so far from home inCalifornia, my family provided me with love, money, and understanding that made thisdissertation possible. In particular I want to acknowledge my brother, Sean Askay, whoseguidance in developing Python code enabled me to acquire much of the data needed toanswer my research question. Additionally I am grateful to Bethany Ritter, who hassupported me in so many ways during this dissertation. I thank you for your enduringpatience and understanding during the late nights that I worked on this project, yourinterest in listening to and thinking through ideas with me, and your willingness toproofread this dissertation. I will always be thankful for you. Lastly, my friends both nearand far have given me much moral support and made my time in graduate school anenjoyable experience. Thank you all.
viTABLE OF CONTENTSCHAPTER I: INTRODUCTION11.1 Rationale for Research31.2 Purpose of the Dissertation61.3 Context of the Dissertation71.4 Organization of the Dissertation8CHAPTER II: LITERATURE REVIEW2.1 Part I: The Crowd11122.1.1 Online Reviews142.1.2 Search versus Experience Goods142.1.3 Distribution of Reviews162.1.4 Business Interests in Crowdsourced Platforms192.1.5 Summary of Crowdsourced Online Reviews212.2 Part II: Crowdsourcing as Collective Action2.2.1 Collective Action22222.2.1.1 Civic Associations and Interest Groups232.2.1.2 Informal Networks and Crowds242.2.2 Crowdsourcing as Collective Action252.2.3 Contemporary Collective Action252.2.4 Collective Action Space282.2.4.1 Interaction292.2.4.2 Engagement302.2.4.3 Quadrants of the Collective Action Space32
2.2.4.3.1 Traditionalistsvii332.2.4.3.1 Enthusiasts342.2.4.3.1 Minimalists342.2.4.3.1 Individualists352.2.6 Summary of Crowdsourcing as Collective Action382.2.7 Research Question39CHAPTER III: METHODOLOGY403.1 Research Site413.2 Description of Yelp433.2.1 How Yelp Makes Money433.2.2 Yelp’s Organizational Structure453.2.2.1 Yelp, Inc.453.2.2.2 Reviewers (The Crowd)453.2.2.3 Scouts453.2.2.4 Elite Squad463.2.2.5 Community Manager483.2.3 Structure of Yelp Website483.2.3.1 Localized Homepage483.2.3.2 Review of the Day493.2.3.3 Business Reviews503.2.3.4 Contributing a Review523.2.3.5 Review Filter533.2.3.6 Flagging Content54
3.2.3.7 Reviewer Profilesviii543.2.3.8 Compliments553.2.3.9 Talk Forums563.2.3.10 Events583.2.3.11 Yelp Elite Squad Page603.2.3.12 Mobile Application633.2.4 Summary of Yelp’s Structure633.3 Philosophy of Science643.4 Research Design673.4.1 Quantitative Data Collection683.4.2 Qualitative Interviews693.4.2.1 Recruitment703.4.2.2 Interviews723.4.3 Organizational Documents733.4.4 Data Analysis733.4.5 Coding76CHAPTER IV: QUANTITATIVE RESULTS4.1 Descriptive Statistics78794.1.1 Descriptive Statistics of Community Managers and Scouts794.1.2 Descriptive Statistics for All Reviews804.1.3 Descriptive Statistics of Elite and Non-Elite Reviewers814.1.3.1 Descriptive Statistics of Non-Elite Reviewers824.1.3.2 Impact of Filtered Reviews on Non-Elite Distribution83
4.1.3.3 Descriptive Statistics of Elite Reviewersix844.1.4 Impact of Elite Reviews on Overall Distribution854.2 Mann-Whitney U-Test and Kolmogorov-Smirnov Test874.2.1 Comparing Distribution of Elite versus Non-Elite Reviews874.2.2 Comparing Distribution of Elite versus Scout Reviews884.3 Chi-Square884.4 Summary of Quantitative Analysis89CHAPTER V: QUALITATIVE FINDINGS5.1 Being Yelpy5.1.1 Balanced Reviews9091925.1.1.1 Elite Bumping Up Reviews945.1.1.2 Non-Elite Overcorrecting Reviews945.1.2 Detailed Reviews5.2 How Yelp Enables Balanced and Detailed Reviews95975.2.1 Organizational Messages975.2.2. Technological Features975.2.3 Community Enforcement995.2.3.1 Yelp Jail1005.3 Eliminating Anonymity5.3.1 Real Photo and Name1031035.3.1.1 Recognized by Elites1055.3.1.2 Recognized by Personal Contacts1065.3.1.3 Recognized by a Business107
5.3.1.4 Anonymity and Non-Elites5.4 How Yelp Encourages Eliminating Anonymityx1091105.4.1 Vetting by the Community1105.4.2 Connecting Personal Contacts1125.4.2 Local Reviews1135.4.3 Libel1145.5 Peer Pressure1165.6 How Yelp Encourages Peer Pressure1185.7 Reviewing Extreme versus Reviewing Everything1205.7.1 Non-Elite Review Extreme Experiences1205.7.2 Elites Review Everything1225.8 How Yelp Encourages Elites to Review Everything1245.8.1 Monitoring by the Community Manager1245.8.2 Ambiguous Criteria for Elite Status1275.8.3 Organizational Messages1285.9 The Elite Squad: Everything to Everyone1305.9.1 Status-Seeking1315.9.2 Instrumental Rewards1335.9.3 Social Rewards1355.9.4 Soapbox1375.10 Methodological Rigor1385.11 Summary140CHAPTER VI: DISCUSSION141
6.1 Discussion of Quantitative Analysisxi1426.1.1 Consistent with Previous Studies1436.1.2 Elite Produce Reviews Differently1446.1.3 Elite Produce Similar Reviews as Scouts1456.1.4 Personalized Profile Images and Moderate Reviews1456.2 Summary of Quantitative Findings1486.3 Discussion of Qualitative Findings1486.3.1 Reactionary versus Pre-Meditated Reviewing1496.3.2 How Yelp Organizes the Crowd1506.3.2.1 From Impersonal to Personal Interaction1536.3.2.2 From Entrepreneurial to Institutional Engagement1546.3.3 Quadrants of ControlCHAPTER VII: IMPLICATIONS, LIMITATIONS, AND CONCLUSIONS7.1 Implications for Theory1561591597.1.1 Community in the Crowd1597.1.2 Extending the Collective Action Space Theory1617.1.3 The Role of Incentives1627.1.4 Personalizing Profiles1637.2 Implications for Methodology1647.2.1 Big Data1647.2.2 Distributions, not Means1667.3 Implications for Practice7.3.1 Overcoming the J-Shaped Distribution168168
7.3.2 Quadrants of Controlxii1687.4 Limitations1697.5 Conclusion171REFERENCES172APPENDIX A: INTERVIEW GUIDE181
xiiiLIST OF TABLESTABLE 1: Descriptive statistics for community managers and scouts79TABLE 2: Descriptive statistics of review ratings82TABLE 3: Cross-tabulation of review rating and Elite versus Non-Elite status89
xivLIST OF FIGURESFIGURE 1: Example of a J-shaped distribution17FIGURE 2: The collective action space29FIGURE 3: The collective action space36FIGURE 4: Distribution of reviews from Citysearch, Yahoo Local, and Yelp42FIGURE 5: Yelp homepage, localized to Charlotte, NC49FIGURE 6: Yelp business profile and reviews51FIGURE 7: Explanation of Yelp’s star ratings52FIGURE 8: Submitting a review53FIGURE 9: Yelp reviewer profile55FIGURE 10: Yelp Compliment options56FIGURE 11: Examples of Yelp Compliments56FIGURE 12: Yelp Talk discussion forums57FIGURE 13: Examples of localized city webpage with and without Events page58FIGURE 14: Yelp Charlotte Events Page59FIGURE 15: Example of Yelp Event60FIGURE 16: Elite Squad Calendar of Events61FIGURE 17: Elite Squad Webpage62FIGURE 18: Examples of Elite Squad behavior guidelines62FIGURE 19: Frequency percentages for community manager and scouts80FIGURE 20: Frequency percentage of all reviews81FIGURE 21: Frequency percentage of Non-Elite reviews83FIGURE 22: Frequency percentage of Elite reviews85
FIGURE 23: Additive impact of elite reviews on overall distributionxv86FIGURE 24: Personalized order of reviews showing Friends first119FIGURE 25: Contributing a review showing reviews from Friends120FIGURE 26: Elite badges131FIGURE 27: Distribution of reviews from users with a blank profile image147FIGURE 28: Distribution of reviews from users with a personalized profile image147FIGURE 29: The collective action space153FIGURE 30: Elite and Non-Elite Reviewers in the collective action space157FIGURE 31: Movement of individuals within the collective action space161
CHAPTER I: INTRODUCTION“How can you get crowds to do what your business needs done?”- Malone, Laubacher, and Dellarocas (2010, p. 21)People have long paid for product (e.g., Consumer Reports Magazine) andbusiness reviews (e.g., Zagat) from experts to guide decisions about products to buy andbusinesses to patron. In the past decade, however, people have increasingly turned to anew communication channel for this information—the online reviews contributed by thecrowd. Moving away from a single rating offered by an expert, these online reviewsaggregate the range of opinions expressed by thousands of people in determining thequality of a product or business. Being perceived to be less biased and more trustworthy,online reviews have become one the most trusted sources of product and businessinformation for the public (Jansen, 2010).The study of online reviews has increased dramatically across disciplines such asinformation systems, management, communication studies, and advertising. On websiteslike Amazon.com, Yelp.com, Google Places, and TripAdvisor.com, online reviewsproduced by the crowd have become a powerful source of influence that guides thepurchase decisions of the public. Although there are some exceptions (e.g., Bronner & deHoog, 2011), the vast majority of these studies focus on online reviews as antecedentsthat predict outcomes such as sales, purchase decisions, or trust.
2Yet, not all reviews are created the same, as some reviews may be consideredmore helpful than other by viewers. That is to say aspects such as extreme versusmoderate ratings, persuasive tactics, and reviewer characteristics can all impact howhelpful reviews are to viewers (Chen, Dhanasobhon, & Smith, 2006; Mudambi & Schuff,2010; Willemsen, Neijens, Bronner, & de Ridder, 2011). This is an importantconsideration for businesses that design and maintain online review platforms, as theyoften compete with other review websites for advertising revenue. The notion thatbus
supported me in so many ways during this dissertation. I thank you for your enduring patience and understanding during the late nights that I worked on this project, your interest in listening to and thinking through ideas with me, and your willingness to proofread this dissertation. I will always be thankful for you. Lastly, my friends both near and far have given me much moral support and .
