Transcription
Qualitative Research Methods inComputer Science27th June 2012Qualitative Research Methodsin Computer ScienceOverviewEdwin Blakeedwin@cs.uct.ac.zaDepartment of Computer ScienceUniversity of Cape Town1. What is research and qualitative research about?2. Examples of Qualitative Research1.2.33.4.5.Case studiesContextual InquiryEthnographyConversation AnalysisGrounded Theory3. Pragmatist Epistemology6. Action Research4. Quality5. Concluding Case Study27/6/12Qualitative Research in CSQualitative ResearchResearch: an activity that contributes to our understandingResearch methods are the means by which a disciplineacquires and constructs knowledge.Different philosophical assumptions about what constitutesrelevant knowledge results in different strategies of enquiry and methods qualitative research, quantitative research and combinations (mixed methods research)27/6/12Qualitative Research in CSOntology, Epistemology and MethodologyWarning to the InnocentThis stuff is controversial!At least in Computer Science (much more accepted inInformation Systems)The terms are loaded.The opposition can be very prejudicedThe practitioners are often puritanicalIf this is for a thesis then choose your examiners well.3IOntological beliefs: beliefs regarding reality. For example:27/6/12University of Cape Town4Ontology, Epistemology and Methodology IIWhat is the researcher’s underlying ontology(fundamental worldview) and epistemology (theory ofknowledge)?What strategies of inquiry governs our choice and use ofmethods?What methods of data collection and analysis do wepropose to use?Epistemological assumptions: assumptions regarding howwe come to know about the worldMethodological choices are the means we choose inattempting to achieve desired ends.Particular ontological beliefs particular epistemologicalassumptions.Particular epistemological assumptions certainmethodologiesQualitative Research in CSQualitative Research in CSYour knowledge claims informs your strategies of inquiryand your choice of methods:nature is an objective reality that exists regardless ofhuman perception , orthere is only a subjective reality, created in our minds.27/6/122527/6/12Qualitative Research in CS61
Qualitative Research Methods inComputer Science27th June 2012Four fundamental approachesI1. Positivist researchFour fundamental approaches2. Critical researchThere is a real objective world that imposes itself on ourminds via the sensesA search for truth, statements can be verifiedFocuses on a critical understanding of the situation or practicein order to plan for transformative action. Emphasizes socialchange.OriginatedOg a ed in Hegelianege a anda d Marxiana a traditions.ad o sPopper: a scientific statement must be falsifiable(Post) Positivism3. Design (Science) researchis generally quantitative in natureuses hypothesis testingmakes claims of replicability, reliability and validitytries to uncover “laws” of natureHelp designers to investigate people, form, and processor the IS term for Experimental Computer Science Confusing term for CS since all research must lead to anartefactMany different epistemologies can lead to a positivistmethod27/6/12IIQualitative Research in CS4. Interpretivist research coming nextMany features in common with other qualitative approaches7Interpretivist Research —Metaphysical AssumptionsThe Observer's Perspective is a Factor:27/6/12Qualitative Research in CS8Interpretivist Research — Data AssumptionsObjectivity, in the sense in which it is used in ScientificResearch, is meaningless, because:in the selection and formulation of Theoryin the formulation of Hypothesesin choices made in the Research Design processin the selectiveness of observationin the process of observationit presumes the existence of a unitary Truthit presumes that Truth to be accessible by humansit overlooks the fact that entities within the domain thinkthey can exercise free willAn Alternative Interpretation:Try to identify Researcher BiasesTry to avoid or allow for Researcher BiasesEnable evaluators to assess Researcher Biases27/6/12Qualitative Research in CS927/6/12Assumes methods arevalue neutral and ahistoricalTreats people as objects ofinquiryactually they are subjectsand themselves initiatorsof action10Types of Qualitative ResearchCriticismsPositivismQualitative Research in CS1.2.3.4.5.6.Qualitative ResearchData is flawed due tosubjectivitySmall samples so noreplicability, reliability norvalidityCase studiesContextual InquiryEthnography & EthnomethodologyConversation AnalysisGrounded TheoryAction Researchis itself a product of ourminds, and so we cannotexclude ourselves from theprocess of creatingknowledge.27/6/12Qualitative Research in CSUniversity of Cape Town1127/6/12Qualitative Research in CS122
Qualitative Research Methods inComputer ScienceCase Studies27th June 2012IFocuses on the characteristics, circumstances, andcomplexity of a small number of casesOften uses multiple methods.Not really a specific method, but a class of studies.Findings can raise awareness of general issues, but theaim is not to generalise the findings to other cases.Case studies primarily use qualitative research techniques,but can exploit quantitative methods.Case StudiesIIStudies a phenomenon in its real-life context (as opposedto experiments, simulations, or surveys or historicalanalyses)Can be positivist, interpretive or critical.types egVarious types,e.g.single case,multiple cases,critical case,exemplary case.Exploratory (develop propositions for further use) versusdescriptive (study incidence and prevalence).27/6/12Qualitative Research in CS13Contextual Inquiry/DesignNot a research method as suchA design-oriented approach aimed at getting a grip on‘context’, what it is, how it interferes.27/6/12Principles of Contextual Inquiry:Apprentice / Master relationship is fundamental for theinvestigationNo explicit teaching, just watching the work, detectingwhat matters, seeing details.Requires humility, inquisitiveness, attention.Ask questions.Qualitative Research in CSCultural probes consisted of:1 Disposable camera2 CD ROM3 Morning task4 Pencil, pen and felt tips5 Easter eggs6 Workbooks7 Diary8 Images for collages9 Stickers10 Images, scissors, glueQualitative Research in CS 11 Information sheet16Principles of Contextual Inquiry:interpretationMaterials for ggenerative session:Interviewer/interviewee: you are not there to get a list ofquestions answered.Expert/novice: you aren’t there to answer questions either.Guest/host: it is a goal to be nosy.University of Cape TowncontextDesign ideas are the end product of a chain of reasoning.Sharing interpretations with users won’t bias the data, butteaches them to see structure in work, and let them fine-tuneinterpretations.Help users articulate their work experience, alternatebetween watching and probing,Teach users how to see work by probing work structure.Avoid relationship models other than Apprentice / Master.Not:Qualitative Research in CS27/6/1215Principles of Contextual Inquiry: partnership27/6/1214Go to where the work is;Summaries versus ongoing experienceAbstract versus concrete dataPractical way to gather information relevant for design,used in HCI,C CSCCSCW,27/6/12Qualitative Research in CS17123456789Play doughSkewersLiquoriceSticky tapeStickersPost-it notesPipe cleanersBalloonsVarious tinkeringmaterials10 Scissors11 Felt tips1227/6/12Glue (not shown)Qualitative Research in CS183
Qualitative Research Methods inComputer SciencePrinciples of Contextual Inquiry:27th June 2012focusEthnographyFrom social and cultural anthropology.Clear focus steers the conversation, focus reveal detail, butconceals the unexpected (look for surprises andcontradictions).Commit to challenging your assumptions and validating them.Rich descriptions based on extended fieldwork of people in theirnatural environment.Aim: understanding how people perceive and organise their world.Cultural and conceptual phenomenaBehavioural patterns and material conditions.Important principle: Immersion – researcher should spend asignificant amount of time in the field. Participant observation is thebasic resource.Popular in HCI (especially CSCW)aim to inform design.You cannot do ethnography without much training (years)Settle for ethnographically inspired — or some such term.27/6/12Qualitative Research in CS1927/6/12EthnomethodologyEthnomethodology the study of people’s methods.Study people’s everyday ways to produce orderly social interaction:How do people give sense to and accomplish their daily actions(communicating, making decisions, reasoning)?Skills and practices that people use understand each other and socialsituations.A central method for ethnomethodologists.Coherent communication is produced according to rules,the aim:to discover these rules, anddescribe the conversational structures they generate.goes beyond grammatical analysis of statements.Relies on detailed transcripts of conversation (naturallyoccurring or interviews).Technique: disrupt what is taken for granted.Answers how-questions rather than what-questions.27/6/12Qualitative Research in CS2127/6/12Grounded TheoryBarney Glaser and Anselm Strauss (1967) criticised “theoveremphasis in current sociology on the verification oftheory and a resulting de-emphasis on the prior step ofdiscovering what concepts and hypotheses are relevantfor the area one wishes to researchresearch”.Argued that any theory that is developed should begrounded in data, not be imposed from above.27/6/12Qualitative Research in CSUniversity of Cape Town20Conversation AnalysisFocus on common-sense practices.Observable and reportable (speech and face-to-face behaviour).Qualitative Research in CS23Qualitative Research in CS22Aim of Grounded Theoryto understand the phenomena in its own way,to generate theory from data not the other way round.(Inductive approach where no pre-conceived theoreticalmodels are applied)27/6/12Qualitative Research in CS244
Qualitative Research Methods inComputer Science27th June 2012Grounded theoryGrounded theory as theory is:Have come to believe that creating knowledge isinextricably intertwined with effective action.“inductively derived from the phenomenon it represents.That is, it is discovered, developed and provisionallyverified through systematic data collection and analysis ofdata pertaining to that phenomenon. Therefore datacollection, analysis and theory stand in reciprocalrelationship with each other. One does not begin with atheory, then prove it. Rather one begins with an area ofstudy and what is relevant to that areas is allowed toemerge.” (Strauss and Corbin, 1990)Knowledge that does not lead to effective action is notreally knowledgeA ffailureiltto createt effectiveff ti systemstiis equivalenti l t tto afailure of understanding.Compatible with Action Research .PRACTICE — A PRAGMATISTEPISTEMOLOGICAL VIEWTheory should ’fit’: the categories must be readily (notforcibly) applicable to the dataTheory should ’work’: be meaningfully relevant and haveexplanatory power27/6/12Qualitative Research in CS2527/6/12Qualitative Research in CSAction Research: OverviewOriginated in social sciences after World War 2(“a therapy for social illnesses”)Aims:Active involvement and interventions,the researchers have a change agenda, a vision of whatcan be done.participants have a view of what they wantPhased and iterative (cyclic):Diagnosing, planning intervention, conductingintervention, evaluating, new diagnosis, etc.Qualitative Research in CSAction ResearchAction Research CycleFacilitating change incommunity throughfacilitating actionCyclical softwaredevelopment process:participatory design prototype evaluation.Diagnosing planning implementingplan observing results reflecting on the resultscontributing to practical concerns (e.g., an organisation inneed of change) and togeneratet new knowledgekl d simultaneouslyi ltl27/6/122627IMotivations27/6/12Qualitative Research in CSAction Research28IIAction research has been typified as a way to buildTo make academic research relevant, researchers shouldtry out their theories with practitioners in real situationsand real organizationsThe emphasis is more on what practitioners do than onwhat they say they dotheory,knowledge, andpractical actionby engagement with the world in the context of practiceitselfKey AssumptionsSocial settings cannot be reduced for study, andActions brings understanding27/6/12Qualitative Research in CSUniversity of Cape Town2927/6/12Qualitative Research in CS305
Qualitative Research Methods inComputer Science27th June 2012Action ResearchIIIA research approach with the dual aims of action andresearchthe researcher is immersed in the communitythe work unfolds in response to the situation and not tothe researcher’s requirementssituated in the local context and all the questions,problems, and issues arise from that contextdescriptions and theories are built up by iteration withinthe context and are tested within the situationthere is close democratic collaboration betweenresearchers and the participantsaction to bring about change in some community ororganisation or program;h tto iincrease understandingd t di on thresearchthe partt off ththeresearcher or the client, or bothThis joint collaboration has to be within a mutuallyacceptable ethical framework27/6/12Qualitative Research in CSFeatures of Action Research3127/6/12Phases of Action Research: DiagramEVALUATINGStudying the outcomesof an action27/6/12Carrying out Action and ResearchActions includes: fieldwork entrance letters, fixingcomputers items, meetings and workshopsAnything that consumes our time in the field is part ofthe actionACTION PLANNINGConsideringg alternativecourses of actionCommon ContextEthical agreementthat constitutes theresearchenvironmentQualitative Research in CSUse some known data analysis techniquesAlign our field notes empirical material in thosetechniquesACTION TAKINGImplementinga course of actionThink and make sense of the actions and resultsSome time is needed away fr
Computer Science 27th June 2012 University of Cape Town 3 Case Studies I Focuses on the characteristics, circumstances, and complexity of a small number of cases Often uses multiple methods. Not really a specific method, but a class of studies. 27/6/12 Qualitative Research in CS 13 Findings can raise awareness of general issues, but the
