Step-by-step Guide To Critiquing Research. Part 2 .

Transcription

Step-by-step guide to critiquingresearch. Part 2: quaiitative researciiFrances Ryan, Michael Coughlan, Patricia CroninAl stractAs with a quantitative study, critical analysis of a qualitative studyinvolves an in-depth review of how each step of the research wasundertaken. Qualitative and quantitative studies are, however,fundamentally different approaches to research and thereforeneed to be considered differently with regard to critiquing. Thedifferent philosophical underpinnings of the various qualitativeresearch methods generate discrete ways of reasoning and distinctterminology; however, there are also many similarities within thesemethods. Because of this and its subjective nature, qualitative researchit is often regarded as more difficult to critique. Nevertheless, anevidenced-based profession such as nursing cannot accept research atface value, and nurses need to be able to determine the strengths andlimitations of qualitative as well as quantitative research studies whenreviewing the available literature on a topic.Key words: Critical appraisal Qualitative researchand quantitative studies are flnidanientally(difFercnc approaches to research and thereforeneed to be approached differently with regard tog. Qualitative research is essentially anassortment of various approaches that have commonalities aswell as differences (Parahoo, 2006).The difierent philosophicalunderpinnings of the various qualitative research methodsgenerate discrete ways of reasoning and distinct terminology;however, there are many similarities within these methods(Burns and Grove, 1999) that can be categorized together.Qualitative research docs not regard truth as objective,but as a subjective reality that is experienced differently byeach individual (Vishnevsky and Beanlands, 2004), Nor doproponents of qualitative research believe that a phenomenoncan be isolated into multiple variables that can be studiedindependently. Qualitative research asserts that a phenomenonis more than the sum of its parts, and must therefore be studiedin a holistic manner. As a result, the purpose of this paradigmis not to attempt to generalize data to the population but toFrances [lyan, Michael Coughlan, Patricia Cronin are Lecturers,School of Nursing and Midwifery, University of Dublin,TrinityCoilege, DublinAccepted for publication: March 2007738explore individuals' experiences (Vishnevsky and Beanlands.2004) and in sonic instances the development of new theory.Elements Influencing beltevabillty of the studyThe elements mtlucncmg bclicvability can help the readerto focus on what to expect from a piece of research byasking questions regarding the researcher's academic andprofessional qualifications and the skills demonstrated inpresenting the study (Ryan-Wenger, 1992).These questionsare similar to those asked when critiquing a quantitativestudy and were discussed in part I of this article (Coughlanet al, 2007). Questions relating to these elements arepresented in Tcihic I.Elements influencing robustness of the studyStatement of the phenomenon of interestMany ot the topics examined in research studies are of anabstract nature in that the particular experience may beinterpreted differently by another individual, or by the sameindividual under different circumstances, e.g. when in pain. Inqualitative research these abstract encounters or experiencesare known as phenomena (Polit and Beck, 2006).The topicbeing studied should be clearly identified by the researcher(Connell Meehan, 1999).Purpose/significance of the studyThe researcher should explain next why the study needsto be undertaken and what he/she expects to glean from it.The researcher should also state why the study will be ofsignificance and how it will add to the general body ofinformation on the phenomenon (Connell Meehan, 1999).At this stage the researcher should also justify the use of aqualitative approach and the qualitative methodology to beused (Connell Meehan, 1999).Literature reviewThe function of a literature review in research smdies is toprovide an objective account of what has been written on agiven subject.This in turn should reflect prominent emergingthemes and inform the conceptual framework of the study.Qualitative research follows the naturalistic paradigmbased on the assumption that multiple realities exist and suchrealities are constructed by the research participants. It aimsto explore the phenomenon in question by focusing on theindividuals who experience it (Vishnevsky and Beanlands,2004). Qualitative methods are concerned with experiences,feelings and attitudes, as opposed to precise measurementand statistical analysis. Qualitative methodologies varyBritish Journal of Nursing. 20(17.Vol Ui. No 12

RESEARCH METHODOLOGIESTable 1. Research questions: guidelines for critiquing a qualitative research studyElements Influencing believabllity of the researchE.lement5Writing styleQuestionsIs the report well written - concise, grammatically correct, avoids the use of jargon?Is it well laid out and organized?AuthorDo the researcher's qualifications/position indicate a degree of knowledge in this field?Report titleIs the title clear, accurate and unambiguous?AbstractDoes the abstract offer a dear overview of the study, including the research problem, sample,methodology, findings and recommendations?Elements Infiuencing robustness of the researchEiementsStatement of thephenomenon of interestQuestionsIs the phenomenon to be studied dearly identified?Are the phenomenon of interest and the research question consistent?Purpose/significance ofthe studyIs the purpose ofthe study/research question deariy identified?Literature reviewHas a literature review been undertaken?Does it meet the philosophical underpinnings of the study?Does the review of the literature fuifli its objectives?Theoretical frameworkHa5 a conceptual or theoretical Framework been identified?Is the framework adequately described?Is the framework appropriate?Method and philosophicalunderpinningsHas the phiiosophical approach been identified?Why was this approach chosen?Have the philosophical underpinnings of the approach been explained?SampleIs the sampling method and sample size identified?Is the sampling method appropriate?Were the participants suitable for informing research?Ethical considerationsWere the participants fully informed about the nature of the research?Was the autonomy/confidentiality of the participants guaranteed?Were the participants protected from harm?Was ethicai permission granted for the study?Data col lection/data analysisAre the data collection strategies described?Are the strategies used to analyse the data described?Did the researcher tbilow the steps of the data analysis method identitied?Was data saturation achieved?RigourDoes the researcher dtscuss how rigour was assured?Were credibility, dependability, transferability and goodness discussed?Fi ndi ngs/discussionAre the findings presented appropriateiy?Hcis the report been placed in the context of what was already known of the phenomenon?Has the original purpose of the study been adequately addressed?Conclusions/implications andrecommendationsAre the importance and implications of the findings identified?Are recommendations made to suggest how the research findings can be developed?ReferencesWere all the books, journals and other media alluded to in the study accurately referenced?regarding the inclusion of a literature review before the datacollection period. Similarly, there is debate over whether theconceptual framework should precede the period of datacollection and data analysis.When critiquing qualitative studies, the reviewer mustdecide whether the researcher has rationalized his/her chosenapproach. A major premise of grounded theory, for example,is that data are collected in isolation from any predeterminedUririshjiiunijl fit'Nursing. 2(HI7.Viil Hi. No 13theory or conceptual framework. The literature review istherefore carried out after the data have been collected. Theaim of this approach is to explore concepts embedded in thedata, thereby allowing theory to be generated from the datarather than vice versa (Robinson, 2002).Similarly, in phenomenological investigations the literaturereview may be delayed until the data analysis is complete.This ensures that the fmdings reflect participants' experiences739

and are truly grounded in the data. This is congruent withthe philosophical orientation of phenoi-nenology, whichviews the subjective experience of participants as central tothe methodology (Burns and Grove, 2001). The conceptualframeworks or themes that emerge from the study maythen be supported by evidence gleaned from a subsequentexamination of the literature.The ethnographic approach attempts to examine theexperiences of the person in the context of his/hernatural world and explores the topic of study through theperceptions of the subjects of study. The literature review inethnographic studies is used to demonstrate knowledge ofprevious work in the area, as well as frameworks used in theanalysis of data.According to Meadows (2003), existing literature providesboth the basis for research and the context for interpretingfindings. When critiquing qualitative research studies it isnecessary to appraise the literature review in the contextof the particular methodology used, if the literature reviewis appropriate only after the period of data collection, thenthe researcher needs to identify how this process is to beachieved and in what way the literature is going to be used todetermine similarities with or differences from the researchfindings. Conversely, if the literature review is carried outbefore data collection and analysis, then it is required toTable 2. Characteristics of qualitative researchTruthTTiere are multiple truths - generalization is not soughtPurposeConcerned with discovery and description aithoughverification is aiso possibleContextThere is attention to the social context in which eventsoccur and have meaningEmphasisThere is an emphasis on understanding the social worldfrom the point oF view of the participants in the study- an emic perspectiveApproachThe approach is primarily inductiveRelationship betweenresearcher and participantThere is integration between researcher andparticipant - interaction is valuedSampleUsually small in number but consists of those who areable and willing to describe the experienceDataElicits 'soft data', i.e. wordsData coiiectionThe major data collection techniques includeinterviewing, participant observation, examination ofpersonal documents and other printed materialsProcedures and tools ibr data gathering are subject toongoing revision in the field situationAnalysisAnalysis is presented for the most part in a narrativerather than numericai Form, but the inclusion of somequantitative measures and numerical expressions Isnot precluded in qualitative researchRigourCredibility, transferabiiity (fittingness), dependability.confirmability. goodness740provide a comprehensive and balanced account of previouswork, identifying, where appropriate, the relevant themes.conceptual models and theoretical frameworks that providea sound background to the research.Research questionIn qualitative research, a research question that reflects theidentified phenomenon of interest is used to direct thecourse of the research. A research hypothesis is never usedin qualitative research, unlike quantitative research (ConnellMeehan, 1999). Depending on the qualitative approachadopted, e.g. grounded theory, the research question maybe modified as new data bring new direction to thephenomenon of interest. Such modifications should beexplained and justified by the researcher when they occur.Theoretical frameworkMany qualitative studies are described as inductive/atheoreticalor theory-generating research. This means that the purposeof the study is to develop theory not test it. Therefore, theresearcher does not use an existing or known theory to directthe study.This approach, which is heavily based on the workof Glaser and Strauss (1967), is known as grounded theory.Ethnography and phenomenology are also classed as theorygenerating. It is important that the researcher indicates thisin the study and justifies the adoption of such a stance, e.g.where little is known about the phenomenon under study orwhere existing theories do not seem to provide the answer(Cronin and Rawlings-Anderson, 2004).Some qualitative studies use known theories to 'frame'their studies {McKenna, 1997). This provides boundaries orparameters for the study and guides all stages, including theliterature review, data collection, analysis and presentation oftlndings (Parahoo, 2006).An important point for quahtative descriptive/exploratoryresearch is that there are some limits or boundaries towhat is being studied. For example, if the researcherwished to study an aspect of patients' postoperative painit would be critical to establish how the particular focuswas determined. This is sometimes done by exploring theliterature and identifying the main themes or concepts,which are then used to focus data collection and/or dataanalysis and presentation of the findings.Methodology: research designDesign in qualitative researcii incorporates a range ofapproaches within what is often referred to as thenaturalistic, interpretive or constructivist world view. Theimportant point here is that such a view of the worldincorporates a set of beliefs about knowledge and howthis knowledge is developed. Qualitative research thereforecomprises a set of characteristics that reflect this world view{Table 2). However, it is also important to note that themain qualitative approaches do differ in their disciplinaryor philosophical origins, hence the focus and manner inwhich they undertake sampHng, data collection and analysiswill vary (Table 3). For the reviewer it is essential that theresearcher outlines and justifies the chosen approach inorder to establish coherence and congruence.British Journa] of Nursing, 2t)O7. Vol !6, No 12

RESEARCH METHODOLOGIESTable 3. Approaches In qualitative researchGeneric qualitative researchPhenomenology/hermeneutksGrounded theoryEthnographyVarious schools ofphiiosophyCuiturai anthropoiogyGeneration of theory from the dataModification/extension of ding/meaning ofthe lived experience/phenomenon under studyDirect description of agroup, culture or communityPurposive/convenienceInitial sampling of people abie to giveinformation on the topic. This is foliowedby theoreticai sampiing where furthersampling is guided by the analysis andemerging theoryPurposivePurposive, non-probability.criterion-basedInterviews often semi-structuredInterviews, participant observation,diaries and other documents.researcher's own experienceUnstructured, formal.multiple interviews.written texts, e.g. diariesObservation, interviews andexamination of documents inthe fieldDataanalysisGeneric data analysis tools.e.g. Miles and Huberman (1994)Constant, comparative analysis - datacollection and analysis are linked fromthe beginning of the researchData analysis tools varydepending on school ofphilosophy adoptedDescription, analysis andinterpretation of the cultureFindingsDescription of the phenomenonunder studyDescription of the emergent theory,incorporating the culturai processesand meaningsDescription/interpretationof the phenomenon understudyThe culture as experienced byits members is presentedOriginSymbolic interactionismBroad term for all qualitativeresearch and may have its origins and sociai sciencesin any of the other disciplinesAimsDescription of the issue understudySamplecollectionSampling111 qiiiilitLitive research, participants are usually recruited toa study because of their exposure to or their experienceof the phenoriienon in question. This type of sample tendsto ensure richness in the data gathered and is known aspurposive or purposeful sampling (Fossey et al, 2002).Samples can also be selected as a result of themes thatemerge from the data analysis. The researcher can thenexplore these themes in more depth and/or develop atheory from these data. Tbis type of sampling is known astheoretical sampling (Fossey et al. 2002) and is frequentlyused in grounded theory.Qualitative samples are often small (Fossey et al, 2002)but this is not usually a problem as the researcher is notattempting to generalize the findings. Data gathered fromparticipants build on the information from previous subjectsand tbe accumulated data can offer a significant depth ofinformation on tbe phenomenon. As a result the researchermay discover that no new material is emerging; at tbis point,data gatbering usually stops (Paraboo, 2006).Ethical considerationsIn qu.ilitative researcb tbe most common tools used fordata collection are interview and participant observation.The participants are therefore known to the researcher andanonymity is not possible.The researcher must therefore assureparticipants that their identities will not be revealed to tbereader and the raw data collected will not be released to anythird party (Parahoo. 2006). Both interviews and obser\'ationsin qualitative research can give rise to ethical dilemmas.Brimh Journal ofNursiri(;.2IHI7,Vol 16, No 12Participants should always have tbe rigbt to give informedconsent regarding tbeir participation in any researcb study.In order to do tbis, participants should be fully aware ofthe purpose of the study, what sort of information is beingsougbt, bow it will be used and tbe implications for tbem ascontributors to the research. This moral principle is knownas autonomy (Beauchamp and Childress, 2001); it alsoimplies that participants have the right to withdraw from theresearch at any time.In qualitative interviews tbe role of the interviewer isto encourage participants to 'open up' and discuss theirexperiences of the phenomenon. In doing this, participantscan inadvertently discuss personal information that theyhad not planned to reveal, or that may rekindle tragicor uncomfortable experiences related to the topic beingstudied. Process consent involves continually negotiatingwitb participants to ascertain wbetber tbey are comfortablecontinuing with the interview or would prefer to discontinueparticipation (Polit and Beck, 2006) and can be a useful toolin tbese situations. However, discontinuing participationalone can be insufficient to meet tbe principle of nonmaleficence, so psychological support sbould be in placeto manage any emotional distress that may result from tbeinterview (Smith, 1992).In qualitative research, if vulnerable groups are beingasked to contribute to the study, it is important toensure tbat tbeir rights are protected. Ethical committeeor institutional review board approval has to be sougbtbefore the researcb can be undertaken. Tbe role of etbicalcommittees and institutional review boards is to determine741

tbat ethical principles are being adhered to and thatparticipants are protected from potential sources of harm(Burns and Grove, 1999). It is important to note that withinqualitative researcb, ethical issues often arise at differentstages in tbe study and may be discussed when they occurratber than under a specific beading.Data collectionIn :i qualitative study any number of strategies can beadopted wben collecting data, including tion-numericalquestionnaires with open-ended questions, interviews(semi-structured and unstructured), participant observation,written texts sucb as diaries or emails, and historical orcontemporary documents. Tbe researcher should outlinethe rationale for the chosen method of data collectionand offer sufficient information of the process. If using aparticular approach, such as grounded theory, it should beevident from the discussion that the researcher has adheredto the processes inherent in tbe methodology (7tj/j/f 3).Interviews are by far the most common method ofdata collection and are mainly either semi-structured orunstructured (Holloway and Wheeler, 2002). If a semistructured interview format is selected it sbould be evidenthow tbe themes or questions were derived. In unstructuredinterviews the initial opening question should be presentedand clearly linked to the purpose of the study. Interviewsare more frequently conducted face to face, but online ortelephone interviews are also used.They can be undertakenwitb individuals or groups, sucb as focus groups, and can beone-off or multiple. Tbe rationale for each of these decisionsshould be clearly presented.Altbougb traditionally associated with grounded theory,'data saturation' is often referred to by some qualitativeresearcbers as a point where tbey claim no new informationwill arise from further samphng. Thorne and Darbysbire(2005) suggest tbat some researcbers use the concept ofdata saturation as a convenient stopping point, and it maybe pertinent to assess wbether the study being evaluated,particularly if it is a small-scale descriptive study, could haveachieved this.Data analysisIn qualitative research tbe process by wbich data analysis isundertaken is fundamental to determining the credibilityof tbe findings. Essentially it involves tbe transformation ofraw data into a final description, narrative, or themes andcategories. There is considerable variation in how tbis isundertaken, depending on the research question and theapproach taken (Visbnevsky and Beanlands, 2004).Some researcbers use generic data analysis tools whereasothers use less structured and more creative approaches.What is important is that the process is described insufficient detail to enable the reader to judge whether thefinal outcome is rooted in the data generated (Hollowayand Wheeler, 2002). The researcher sbould demonstrateunderstanding of concurrent data collection and analysis,the processes of organizing and retrieving data, as wellas the steps in coding and thematic analysis. In addition,verification strategies, if used, should be presented. Examples742include use of an expert panel or member cbecking(verifying witb participants).Several computer-assisted packages are available to assisttbe qualitative researcher during analysis, e.g. NUD*1ST(Non-nutnerical Unstructured Data Indexing, Searchingand Theorising), Etbnograph and NVivo (Robson. 2002).Tbere are inberent advantages to tbese packages in termsof bandling large amounts of data and assisting witb codingand organizing tbe material. However, the rationale for bowand wby a particular tool was chosen sbould be evident.Although data analysis is central to qualitative research, itis often poorly delineated in research publications. Very fewoffer sufficient detail to determine the emergence of thefindings from tbe raw data, with the result that readers areasked to 'accept' what tbey see. According to Thorne andDarbyshire (2005), the obligation to show the data that ledto tbe fmdings is a reasonable one.Rigour (trustworthiness)Unlike the quantitative (positivist) paradigm tbat seeks toexamine objective, measurable data and causal relationsliipsbetween variables, qualitative researcb or naturalistic inquiryconcerns itself with processes and meanings that cannotalways be experimentally examined. Socially constructedrealities and relationsbips between the researcher and wbatis being studied are essential components of qualitativeinquiry (Denzin and Lincoln, 2003). Proponents ofqualitative approaches emphasize the value-laden natureof naturalistic inquiry; a commonly heard criticism is thatqualitative research is subjective, anecdotal and subject toresearcber bias (Koch and Harrington, 1998).Pilkington (2002) suggests that because qualitativemethods are aimed at primarily understanding humanexperiences and ultimately theory development, alternativecriteria are required for ensuring the scientific merit ofqualitative research studies. As quantitative studies areconcerned with the generalizability and reproducibilityof findings, the concepts of reUability and validity areseen as appropriate criteria to use when evaluating tbeadequacy or robustness of quantitative researcb. Tbere ismuch discussion about tbe applicability of validity andreliability to qualitative researcb (Kocb and Harrington,1998; Tobin and Begley, 2004; Hoye and Severinsson,2007).The challenge to alternative paradigms or qualitativeapproaches is to produce plausible, robust researcb and todemonstrate rigour.Rigour is the means of demonstrating tbe plausibility,credibility and integrity of the qualitative research process.Tbe rigour, or trustwortbiness, of a study may be establishedif the reader is able to audit the actions and developmentsof the researcher (Koch. 2006). According to Burns andGrove (2001), tbe critique of qualitative researcb requires anappraisal of the rigour in documentation, procedural rigour,and ethical rigour: Rigour in documniuttioti ensures there is a correlationbetween tbe steps of tbe research process and the study inquestion, commencing with the phenomenon of interestand following through to the recommendations andimplications for practice.Britishjounial of Nursing. 21MJ7, Vol 16, No 12

RESEARCH METHODOLOGIES Proceduralrigourrefers to appropriate and precise data inquiry and must be reflected in the entire study. Goodnesscollection techniques and incorporates a reflective/criticalneeds to be evident in the philosophical background andcomponent in order to reduce bias and misinterpretations.study design, providing explicit explanations regarding Ethical rigour describes bow confidentiality issues and tbe tbe study context, data collection and management andrigbts of participants are dealt with during the researcbtbe interpretation and presentation process. Goodness,process.tberefore, is a principle that sbould be present duringallstages of the research study and explicit in tbe finalThe most common criteria used to evaluate qualitativewritten report.research studies are credibility, dependability, transferabilit 'and confirmability (Table 2). Other terms such as goodness.md fruitflilness may also be used (Lincoln and Guba, 1985). Findings and discussionrbe researcber needs to identify tbe criteria used and tbe As stated above, findings from qualitative studies can bereader sbould be able to clearly follow each step of tbe represented as a narrative (story), themes, description ofresearch process:tbe phenomenon under study or an interpretive account Crcdiliillty refers to the fiiithflilness to the description of tbe of the understanding or meaning of an experience.phenomenon in question (Koch and Harrington, 1998). It Regardless of how the final outcome is presented, theaddresses the issue of whether there is consistency between researcber sbould discuss tbe findings in the context ofthe participants' views and the researcher's representationwhat is already known.of tbem. Koch (2006), asserts tbat credibility may beFor many this will involve further literature review relatedenhanced by the researcher describing and interpreting to the final outcome. However, the reviewer should bewarehis/her experiences as researcber, and also by consulting of exaggerated claims as to the significance of the researchwitb participants and allowing them to read and discuss tbe and implications for practice, and further researcb sbouldresearcb findings. Credibility may also be demonstrated by be located in tbe study's findings. Moreover, tbe researcberprolonged engagement, observation and audit trails.sbould relate the findings of the study back to the original DcpaulabiUt)' (auditability) is an integral component of research purpose, and illustrate whether or not it has beenrigour and involves the researcber giving the reader adequately addressed (Thorne and Darbysbire, 2005).sufficient information to determine how dependable tbestudy and the researcher are. A study may be deemed Conclusions, implications and recommendationsauditable wben another researcher can clearly follow the The researcher sbould conclude by placing tbe findingstrail used by the investigator and potentially arrive at tbe in a context tbat indicates bow tbis new information is ofsame or comparable conclusions. A research study may interest, and its impHcations for nursing. These conclusionsbe shown to be dependable by producing evidence of asbould reflect tbe study's fmdings and ideally sbould offerdecision trail at eacb stage of the research process.According recommendations as to how tbey may be developed.to Koch (2006), tbis provides the reader witb evidence ofthe decisions and choices made regarding theoretical andReferencesmethodological issues throughout the study and entails An accurate list of all the books, journal articles, reports anddiscussing explicitly tbe reasons for such decisions. It is also otber media referred to in the study sbould be included in anecessary for each stage of the research to be traceable and reference list at the end of tbe study (Polit and Beck, 2006).clearly documented.For tbe reader who has an interest in the topic that was Transferabiiity (fittingness) refers to wbetber or not findings studied, the reference list can be a good source of furtbercan be applied outside tbe context of the study situation. reading.When critiquing qualitative research, a study can bedeemed to have met the criterion of transferabiiity wben Conclusionthe findings can 'fit' into other contexts and readers canAs witb a quantitative study, critical analysis of a qualitativeapply the findings to their own experiences. Transferabiiity study involves an in-deptb review of how each step of theis also enhanced when the results are meaningful to research was undertaken. Because of tbe subjective nature ofindividuals not involved in tbe research study.qualitative research it is often regarded as more difficult to Confirmability requires the researcber to demonstrate how critique. However, an evidenced-based profession cannotconclusions and interpretations have been reacbed. It is accept any researcb at face va

Step-by-step guide to critiquing research. Part 2: quaiitative researcii Frances Ryan, Michael Coughlan, Patricia Cronin Al stract As with a quantitative study, critical analysis of a qualitative study involves an in-depth review of how each step of the research was under