WIXLIB

Ohle et al. BMC Medicine 2011, egions illustrating the precision with which pooledvalues are estimated and a 95% prediction region, illustrating the amount of between-study variation. Analyseswere carried out using STATA software (StataCorp LP,College Station, TX, 77845, USA), using the “metandi”command [16,17].Calibration analysis of the Alvarado scoreThe initial derivation study of the Alvarado score wasused as the predictive model against which all validationstudies were compared [5]. The number of patientsdiagnosed with appendicitis as estimated by the Alvarado score (predicted events) was compared to the actualnumber of patients with appendicitis (observed events)in each of the validation studies. The analysis was performed separately across three risk strata of the Alvarado score (low risk, score 1 to 4; intermediate risk,score 5 to 6; and high risk, score 7 to 10) (Figure 1).Within each risk stratum, each of the three main studypopulations, men, women and children were analysedseparately [8,10-12,18].The results from the calibration assessment were presented as risk ratios (RRs with 95% confidence intervals)and are illustrated as forest plots. RR 1.00 indicates anunder-prediction of appendicitis by the score (observednumber with appendicitis is greater than the predictednumber) and RR 1.00 indicates an over-prediction ofappendicitis by the score (observed number with appendicitis is less than the predicted number). RR 1 indicates a matched calibration between observed andpredicted numbers. Review Manager 5 software fromthe Cochrane collaboration was used to perform thepooled analysis, determine heterogeneity and producethe forest plots. RRs with their 95% CIs were computedby the Mantel-Haenszel (M-H) method. A randomeffects model was used and heterogeneity assessed by I2statistic.Prevalence was investigated as a source of heterogeneity in a subgroup analysis. Studies were dichotomised,based on their prevalence, as being either higher orlower than the Alvarado’s derivation study; the effect onheterogeneity and the calibration of the score were alsoinvestigated.ResultsThe literature search yielded 3,000 titles and abstractsfor screening. The full text of 91 articles met the eligibility criteria, and these articles were retrieved (Figure2). Thirty-seven articles were included from the search,and a further five articles were retrieved after citationsearching, with a total of 42 articles meeting the inclusion criteria. The included studies came from a varietyof settings and countries (Table 1). Nine studies tookplace in a surgical ward; three studies only specified thatpatients were hospitalised, all remaining studies werePage 4 of 13performed in an emergency department setting. Detailedcharacteristics of all included studies are presented inTable 1.Results of the quality assessment are shown in Figure3. The overall quality of the included studies is considered acceptable for most of the quality items. Theassessment of the clinical variables composing theAlvarado score and the reference standard for diagnosis(histological results of the appendectomy) were interpreted independently in most studies. The retrospectivestudies rarely reported if the scorer was aware of thefinal diagnosis (blind assessment). The quality item,‘time between tests’, is the time between administeringthe Alvarado score and verifying the diagnosis withpathology or follow-up and was very poorly reported. Aspart of our inclusion criteria, all studies had to confirmthe diagnosis of appendicitis on those undergoingappendectomy; however, follow-up of those dischargedwas poor in the majority of studies (item ‘All verifiedwith reference test’).Diagnostic accuracy of the Alvarado scoreThe Alvarado score discriminated well as an observation/admission criterion (cut point of 5) by achievinghigh pooled sensitivity of 99% overall (n 28 studies,[5,8,10,18-42]) and in studies where data were available,it also performed well in the subgroup analysis for men,woman and children (pooled sensitivities: 0.96 for men,n 5 [23,30,33-35]; 0.99 for women, n 5[23,30,34,35,43] and 0.99 for children, n 9[10,21,23,27,28,30,40-42]) (Table 2 and Additional file 1- Figure S1). In patients presenting with higher Alvaradoscores (cut point of 7, the criterion for surgery), pooleddiagnostic accuracy results had more limited clinicalvalue (pooled specificity for all studies 0.82, n 29,[5,8,10,11,18-25,27-32,34-38,41,42,44-47]), with pooledspecificities ranging from 0.57 for subgroup analysis ofmen (n 6, [9,23,30,34,35,45]), 0.73 for subgroup analysis of women (n 7, [9,23,30,33-35,45]) and 0.76 forsubgroupanalysisofchildren(n 9,[10,21,23,27,28,30,41,42,47]) (Table 2 and Additional file1 - Figure S1).Overall, heterogeneity was high when all studies wereincluded and was particularly high in the children subgroup as indicated by the variance logit transformedsensitivity and specificity (Table 2) and the predictionellipses on the SROC curves Additional file 1 - FigureS1).Calibration of the Alvarado scoreThe Alvarado score performed well in all three riskstrata for men: (low risk RR 1.06, 95% CI 0.87 to 1.28;intermediate risk 1.09, 0.86 to 1.37 and high risk 1.02,0.97 to 1.08). In women, there was a systematic over-

Ohle et al. BMC Medicine 2011, age 5 of 13Table 1 Characteristics of included studiesFirst author,study pendicitisprevalence(%)CountrySettingStudy typePatient populationAbdeldaim2007 [19]242Median 42Range 8 to 76Male veRight iliac fossa painAl Qahtani 2004[8]211Mean 32Range 13 to 70Male 125Female cted acuteappendicitisAlvarado 1986[5]277Mean 25.3Range 4 to 80Male 131Female 9682USAHospitalinpatientsProspectiveAbdominal painArain 2001 [20]100Mean 19.9Males 44Females5648PakistanSurgical unitProspectiveSuspected acuteappendicitisBaidya 2007[44]231Mean 26.3Range 16 to 65Male 141Female 9052IndiaEmergencydepartmentProspectiveRight iliac fossa painBond 1990 [21]187Range 0 to 1861USAEmergencydepartmentProspectiveAbdominal painBorges spected acuteappendicitisCanavosso veRight lower quadrantpainChan 2001 [25]148Mean 29Range 10 to 73Male 107Female cted acuteappendicitisChan 2003 [24]175SingaporeRight iliac fossa Prospective221Male 130Female 45Male 112Female10943Denizbasi 2003[45]Mean 30Range 8 to 73Mean 26.6ProspectiveAbdominal pain andsuspected acuteappendicitisEscriba 2011[42]99Mean 11.2Range 4 to 17.8Male 62Female 3742SpainEmergencydepartmentProspectiveAbdominal pain/suspected appendicitisFarahnak dominal painGwynn Abdominal painHsiao 2005 [28]222Mean 9.4Range 1 to 13Male 146Female d acuteappendicitisKang 1989 [46]62Mean 45.8Range 18 to 78Male ted acuteappendicitisKhan 2005 [29]100Mean 20.2Range 9 to 56Female 59Male 4154PakistanSurgicalwardProspectiveSuspected acuteappendicitisKim 2006 [9]211--83KoreaSurgicalwardRetrospectiveSuspected acuteappendicitisKim 2008 [18]157Mean 37.1Range 15 to nalstudyAbdominal painLada 2005 [10]83Mean 27.5Range 15 to 75Male 43Female ed acuteappendicitisMalik 2000 [56]100Mean 22Range 14 to 18Male 81Female 1992PakistanSurgical unitProspectiveSuspected acuteappendicitisMcKay 2007[31]150Mean 34Range 18 to 76Male 78Female 6532USAEmergencydepartmentRetrospectiveAbdominal painMemon 2009[32]10091PakistanMuenzer 2010[47]28Mean age 24Male 65Range 13 to 55Female 35Test Cohort Mean 11TestValidation cohortcohortMean 11Male 10Range 2 to 17Female 10ValidationcohortMale 4Female udyUnclearSuspected acuteappendicitisAbdominal painAge 2 to 6 23 Age Male 407 to 10 38 Age Female 3610 15Mean 26.65Male 117Range 13 to 82Female 10

Ohle et al. BMC Medicine 2011, age 6 of 13Table 1 Characteristics of included studies (Continued)Male 75Female bservationalstudyGP referral forSuspected acuteappendicitisMale:Median 29 yrsRange 3 to 85 yrsFemale:Median 34 yrsRange 2 to 86 yrsMale ight lower quadrantpain and suspected acuteappendicitisMean 8.5Range 3 to 16Male 43Female cted appendicitisMale 49Female 7935IranEmergencydepartmentProspectiveSuspected acuteappendicitisMale veRight iliac fossa pain andsuspected acuteappendicitis33USAMale pected acuteappendicitisSuspected acuteappendicitisMale 45Female 3078IndiaEmergencydepartmentProspectiveSuspected acuteappendicitisMean 22.6 Median 25Male 55Female 4562IndiaSurgicalWardProspectiveSuspected acuteappendicitis227Mean 20.47Range 10 to d acuteappendicitisStephens 1999[37]94Mean 44Range 3 to 79Male 150Females77Males 46Female 4889USATade 2007 [38]100Range 17 to 56Males 63Female 3734NigeriaEmergencydepartmentProspectiveWani 2007 [30]96Mean 25.46Range 7 to 70Male 48Female 4870IndiaSurgical unitProspectiveYildirim 2008[39]143Mean 34Range 18 to 76Male tudyAbdominal painWinn 2004 ed acuteappendicitisSubotic 2008[59]57Mean 27.5Range 16 to 70Male 27Female 3084SerbiaEmergencydepartmentProspectiveSuspected acuteappendicitisAndersson 2008[60]229Mean 23Males eSuspected acuteappendicitisPrabhudesai2008 [61]60Mean 25.4Male 27Female 3340UKEmergencydepartmentProspectiveSuspected acuteappendicitisOwen 1992 [11]215Petrosyan 2008[33]1,630Rezak 2011 [40]59Saidi 2000 [43]128Sanabria 2007[34]374Mean 29.5Range 15 to 71Schneider 2007[12]Shreef 2010 [41]588Median 11.9Range 3 to 21Mean 9.3Range 8 to 14Shrivastava2004 [57]100Singh 2008 [35]100Soomro 2008[36]350prediction across all risk strata: low risk (RR 5.35, 2.17to 13.19), intermediate risk (RR 1.82, 1.20 to 2.78) andhigh risk (RR 1.14, 1.04 to 1.25). In children, there wasa non-significant trend towards over-prediction in thelow risk strata (5.03, 0.52 to 48.82) and a significantover-prediction in the intermediate risk category (1.81,Surgical unit RetrospectiveAll patients whounderwentappendectomy forsuspected acuteappendicitisRight iliac fossa pain andsuspected acuteappendicitisSuspected acuteappendicitis1.13 to 2.89) and high risk strata (1.13, 1.01 to 1.27)(Figures 4, 5, 6). Heterogeneity in terms of betweenstudy predicted/observed risk ratio estimates is apparentin children across all risk strata and in women at a highrisk (I2 50%), and, therefore, these pooled estimatesshould be treated with caution.

Ohle et al. BMC Medicine 2011, age 7 of 13Table 2 Summary estimates of sensitivity and specificity calculated by a bivariate random-effects modelObservation/Admission(Cut point 5)Surgery(Cut point 7)StudiesnSensitivity(95% CI)Variance logit (sensitivity)Specificity (95% CI)Variance logit (specificity)All studies280.99 (0.97 to 0.99)3.370.43 (0.36 to 0.51)0.61Men50.96 (0.88 to 0.99)1.090.34 (0.24 to 0.47)0.06Women50.99 (0.92 to 0.99)2.120.35 (0.14 to 0.64)1.51Children*All studies9290.99 (0.83 to 1.00)0.82 (0.76 to 0.86)8.990.480.57 (0.41 to 0.72)0.81 (0.76 to 0.85)0.790.46Men60.88 (0.75 to 0.95)1.150.57 (0.40 to 0.73)0.44Women70.86 (0.78 to 0.92)0.440.73 (0.58 to 0.84)0.62Children*90.87 (0.76 to 0.93)0.980.76 (0.55 to 0.89)1.50* For the purpose of this study Children are defined as any participant under the age of 18 years of age.Figure 4 Low risk group (1 to 4): predicted versus observed cases with appendicitis in children, women and men.

Ohle et al. BMC Medicine 2011, age 8 of 13Figure 5 Intermediate risk group (5 to 6): predicted versus observed cases with appendicitis in children, women and men.In a subgroup analysis based on prevalence (Additional file 1 - Figure S2), the high prevalence categoryconsisted of six studies [9,10,23,32,37,39] - the scorepredicted well in this group and heterogeneity wasbelow 50% in the high and low risk groups (low risk RR0.65, 95% CI 0.25 to 1.75, I2 34%; intermediate riskRR 0.99, 95% CI 0.70 to 1.40, I 2 72%; high risk RR0.99, 95% CI 0.96 to 1.02, I2 0%). The low prevalencesubgroup consisted of 24 studies, there was a significantoverprediction across all risk strata; however, heterogeneity was extremely high (I2 78% to 85%) suggestingthat other factors, perhaps age and gender, contributedto the high levels of heterogeneity in this group. Unfortunately, not enough studies had age and gender information to allow us to do further subgroup analysis.all patient groups with suspected appendicitis. Pooleddiagnostic accuracy in terms of “ruling in” appendicitisat a cut-point of seven points is not sufficiently specificin any patient group to proceed directly to surgery. Interms of calibration, the observed, predicted estimates inmen suggest the score is well calibrated across all riskstrata. Application of the Alvarado score in womenover-predicts the probability of appendicitis across allstrata of risk and should be used with caution. Thevalidity of the Alvarado score in children was inconclusive; the calibration analysis showed high levels of heterogeneity across all risk strata. Further validationstudies are required before clinical implementation ofthe Alvarado score for this age group could berecommended.DiscussionClinical implicationsPrincipal findingsA recent clinical policy document from the AmericanCollege of Emergency Physicians reviewed the value ofusing clinical findings to guide decision making in acuteThis systematic review shows that the Alvarado score atthe cut point of 5 performs well as a “rule out” CPR in

Ohle et al. BMC Medicine 2011, age 9 of 13Figure 6 High risk group (7 to 10): predicted versus observed cases with appendicitis in children, women and men.appendicitis [7]. They state that combining various signsand symptoms, as in the Alvarado score, may be moreuseful in predicting the presence or absence of appendicitis. This systematic review supports the use of theAlvarado score as a triage CPR that can be applied to‘rule out’ appendicitis at a score below five points (sensitivity 94% to 99%), but not as a ‘rule in’ for appendicitis.Patients with a score less than 5 can be considered fordischarge with the proviso that watchful waiting and reassessment may be required if symptoms change ordeteriorate. The advantage of applying the Alvaradoscore in this way is that resources in terms of admittinga patient to hospital or performing diagnostic imagingcan be reserved for higher-risk scoring patients. Such anapproach may be particularly useful in low-resource settings where diagnostic testing is limited or not available[38].Based on the results of this review, the Alvarado scoreat a cut-off of five points compares favourably withother CPRs used in clinical practice. The Ottawa ankle

Ohle et al. BMC Medicine 2011, nd knee rules represent “rule out” CPRs of similarlyhigh sensitivity that are used in emergency departmentsto decide if a patient should be referred for radiographyto determine if their ankle or knee is fractured. Theapplication of these CPRs is to identify those patientswith a very low risk of fracture, where fracture can beconfidently ruled out and the patient can be dischargedwithout unnecessary imaging. For this purpose, it isimportant that such CPRs have high sensitivity. Metaanalysis of validation studies show these rules achievehigh sensitivity that is comparable to the Alvarado scoreat a cut-off of five points (ankle rule - 97.6% [48], kneerule - 98.5% [49] and Alvarado score at cut-off of fivepoints - 99%).The use of the Alvarado score as a ‘rule in’ CPR forsurgery at a cut point of 7 is not supported by our diagnostic test accuracy results. Our analysis indicates thatthe Alvarado score has moderate to high sensitivity (allstudies 82%, men 88%, women 86% and children 87%)and a moderate specificity (all studies 81%, men 57%,women 73% and children 76%), suggesting it is not sufficiently accurate to rule in or rule out surgery (Table 2).However, several studies report that the application ofAlvarado score as a sole decision criterion for surgery(cut point of 7) produces negative appendectomy ratesof 13.3%, 15.6%, 16.2% and 14.3%, respectively, withoutan increase in perforations [11,20,29,35]. This is comparable with a clinician’s judgment in other reports(17.1%, 12%, 12.5% and 11%) [5,8,19,27]. An Alvaradoscore 7 is useful at identifying those at high risk ofacute appendicitis who require a surgical consultation orfurther diagnostic imaging, it should not be used as thesole criterion for ruling in surgery in any patient group.During the last 10 years, the diagnostic imaging by CTscan in the diagnosis of appendicitis has become a common practice. In some centres over 90% of the patientspresenting with suspected appendicitis undergo CT imaging. CT has a high sensitivity and specificity for thediagnosis of appendicitis and it considerably reduces thelevel of negative appendectomy. However, some studieshave shown that the use of CT does not necessarilychange the clinical management of a patient, especiallyin those at high risk [33,50]. CT imaging may also delaythe time of operation and, therefore, may increase thesubsequent risk of perforation [51]. Assessing the use ofthe Alvarado score and CT imaging as a series of diagnostic investigations on all these types of outcomes iswarranted.Lastly, the results of this systematic review haveimportant implications in low-resource countries. First,in low-resource settings where the decision to operatemay be based on a clinical judgment, the Alvarado scoreprovides an accurate and consistent triage tool for rulingout appendicitis and identifying those at higher risk whoPage 10 of 13would benefit at most from an admission to a hospital.Second, the Alvarado score could serve as a simplifiedtool for the emergency physician in order to stratifypatients for referral for surgical consultation.Context of other researchAlthough the Alvarado score was developed in a mixedgender population, the ratio of men to woman was 1.4:1and the s

two cut-points commonly used for decision making, and calibration of the score). As studies have suggested that the accuracy of the Alvarado is affected by gender and age [8-12], we focused our analysis on three separate groups of patients: men, women and children. Methods Data sources and search strategy An electronic search was performed on .