Transcription
MIDDLESEX-LONDON HEALTH UNIT – Evaluation of tykeTALK Phase 2: Family Satisfaction Survey ReportEvaluation of tykeTALK:Phase2Family Satisfaction Survey ReportOctober 2005For information, please contactJames Madden, MScProgram Evaluator, Research, Education,Evaluation & Development ServicesMiddlesex-London Health Unit50 King StreetLondon, OntarioN6A 5L7phone: 519-663-5317fax: 519-432-9430e-mail: jim.madden@mlhu.on.ca
MIDDLESEX-LONDON HEALTH UNIT – Evaluation of tykeTALK Phase 2: Family Satisfaction Survey Report Copyright 2005Middlesex-London Health Unit50 King StreetLondon, OntarioN6A 5L7Cite reference as: Middlesex-London Health Unit (2005). Evaluation of tykeTALK: Phase 2: Family SatisfactionSurvey Report. London, Ontario: Author.Author:James Madden, MSc, Program EvaluatorAll rights reserved.
MIDDLESEX-LONDON HEALTH UNIT – Evaluation of tykeTALK Phase 2: Family Satisfaction Survey ReportTable of ContentsAcknowledgements.iExecutive Summary .1Purpose of the Survey . 1Method . 1Profile of Respondents . 1Respondents’ Overall Assessment of Change Due to tykeTALK Services. 1Convenience of tykeTALK Services . 2Satisfaction with the Intake Process. 2Experience with the Assessment Process. 2Respondents’ Evaluations of Five Specific tykeTALK Interventions . 2Extent of Experience with Interventions . 2Perceptions of Impact of Interventions. 2Group versus Individual Therapy . 3Effect of Number of Sessions on Perception of Impact . 3Parental Involvement in Child’s Care. 3Conclusion and Recommendations . 4Introduction.5Purpose of Phase 2 . 5Prior Steps and Products in the Evaluation Process . 5Program Logic Model . 5Phase 1: SLP Focus Groups. 5Let’s Grow Survey . 6Overview and Organization of this Report. 6Methods .7Survey Design. 7Questionnaire Development . 7Pilot. 7Sampling Procedures . 7Inclusion/Exclusion Criteria. 7Sampling Frame . 7Drawing the Sampling . 7Questionnaire Distribution, Return Procedures and Return Rate . 8Data Entry. .8Data Analysis . 8Report Writing . 8Profile of Respondents .9Agency Providing Service. 9Is Child Currently Receiving Services? . 9Reason No Longer or Not Currently Receiving Service. 9Duration of Service (in Months). 10Age of Child at Time of Survey . 10Sex of Child Receiving Service . 10Respondents’ Sex. 11Respondents’ Relationship to Child Receiving Service. 11Respondents’ Age. 11Respondents’ Marital Status . 11Number of Children ( 19-Years-Old) Living at Home . 11Highest Level of Education Completed . 11Comparison of tykeTALK Sample with General Population . 11Respondents’ Employment Status. 12Status of Those Not Employed . 12Part-time versus Full-time Workers. 12Job Classification of Those Employed. 13
MIDDLESEX-LONDON HEALTH UNIT – Evaluation of tykeTALK Phase 2: Family Satisfaction Survey ReportLanguage and Citizenship Status . 13Respondents’ First Language. 13Language Spoken at Home . 13Respondents’ Ability to Read and Write English or French . 14Respondents’ Citizenship Status . 14Years Living in Canada. 14Income . 14Section A: Respondents’ Overall Assessment of Change Due to tykeTALK Services.17Purpose of the Section . 17Findings . 17Conclusions About Respondents Overall Assessment of Change Due to tykeTALK . 18Section B: Convenience of tykeTALK Services.19Purpose of this Section . 19Findings . 19Conclusions About Convenience of tykeTALK Services . 20Section C: Getting Connected to tykeTALK .21Purpose of this Section . 21Findings . 21Satisfaction with the Warmline . 21Conclusions with the Intake Process . 22Section D: Experience with the Assessment Process.23Purpose of this Section . 23Findings . 23Conclusions about the Assessment Process . 24Section E: Respondents’ Evaluations of Five tykeTALK Interventions .25Purpose of this Section . 25Percentage of Respondents that Received Each Type of Interventions . 25Number of Sessions Attended (Parent Group, Individual and Group Therapy). 26Comparative Ratings of Each of Five Types of Intervention . 26Perception of Impact of Individual versus Group Therapy . 27Comparison of Means Analysis . 29Impact of the Number of Sessions on Improvement Ratings . 31Conclusion About Impact of Number of Sessions on Improvement Ratings . 32Assessment of Overall Impact and Overall Satisfaction with Service . 32Conclusions about Respondents’ Perceptions of Impact of tykeTALK Interventions . 34Section F: Parental Involvement in Child’s Care .35Purpose of this Section . 35Measure of Processes of Care . 35Scores on the Individual Items . 35MPOC Scale Scores. 37Differences between Agencies in MPOC Scale Scores . 37Differences in MPOC Scale Scores Based on Type of Intervention . 37Comparison of tykeTALK MPOC Scale Scores with Ontario-Wide Scores. 38Conclusions About Parental Involvement . 38Conclusions and Recommendations .39References .41AppendicesAppendix A–Program Logic Model. 43Appendix B–Excerpt from Evaluation of the Tri-County Let’s Grow Program, Phase 3, Summary Report. 47Appendix C-Questionnaire . 49
MIDDLESEX-LONDON HEALTH UNIT – Evaluation of tykeTALK Phase 2: Family Satisfaction Survey ReportCharts & TablesCharts & FiguresMethodsChart 1: Months of Service Received.10Chart 2: Child’s Age at Time of Survey .10Chart 3: Comparison of Highest Level of Education tykeTALK Sample with Thames Valley .12Chart 4: Respondents’ Employment Status .12Chart 5: Comparison of Total Household Income tykeTALK with Thames Valley .15Section A: Respondents’ Overall Assessment of Change Due to tykeTALK ServicesFigure A.1: Respondents’ Assessment of Difference tykeTALK Made for Child’s Speech & Language.17Section C: Getting Connected to tykeTALKFigure C.1: What Happened First Time Called tykeTALK?.22Section E: Respondents’ Evaluation of Five InterventionsFigure E.1: Proportion of Sample receiving Individual and/or Group Therapy .26Figure E.2: Comparison of Improvement Ratings of Individual vs. Group Therapy .28Figure E.3: Improvement Rating on BOTH Individual AND Group Therapy .28Figure E.4: Improvement Ratings on Individual OR Group Therapy .29Figure E.5: Relationship Between Sample Subgroups & Unit of Analysis for Comparison of Means Analysis .30Figure E.6: Comparison of Mean “Improvement Rating” Scores for Group & Individual Therapy .30Figure E.7: Effect of Total Number of Sessions on Improvement Ratings .31Section F: Parental Involvement in Child’s CareFigure F.1: Comparison of Findings on 3 MPOC Scales .37Tables:IntroductionTable 1: Evaluation Plan . bleTable1: Return Rate Information . 82: Agency Providing Service. 93: Is Child Currently Receiving Service? . 94: If Not Currently Receiving Service . 105: Respondents’ Sex. 106: Respondent’s Relationship to Child Receiving Service. 117: Respondents’ Marital Status . 118: Number of Children ( 19 years old) Living at Home . 119: Highest Level of Education Completed. 1110: If Not Currently Employed, What is Status? . 1211: If Currently Employed, Full or Part Time? . 1212: Classification of Respondents’ Primary Paid Employment . 1313: First Language (Still Understood) . 1314: Other First Language Still Understood . 1315: Language Spoken at Home Most Often . 1316: Other Language Most Often Spoken at Home . 1417: Respondent’s Ability to Read and Write English or French. 1418: Respondent’s Citizenship Status . 1419: Years in Canada (Not Canadian by Birth) . 1420: Respondent’s Total Household Income Before Taxes. 15Section A: Respondents’ Overall Assessment of Change Due to tykeTALK ServicesTable A.1: Possible Changes Observed in Child Since Starting tykeTALK . 18
MIDDLESEX-LONDON HEALTH UNIT – Evaluation of tykeTALK Phase 2: Family Satisfaction Survey ReportSection B: Convenience of tykeTALK ServicesTable B.1: Respondent Ratings of Convenience of Service . 19Table B.2: Convenience of Parking by Agency . 19Table B.3: Other Possible convenience Related Problems . 20Section C: Getting Connected to tykeTALKTable C.1: How Respondent Found Out About tykeTALK. 21Table C.2: Respondents’ Assessment of the Intake Process . 22Section D: Experience with the Assessment ProcessTable D.1: Respondents’ Ratings of Aspects of Assessment Process . 23Table D.2: Understood What to Expect After Assessment by Agency . 23Table D.3: Overall Satisfaction Rating for Assessment Procedure. 24Section E: Respondents’ Evaluation of Five InterventionsTable E.1: Percentage of Respondents That Received Each Type of Interaction. 25Table E.2: Number of Sessions Attended . 26Table E.3: Ratings of Extent to Which Various Interventions Had a Positive Impact . 27Table E.4: Assessment of Overall Impact by Type of Intervention . 33Table E.5: Overall Satisfaction with Service by Type of Intervention . 33Table E.6: Extent to Which Needs Were Met by Type of Intervention . 34Table E.7: Whether Would Recommend tykeTALK by Type of Intervention . 34Section F: Parental Involvement in Child’s CareTable F.1: Frequency Distribution for 11 Parental Involvement Indicators, Grouped by MPOC Scale . 35Table F.2: Mean Scores 11 Parental Involvement Indicators, Grouped by MPOC Scales . 36Table F.3: Items with Lowest Score. 36Table F.4: Differences Between Agencies on Enabling and Partnership Scale . 37Table F.5: Difference in Enabling and Partnership Scale by Type of Intervention. 38Table F.6: Comparison of tykeTALK Respondents’ Scores on MPOC Scales with an Ontario-Wide Sample . 38
MIDDLESEX-LONDON HEALTH UNIT – Evaluation of tykeTALK Phase 2: Family Satisfaction Survey ReportAcknowledgementsThis report is the result of the collaborative effort of many individuals. Many thanks to the following people.Michelle Sangster Bouck, Research Associate, Research Education Evaluation and Development (REED) Services,Middlesex-London Health Unit (MLHU). In partnership with the report author, Michelle collaborated closely indeveloping the evaluation plan, conducting Phase 1 of the evaluation, and the development of the questionnaireused in this study.Members of the tykeTALK Evaluation Committee collaborated in the development of the evaluation plan.Committee members included: Debbie Shugar, System Facilitator, tykeTALK. Debbie provided an invaluable insider perspective,constructive critical feedback and support throughout the process. Deborah Almost, Director, Speech-Language Pathology Services, Woodstock General Hospital Joanne Dunne, Program Supervisor, All Kids Belong, Merrymount Children’s Centre. Deborah Maund, Speech and Language Pathologist, Thames Valley Children’s Centre Rosine Salazer, Past Chair of the Alliance Steering Committee and Learning Supervisor, School andCommunity Services, Thames Valley District School Board Sharon Buccione, Chair, Thames Valley Preschool Speech and Language Alliance Steering Committee.Sharon collaborated in planning sessions to present results to front-line service providers and the AllianceSteering Committee, and gave feedback and suggestions regarding data analysis and reporting of findings.Mandy Ho, Research Assistant, REED Services, MLHU. Mandy conscientiously performed the very large task ofimplementing questionnaire distribution and collection and data entry procedures.Piotr Wilk, Community Educator/Researcher, Public Health Research Education and Development (PHRED)Program, REED Services, MLHU. Peter provided consultation and support for analysis reported in Sections E andF.Members of the tykeTALK administrative staff: Diane Belanger, Intake Coordinator. Diane provided support in accessing and managing the database forsampling purposes and provided other administrative support. Gail Sheehan, Administrative Assistant Shirley Vanderploeg, Administrative AssistantStephanie Totten, Administrative Assistant, REED Services, MLHU. Stephanie assisted in the formatting of thereport.Nancy Forbes, Administrative Assistant, REED Services, MLHU. Nancy finalized the formatting of this report.Mr. Larry Stitt, Biostatistical Support Unit, University of Western Ontario. Mr. Larry Stitt provided consultation onthe sampling strategy.i
MIDDLESEX-LONDON HEALTH UNIT – Evaluation of tykeTALK Phase 2: Family Satisfaction Survey Reportii
MIDDLESEX-LONDON HEALTH UNIT – Evaluation of tykeTALK Phase 2: Family Satisfaction Survey ReportExecutive SummaryPurpose of the Surveyapproximately two thirds of the respondents receivingbetween 4 and 25 months of service.The purpose of the survey was to determine theextent to which parents/caregivers perceive thetykeTALK system is achieving the following intendedoutcomes: Provision of services at convenient times andlocations High family satisfaction with the intake process High family satisfaction with the assessmentprocess High family satisfaction with interventions High parental/caregiver involvement ininterventionsThe great majority of respondents (93%) were themothers of the children receiving service. Themajority of the children receiving service (68%) weremale. In terms of marital status, 86% of therespondents were married, 4% were single and 4%were divorced or legally separated.The majority of respondents (50%) indicated that theyhad completed trade school, college or university.Relatively few (5%) had less than a high schooleducation. A relatively high percentage ofrespondents (17%) had at least some post-graduateuniversity education. There were significantly fewerpeople with less than high school diploma in thetykeTALK sample (5%) compared with the generalpopulation in Elgin, Middlesex and Oxford counties(21%). The percentage that had completed highschool and/or had some post-secondary educationwas almost identical at 28% to 29%. A significantlygreater percentage of tykeTALK survey respondents(68%) had completed trade school, college oruniversity, compared with 51% among the generalpopulation in Elgin, Middlesex and Oxford.MethodA questionnaire was developed collaboratively bymembers of an evaluation committee made up ofrepresentatives from tykeTALK systemadministration, front-line Speech and LanguagePathologists, and evaluation researchers fromMiddlesex-London Health Unit. A stratified randomsampling procedure was conducted, in order to selecta representative sample of clients from each of thefive service-providing agencies in the system.Questionnaires were mailed to respondents with apostage-paid return envelope provided. Two follow-upmailings were conducted. The data were collectedbetween April 5th and July 20th, 2004. Based on anadjusted sample of 1153 clients, 609 questionnaireswere returned with usable data, yielding a return rateof 52.8%. Preliminary analysis of data wasconducted by the MLHU Program Evaluator, andpresented to members of the evaluation committee fordiscussion and feedback in order to inform furtheranalysis. A preliminary report was written andpresentations made to front-line service providersand members of the tykeTALK Alliance SteeringCommittee in order to obtain further feedback beforecompleting this report.In terms of citizenship status, 87% indicated theywere Canadian by birth and 9% indicated they werelanded immigrants. Of those not Canadian by birth(a total of 71 of 609 respondents) 16% had lived inCanada for 5 years or less. Ninety seven per cent(97%) of all respondents
MIDDLESEX-LONDON HEALTH UNIT - Evaluat ion of tykeTALK Phase 2: Family Satisfaction Survey Report Evaluation of tykeTALK: Phase2 Family Satisfaction Survey Report October 2005 For information, please contact James Madden, MSc Program Evaluator, Research, Education, Evaluation & Development Services Middlesex-London Health Unit 50 King Street .
