WIXLIB

NM Evaluation Lab @ UNM7activities to aid infants, toddlers, and their parents and families. One of the initialengagement activities of the CIP and BH programs is an intake process, duringwhich clients are assessed and connected to the appropriate in-house services orexternal referral agencies or other community resources to best address the client’sneeds. At intake, LCCS collects demographic information, a client’sproblem/psychosocial history, builds a list of symptoms and prepares an initialtreatment plan for clients in the CIP and BH programs. Treatment activities mayinclude art therapy, play therapy, child-parent psychotherapy, DialecticalBehavioral Therapy (DBT), and Circle of Security Parenting. These activities areconducted at home or on-site, and include individual therapy, parent-child dyadtherapy, and parenting support groups. Throughout treatment services, data arecollected through multiple screens and surveys, including a pre/post clientsatisfaction survey, validated assessment measures, and client progress tracked byclinicians in an electronic medical record.The outputs of the programs include a treatment plan that focuses on evidencebased practices and measures for assessing goals and treatment completion. Thereare two types of output measures, count and index measures. Count measuresinclude the numbers of visits, clients served, discharges, scheduling times, and thenumber of clients on a waitlist. Index measures include, but are not limited to,indicators of depression, anxiety, social/emotional development, and adversechildhood experiences. Unlike outcome measures, output measures count what wasproduced through a program’s activities, they do not measure the impact or value ofa program’s services.Increased safety, parental confidence, and improved parent/child relationship areamong the three primary outcomes the CIP and BH programs seek to achieve.Safety outcomes include increased protection factors and safety in housing,transportation, and nutrition. Confidence outcomes include reduced parentaldepression and anxiety, increased ability to respond to emotional needs andmanagement of behaviors, and the ability to establish a support system betweenparent and child. Relationship outcomes include a stronger parent/childrelationship, better ability to address a mismatch in temperament, the developmentof coping mechanisms, and an improved level of comfortability in sharingparent/child moments (Ippen et al., 2011; Cohen, 2016).The expected impact is an improvement of children and parent’s emotional andsocial health. These impacts include an improvement in environmental conditionsand safety, connections to community resources, an increase in a parent’sunderstanding of their child and the ability to address the child’s needs, andincreased confidence as a parent or caregiver. Another expected impact is growth inthe ability to nurture a secure and healthy parent/child relationship.7

Las Cumbres Community Services Evaluation Plan 8The logic model will guide the focus of the evaluation, investigating what data isbeing captured in EMR-Bear (activities), how well EMR-Bear is being used byLCCS clinicians and staff (activities), that leads to generating reports (outputs), toassess whether LCCS services meet the goals and expectations of the programs theyprovide (outcomes).4. Evaluation PlanThis is the second year LCCS has worked with the Evaluation Lab. In its first year,the Evaluation Lab Team deduced that LCCS clinicians were not consistent in theirdata collection. This discovery led to a focus group where clinicians disclosed thatthey were not sure where, on paper or in the EMR system, and with what tool theyshould track certain aspects of client development and growth. Building on thefindings from the previous year, the scope of work includes two evaluation projects:1) Generate reports of EMR-Bear data, understand what is and is notcaptured by EMR-Bear, and create a template of measures based on datacaptured in EMR-Bear.2) Develop instruments to evaluate how well staff and clinicians utilize EMRBear.1) The first project is based on the following question: What reports, tracking, andinformation are captured in EMR-Bear for process indicators that measure programoutcomes?LCCS would like to measure how the CIP and BH programs are performing. Thiswill include an overview of current process indicators (see Appendix D), and areview of client outcome measures. The following activities will support thisproject: Running blank reports from EMR-Bear to:o Generate a list of what data/reporting measures are availableo Match data fields with process indicator listo Describe the process on how to pull periodic reportsWorking with the Evaluation Coordinator to pull at least two decoded reportsof client level data (SMART objectives and treatment goals).Drafting a flow diagram of the “gold standard” of how/when data should becollected for each client throughout their interaction with the organization;and sketch a visual diagram of how the data collection instruments arerelated to one another.Pull at reports of data that match process indicator list; observe real-timedata entry, if feasible.8

NM Evaluation Lab @ UNM 9Drafting a report on findings to include:o An inventory of reports and data fieldso An assessment of what fields are consistently being captured andwhich are noto What information and questions was generated by the sets of pulledreportso Recommendations on how to improve the data collection process andon which process indicators are best captured/measured by EMR-Bearo Revise current indicator list.2) The second project is based on the following question: How well is the EMR-Bearsystem utilized by the CIP and BH programs to track client outcomes and goals?This is an initial stage of the second project, since LCCS staff and EMR-Beardevelopers are currently working to develop EMR-Bear data entry fields. None-theless, understanding how clinicians are using EMR-Bear, and why the software isnot used consistently by clinicians and across programs, is pivotal to this evaluationproject. EMR-Bear populated menus generate the majority of quantitative dataavailable for evaluation. However, clinicians do not regularly use these populatedmenus. This information will lead to better training and use of the software,generating consistent client data, and SMART outcomes and goals. Activities tosupport this initial stage of the second project include: Tracking EMR-Bear training outcomes for clinicians to become competentusers of EMR-Bear through a pre and post survey to be developed by theEvaluation Lab Team. The following training outcomes will be used:o TO BE DETERMINED/RESEARCHEDTracking clinician use of EMR-Bear populated menus and documenting thesuccesses and challenges of using the software through surveys developed bythe Eval Team and through data checks from pulled reports.o Observe real-time data entry in EMR-Bear.5. TimelineThe evaluation activities will continue from October 2016 through March 2017, withreport presentations and revisions April to May 2017.October Submit draft evaluation plan to LCCS for feedback and suggestionsFinalize Logic Model9

Las Cumbres Community Services Evaluation Plan 10November Finalize Evaluation PlanRun preliminary reports of EMR-BearDecember Meet and discuss missing data and information in EMR-BearRun reports from EMR-Bear, determine how and where they are available (Clientlevel data and Process Indicators)January Continue to run reports from EMR-Bear (Client level data and ProcessIndicators)Construct matrix of process indicators available in current reports.Develop survey for clinicians focused on their training of EMR-BearDraft a flow diagram of the “gold standard” of how/when data should be collectedfor each client throughout their interaction with the organization.Sketch a visual diagram of how the data collection instruments are related to oneanother.Revise list of process indicators.February Revisions of survey for clinicians on their experience of using EMR-BearRevise matrix, flow diagram, and flow chart.Observe real-time data entry.Begin drafting final evaluation report.MarchDraft evaluation report to organization to include report on data and measuresand results from surveys given to clinicians Organization receives report March 31st. April Meeting with LCCS to present and discuss evaluation reportEvaluation report revisionsEvaluation Poster PresentationMay Final evaluation report with organization comments finalized, May 1110

NM Evaluation Lab @ UNM11ReferencesCohen, S. (2016). Applying the Science of Child Development in Child WelfareSystems. Center on the Developing Child at Harvard elitti V., Anda, R., Nordenberg, D., Williamson, D., Spitz, A., Edwards, V., Koss,M., & Marks, J. (1998). Relationship of childhood abuse and householddysfunction on many of the leading causes of death in adults. The AdverseChildhood Experiences (ACE) Study. American Journal of PreventiveMedicine, 14(4), 245-258.Francis, D. (2016, December 7). Poverty and Mistreatment of Children go Hand inHand. The National Bureau of Economic Research. Retrieved en, C., Harris, W., Van Horn, P., & Lieberman, A. (2011). Traumatic andstressful events in early childhood: Can treatment help those at highest risk?.Child Abuse & Neglect, 35, 504-513.Shonkoff, J., & Fisher, P. (2013). Rethinking evidence-based practice and twogeneration programs to create the future of early childhood policy.Development and Psychopathology, 25, 1635-1653.11

Las Cumbres Community Services Evaluation Plan 12Appendix AFigure A-1: Type of Maltreatment Victims, 2014Source: U.S. Department of Health & Human Services, Children’s Bureau; Child MaltreatmentReport 2014Relative to four types of maltreatment, there is variation between doing better andworse in New Mexico compared to the United States. In 2014, the types ofmaltreatment of reported child victims show varying contexts of neglect and abuse.The percent of child victims in 2014 for neglect was higher in New Mexico (82.4%)compared to the United States (75%). The percent of psychological maltreatment ofchildren in New Mexico is 21.8%, well above the national percentage, which is 6%.New Mexico has lower percentages of physical abuse (13.4%) relative to the nationalpercentage (17%), and New Mexico has lower rates of child sexual abuse (3.3%)compared to the United States overall(8.3%). This variation may speak to theextreme poverty seen in the state, which is more often associated with neglect thanphysical abuse (Francis, 2016).12

NM Evaluation Lab @ UNM13Figure A-2: First-time Child Victims by YearSource: U.S. Department of Health & Human Services, Children’s Bureau; Child MaltreatmentReport 2014The first-time child victim rate of maltreatment per 1,000 children is relativelystable nationally from 2011 with 6.8 per 1,000 children to 2014 with 7.0 in 2014 per1,000 children. The first-time child victim rate for New Mexico, however, hassteadily increased from 2011 to 2014, increasing from a rate of 8.1 per 1,000children in 2011 to a rate of 11.3 in 2014.2This is due, in part, to a greater awareness of child abuse and an increase in the number ofincidents reported to the authorities. See Terrell, S. (11 April 2014) Child abuse reports in state areon the rise. Santa Fe New Mexican. Retrieved from ild-abuse-reports-in-state-are-on-the-rise/article dbe83470-113e-5b23-8c64fe90bdf4355b.html.213

Las Cumbres Community Services Evaluation Plan 14Appendix B14

NM Evaluation Lab @ UNM15Appendix CThis literature review section focuses on elements that develop LCCS’s theory ofchange. The primary expectation of the CIP and BH programs is to develop andpromote a healthy emotional and social well-being of children who experiencetrauma, as well as establish parental capabilities in establishing healthyrelationships with their child. Scholars have studied how certain childhoodexperiences negatively influence the lives of children, which affect their social andemotional well-being from childhood all the way to adulthood. There is congruencebetween scholars and the purpose of LCCS’s CIP and BH programs that links theimportance of early treatment related to trauma and stressful events in order toensure that every child gets a chance to develop behaviors for a healthy, successfullife.The science of child development in child welfare system.Scholars at the Center on the Developing Child at Harvard University take a broadand basic approach to why investing in treating children with early childhoodtrauma is a concern for child welfare systems. Cohen (2016) argues that in order toimprove child development, one must understand how the human brain develops,and how it influences the mental, emotional, and social development of children.This article raises the question of “how can we use insights from cutting-edgescience to improve the well-being and long-term life prospects of the mostvulnerable children in our society.” Cohen (2016) recommends that child welfarepolicy and practice should recognize the needs of infants and toddlers forestablishing healthy behaviors and learning since the brain develops the most inearly childhood (p.15).Cohen does a great job of explaining basic concepts, yet, highlights important termssuch as neglect and self-regulation from a scientific viewpoint. These key terms arecrucial to understanding the science behind the consequences of child developmentunder the strain of constant stress and trauma. His explanation centers on theinterruption of brain architecture due to traumatic early experiences that impedethe foundation for healthy learning, behavioral development, and healthy growth.Experiences play a major role in children’s development and negative ones tend toleave a lasting impact on brain development.Another aspect Cohen emphasizes is the responsive and reciprocal interactionsbetween children and their caregivers. However, building responsive relationshipsis difficult under external circumstances of trauma and environmental stressors(i.e., dangerous housing, insufficient food). These circumstances cause stress amongadults, thus, children sense and experience the stress that adults are going throughand then children develop stress because of the reciprocal interaction between childand caregiver. Consequently, the relationships between parents and childrenweaken and both can develop mental and physical health challenges (Cohen, 2016).15

Las Cumbres Community Services Evaluation Plan 16Cohen strongly advocates for early treatment and prevention. He advisespolicymakers to invest in programs that promote better mental and physical healthoutcomes among children and their caregivers. He offers three main solutions: 1)Reduce external sources of stresses (such as programs that relieve financialburden), 2) develop responsive relationships (promote positive interactions betweenchildren and their caregivers), and 3) strengthen core life skills (promote programsthat help adults reach their goals) (Cohen, 2016). These are practicalrecommendations that not only benefit the children but their caregivers as well, andif the two parties receive appropriate care, a healthy and harmonious relationship islikely, which, as shown by research, is essential to the development of the child.The ACE studyMany scholars who study early childhood development and the variation ofdevelopment among children turn to the Adverse Childhood Experiences Study as abase for addressing early adverse experience in children before unhealthy behaviorsdevelop. In their article, Felitti et al. (1998) find an association between childhoodabuse and household dysfunction as a leading cause of unhealthy behaviors inadults that lead to early death. They conducted a study in which they seek to see ifadults who are exposed to traumatic events early on in their lives are likely to havechallenges as adults with their physical health and mental behavior. Adults whoexperience adverse childhood experiences tend to adopt behaviors, such as smoking,drinking, substance abuse, overeating, and others, that have long-termconsequences that ultimately increase risks to their lives (Felitti et al., 1998; Cohen,2016). Those adults who experience four or more categories of adverse experiencesare four to twelve times more likely to develop unhealthy adult habits (Felitti et al.,1998; Cohen, 2016). These habits contribute to the development of diseases, such asheart and lung diseases. They recommend early treatment of people who wereexposed to ACEs as early as possible to prevent these long-term consequences.Traumatic and stressful events in early childhood and treatmentIppen, Harris, Van Horn, and Lieberman (2011) have a similar focus as Felitti et al.(1998), as they focus on how to treat those that have experienced traumatic eventsas a young child. Their approach is different because they reanalyze data from aprevious study on whether child-parent psychotherapy (CPP) is effective f

Felitti et al. shows that there is an association between adverse childhood experiences (ACEs) and unhealthy behaviors affecting physical and mental health in adulthood (1998). Traumatic stressors in childhood can lead to neurological processes that result in poor health. Therefore, strategies to address early childhood trauma and stress can