Introduction

Parental substance misuse appears to be a major contributing factor for child maltreatment and subsequently to the placement of children in out-of-home care. It is estimated that 36% of child victims had a caregiver with a drug abuse problem (U.S. DHHS, 2021), and parental alcohol or drug misuse was identified as a condition of removal for almost 42% of all children placed in out-of-home care (U.S. DHHS, AFCARS, 2022a).

Research focusing on substance abuse among caregivers has consistently documented various adverse outcomes and has demonstrated significant risks for their children’s health, well-being, and behavioral health problems. In particular, parental substance misuse has been shown to be associated with children’s short- and long-term physical and emotional health, cognitive development, and behavioral problems (Bountress & Chassin, 2015; Seay & Kohl, 2015; Smith et al., 2016). Further, substance misuse among parents has been shown to disrupt family stability, negatively affect children’s academic performance, and was linked to the elevated risk of various types of child maltreatment (Berger et al., 2010; Freisthler et al., 2017; Lowthian, 2022; Ryan & Huang, 2014; Staton-Tindall et al., 2013).

A large body of evidence also highlighted that families with substance misuse problems were more likely to get involved with the child welfare system, and once involved, were more likely to experience undesirable outcomes. For example, studies indicated that caregivers with substance misuse problems were at increased risk of substantiated child maltreatment, and entry into out-of-home care (Ghertner et al., 2018; Hafekost et al., 2017). In addition, parental substance misuse has been shown to be associated with lower reunification rates, higher foster care reentry rates, and loss of custody (Brook et al., 2010; Courtney & Hook, 2012; Grella et al., 2009).

Considering the number of child welfare involved parents with substance use problems and the negative outcomes including child removal associated with this issue, interventions that focus on service provision for child welfare involved parents with substance misuse issues has become a public priority.

The Family First Prevention Services Act (FFPSA) was passed into law on February 9, 2018, as part of the Bipartisan Budget Act of 2018 and has several provisions to enhance support for families to help children and youth remain at home, reduce the use of congregate care, and build the capacity of communities to support children and families (Family First Act, 2022). This act challenged states to redesign their child welfare systems, putting the focus on preventing children from entering foster care and when necessary, ensuring that children are cared for in the best, family-like settings when removal is necessary.

Program Description

The Family Intensive Treatment (FIT) team model was developed by a group of key behavioral health stakeholders in Florida to provide “intensive team-based, family-focused, comprehensive interventions targeting high-risk families with child welfare involvement due to parental substance use and co-occurring mental health disorders” (Florida Department of Children and Families, 2020, p. 4). The FIT model was first implemented in Florida in 2014 in consecutive phases across the state, and currently there are 28 FIT teams offered by 20 community-based behavioral health providers subcontracted through Managing Entities.

The goals of the FIT program include to: (a) provide early identification of at-risk families and immediate access to intensive substance use and co-occurring mental health treatment services for parents/caregivers in the child welfare system with early engagement strategies, (b) promote engagement and retention in treatment, (c) support recovery, and (d) improve parenting capacity. This was expected to result in improvement of child safety, expedited permanency, and positive effect on child wellbeing. Several unique features distinguished the FIT model. First, the FIT model utilizes the intensive team-based approach that consists of multiple components including comprehensive assessments, individualized treatment planning, coordinated specialized services and linkages to community resources, multi-disciplinary treatment, and discharge planning. Second, FIT services involve cross-system collaboration between child welfare, judicial, and behavioral health systems and 24/7 access for crisis management. Third, the FIT model includes the availability of case management services to address the basic support needs of the family and coordinate the therapeutic aspects of services provided to all family members regardless of payer source. Fourth, the FIT model offers peer support services to promote recovery, engagement and retention in treatment. Fifth, mandatory assessments are conducted, and within two business days of completing initial assessments, FIT clients receive treatment services from a clinician. Within 30 calendar days of enrollment the FIT team creates a comprehensive family care plan. This care plan includes: (a) the alignment of clinical services received by children with clinical services received by parents, (b) identification of how support will be provided to parents/caregivers to address the child’s needs, (c) the alignment of the treatment plan for enrolled parents/caregivers to the child welfare case plan, and (d) a review of the care plan every 3 months or as needed. The FIT team model also mandates the provision of process updates to child welfare case management and the conduction of a multidisciplinary team (MDT) meeting prior to discharge from the treatment. The treatment completion is determined during the MDT meeting.

Eligibility criteria include: (a) a person has to meet criteria for a substance use disorder, (b) the family has to have at least one child between the ages of zero and ten; and (c) the child has to be determined “unsafe” as a result of the child protection investigation. Referrals for services can be made by the child welfare professional, including the child protective investigator, child welfare case manager, or community-based care lead agency. The average length of service is 4–9 months (DCF Office of Substance Abuse & Mental Health, 2022). Because the parent/caregiver can be court ordered to participate in FIT services, almost 100% of families referred to FIT services are typically enrolled.

Purpose of the Evaluation

The purpose of this evaluation was to determine the effectiveness of the FIT model with families involved in the child welfare system experiencing parental/caregiver substance use. Specifically, analyses were conducted to expand prior research completed by the University of South Florida (USF) research team (Robst et al., 2019) under contract with Florida Department of Children and Families and Casey Family Programs. Based on various administrative data sets including child welfare and substance and mental health data, this prior study employed a quasi-experimental design and demonstrated that participation in FIT was associated with greater increases in Caregiver Protective Capacities, reduction of child maltreatment re-reports, and increase in permanency rates. To ensure that an intervention has a beneficial long-term effect on child welfare involved caregivers, this evaluation was designed to include more recent data, additional cohorts of caregivers, and address evaluation questions relevant to the child welfare system goals (The Adoption and Safe Families Act of 1997 [ASFA]). Specifically, the evaluation examined outcomes related to child safety, permanency, and family well-being.

Methods

Sample

Participants in the intervention group were child welfare involved caregivers with substance misuse issues who were enrolled in and received the FIT intervention in fiscal years 2016–17 through 2019–20 (N = 3025). At the time of the referral to FIT, these caregivers had at least one child between the ages of 0 and 10 years old who were determined to be “unsafe” as per Florida’s Child Welfare Practice Model and in need of child welfare case management. If two parents of the same child were enrolled in the FIT program, one parent/caregiver was randomly selected to avoid non-independence of observations. Caregivers were tracked for 6 and 12 months after admission to FIT. The comparison group included 2976 child welfare involved parents/caregivers who did not receive FIT, but otherwise were similar to the FIT participants.

Evaluation Design

A longitudinal quasi-experimental design with a two-group comparison using propensity score matching was used in this evaluation study. The two groups included the intervention group (i.e., FIT) and the comparison group (i.e., child welfare involved parents/caregivers who did not receive the FIT intervention). The study includes all of the families that began the FIT program even if they did not fully complete the program. Four successive cohorts were examined including cases investigated during state fiscal years 2016–17, 2017–18, 2018–19, and 2019–20. Because the FIT model was implemented in consecutive phases across the state, the potential participants in the comparison group included all child welfare involved parents/caregivers who were investigated in the counties where FIT was not implemented yet. Both FIT participants and child welfare involved parents/caregivers selected in the comparison group were followed up until June 30, 2021, to allow for a 12-month follow-up period.

The propensity score matching was used to control for initial differences across multiple background characteristics and baseline variables (Rosenbaum & Rubin, 1984). We followed the suggestion made by Rubin and Thomas (1996) and Rubin (1997) advising that all variables presumptively related to an outcome, even if weakly so, should be included in the equation. All variables selected to estimate the propensity score were selected based on findings from previous research. Studies have demonstrated that demographic characteristics of the caregiver, substance misuse and mental health problems in the family, and type of child maltreatment were associated with recurrence of maltreatment and re-referrals (Bae et al., 2007; Dakil et al., 2011; Fuller & Nieto, 2009; Jonson-Reid et al., 2010; Proctor et al., 2012). In addition, county variable was included to account for differences in a number of important indicators, such as median county level, per capita crime rate, and percent of children living in poverty. The aim of this matching was to control for observed differences in child welfare involved parental characteristics.

The propensity score was calculated using logistic regression to obtain the predicted probability of being in the intervention group (Rosenbaum & Rubin, 1984). As a result, each parent/caregiver in the database had an estimated probability of being in the intervention group (i.e., FIT). After the propensity score was calculated, cases were matched using greedy nearest neighbor matching technique, in which the propensity score in the comparison group closest to the propensity score in the intervention group (i.e., FIT) was selected (Dehejia & Wahba, 2002). After matching was completed, the intervention and the comparison groups were checked for balance on all parent/caregiver characteristics included in the calculation of propensity score. No significant differences between groups were found when the groups were examined on each of the covariates included in the propensity score.

Data Sources

Three data sources were utilized including (a) the Florida Safe Families Network (FSFN), (b) the Family Intensive Treatment (FIT) database, and (c) the Financial and Services Accountability Management System (FASAMS). The FIT database contains records for each parent/caregiver including the date of enrollment in the FIT program, the completion status, demographic characteristics, parental assessment and functioning, and substance use or mental health diagnosis. The FSFN database contains records for each child and the alleged perpetrator in the child welfare system, information about child maltreatment reports, parent/caregiver and child demographics, the findings of child protective investigations, dates of children’s entry into out-of-home care, caregiver protective capacity assessment, and dates of discharge. Finally, the FASAMS database contains service records and behavioral health diagnoses for individuals receiving Florida Department of Children and Families (DCF) funded mental health and substance abuse services.

Predictor Variables

The predictor variables or covariates included the parent’s/caregiver’s demographic characteristics and participation in the FIT program. A description of each one follows:

Participation in FIT Participation in the FIT program was defined as the enrollment of a person in the FIT program regardless of whether the parent/caregiver completed treatment or was disengaged from treatment. Participation in FIT was coded as 1 and the comparison group was coded as 0.

Parental demographic characteristics Demographic characteristics included gender, age, and race/ethnicity. Gender consisted of two categories—male and female. Age was a continuous variable measured at the time of enrollment in the FIT program for FIT participants or at the time when the first maltreatment report was received for those in the comparison group. Race/ethnicity included the following categories: White, Black, Multiracial, and Hispanic. Other categories were excluded from the analysis because there were very few cases available for obtaining stable and reliable estimates.

Measures (Outcomes)

A range of primary outcome measures were examined and reported in this study, including safety, permanency, and family functioning indicators. Safety indicators consisted of repeated child maltreatment reports, recurrence of verified maltreatment, and reentry into out-of-home care. Permanency indicators consisted of achieving permanent placement for the child and reunification with the original caregiver. Timeframes for child safety and permanency outcomes were selected and based on the Child and Family Services Reviews (CFSR) national data indicators (U.S. DHHS, 2022b). Other measures of family well-being include Caregiver Protective Capacities, Functional Assessment of Mental Health and Addiction (FAMHA) score, Adult Adolescent Parenting Inventory-2 (AAPI-2) score, and a Daily Living Activities measure, described in more detail below.

Child maltreatment re-reports within 6 months This indicator was based on entry cohorts, that is, all children who were brought in contact with the child welfare system and subsequently investigated for alleged child maltreatment. For the FIT group, child maltreatment re-report was defined as a subsequent investigated child maltreatment report within 6 months after the enrollment in the FIT program, regardless of the disposition. For the comparison group, child maltreatment re-report was defined as a second investigated child maltreatment report within 6 months of the initial report regardless of the disposition.

Child maltreatment re-reports within 12 months This indicator was based on entry cohorts, that is, all parents/caregivers who were reported and subsequently investigated for alleged child maltreatment. For the FIT group, child maltreatment re-report was defined as a subsequent investigated child maltreatment report within 12 months after the enrollment in the FIT program, regardless of the disposition. For the comparison group, child maltreatment re-report was defined as a second investigated child maltreatment report within 12 months of the initial report regardless of the disposition.

Recurrence of verified child maltreatment within 6 months This indicator was based on entry cohorts, that is, all parents/caregivers who were reported, subsequently investigated for alleged child maltreatment, and as a result of the child protection investigation, child maltreatment was found verified. For the FIT group, recurrence of maltreatment was defined as subsequent verified child maltreatment report within 6 months after the enrollment in the FIT program. For the comparison group, recurrence of maltreatment was defined as a second incident of verified maltreatment within 6 months of a child’s first verified maltreatment incident. Only children with “verified” maltreatment (i.e., when the protective investigation resulted in a verified finding of abuse, neglect, or threatened harm) were included in the analysis. The first and second episodes of maltreatment were selected based on the dates the reports of child maltreatment were received.

Recurrence of verified child maltreatment within 12 months This indicator was based on entry cohorts, that is, all parents/caregivers who were reported, subsequently investigated for alleged child maltreatment, and as a result of the child protection investigation, child maltreatment was found verified. For the FIT group, recurrence of maltreatment was defined as subsequent verified child maltreatment report within 12 months after the enrollment in the FIT program. For the comparison group, recurrence of maltreatment was defined as a second incident of verified maltreatment within 12 months of a child’s first verified maltreatment incident. Only children with “verified” maltreatment (i.e., when the protective investigation resulted in a verified finding of abuse, neglect, or threatened harm) were included in the analysis. The first and second episodes of maltreatment were selected based on the dates the reports of child maltreatment were received.

Reentry into out-of-home care This indicator was defined as reentry into out-of-home care within 12 months of their most recent discharge. This measure is based on the exit cohort. An exit cohort is defined as all children who exited out-of-home care during a certain time period as indicated by a Discharge Date in FSFN. Children were followed for 12 months from the date of discharge from out-of-home care to determine whether they were subsequently placed in out-of-home care as indicated by a new (second) Removal Date in FSFN.

Permanency This measure is based on an entry cohort, that is, all children who were placed in out-of-home care during a specific fiscal year as indicated by the “removal date” in FSFN. Children were followed for 12 months from the date of removal from home to determine whether they were discharged from out-of-home care, as indicated by Discharge Date in FSFN, and achieved permanency. Permanency is defined as discharge from out-of-home care to a permanent home for the following reasons: (a) reunification, that is, the return of a child who has been removed to the removal parent or other primary caretaker, (b) permanent guardianship (i.e., long-term custody or guardianship) with a relative or non-relative, and (c) adoption finalized, that is, when the court enters the verbal order finalizing the adoption.

Reunification with original caregivers This measure is based on an entry cohort. An entry cohort is defined as all children who were placed in out-of-home care during a given fiscal year and it is based on the date the child was removed from his/her home as indicated by a Removal Date in FSFN. Children were followed for 12 months from the date of removal from home to determine whether they were discharged from out-of-home care as indicated by Discharge Date in FSFN and achieved reunification, that is, the return of a child who has been removed to the removal parent or other primary caretaker.

Caregiver Protective Capacities (CPC) This measure was developed as part of the Safety Decision Making Methodology by Florida DCF in consultation with the National Resource Center for Child Protective Services (NRCCPS), ACTION for Child Protection, and the Children’s Research Center. The CPC assesses three capacity categories—behavioral, cognitive, and emotional, which are comprised of 19 specific capacities. The CPC utilizes a four-point scale ranging from “excellent” to “intensive support needed” in each capacity.

The total score on the CPC measure and each of the three capacity categories were included in the analysis as separate variables. The CPCs from the Functional Family Assessment (FFA) Ongoing that was closest to the FIT enrollment date were used as the baseline measure, while the FFA Ongoing closest to the discharge date was used as the ending measure. For parents in the comparison group, the first ongoing FFA was used as a baseline and the subsequent FFA was used as a post-test.

Functional Assessment of Mental Health and Addiction (FAMHA) score The FAMHA is a 44-item clinician-assessment tool designed to assess functioning in six domains: substance misuse and criminality, community living skills, interpersonal skills, mood, psychological state, and health and physical functioning (Anderson & Bellfield, 1999). Ratings range from 1 to 7 for each question with the total score equal to the sum of all ratings divided by 3.08. A higher score indicates greater functioning. The assessment was performed within 30 days of enrollment into the FIT program and at discharge. This measure was not administered to the comparison group.

Adult-Adolescent Parenting Inventory-2 (AAPI-2) score The AAPI-2 is a 40-question assessment tool designed to assess parenting and child-rearing attitudes (Bavolek & Keene, 2005). The AAPI-2 encompasses five different types of behaviors, including parental expectations, parental lack of empathy towards children’s needs, use of corporal punishment as a means of discipline, parent–child family responsibilities, and children’s power and independence. Higher scores indicate more optimal attitudes. The assessment was performed within 30 days of enrollment and at discharge. This measure was not administered to the comparison group.

Daily Living Activities (DLA) score The DLA-20 Functional Assessment is a comprehensive tool for behavioral health providers to measure their clients’ level of functioning in daily living activities that can be impacted by mental illness or disability (Scott & Presmanes, 2001). It assesses the current behavior in 20 activities of daily living considering 10 areas: health practices, household stability, communication, safety, managing time, nutrition, relationships, alcohol and drug use, sexual health and behavior, and personal care and hygiene. The behaviors are ranked by comparing them to qualifiers on a scale from 1 to 7 to determine areas of success as well as areas of concern. This measure was not administered to the comparison group.

Analytic Approach

Several analytic techniques were utilized. First, descriptive statistics were used to detect data input errors, outliers, missing data patterns, and to describe the distributions for each measured variable. Second, survival analysis, specifically the Kaplan–Meier procedure (Kaplan & Meier, 1958) was used to estimate the percent of individuals who experienced an outcome of interest at a certain time point. Third, to examine time to event outcomes, such as time to reunification, Cox regression analysis was used (Cox, 1972). Cox regression is a type of event history analysis that is used extensively in outcomes research because of its ability to simultaneously examine both the risk of an event occurring and potential deferential effects related to the timing of that event (Cox, 1972). The major advantage of using Cox proportional hazards modeling in this study is that it utilizes information about parents who experienced an event (e.g., recurrence of maltreatment) and those who did not experience the event of interest or did not have another child maltreatment report (i.e., censored observations). To facilitate model interpretation, hazard ratios were used to assess the magnitude of the effect of each predictor on time to the event of interest. Fourth, to examine the effect of FIT on continuous outcomes (e.g., total score on Caregiver Protective Capacity measure), multiple linear regression was used. Finally, to examine the difference between the mean score at pre-test and the mean score at post-test, a paired-samples t-test was utilized. All analyses were conducted using IBM SPSS Statistics (Version 28), a statistical software platform.

Findings

Participants

Descriptive statistics were used to examine the obtained sample and compare parent/caregiver characteristics for both groups. As shown in Table 1, these analyses revealed that the majority of the study sample were females (approximately 74%) and White (approximately 80%). The average age of the participants was approximately 32 years. Table 1 also presents the distribution of parent/caregiver characteristics at the time they were either enrolled in the FIT program or were brought in contact with the child welfare system for the first time during a specific fiscal year. A substantial proportion (48% for the intervention group and 42% for the comparison group) of these parents/caregivers were investigated by the child protection system for child neglect. Approximately 30% of the parents/caregivers in each group had a history of domestic violence. A substantial proportion of parents/caregivers in both groups used opioids (36% in the intervention group and 31% in the comparison group), followed by cocaine and stimulants (approximately 21%). In addition, mood disorder was the most prevalent mental health diagnosis, with one fourth of the sample having this diagnosis. Smaller proportions of parents/caregivers were investigated for sexual abuse (approximately 1.5%) or emotional abuse (approximately 1%) and were diagnosed with personality disorder (less than 1%).

Table 1 Descriptive statistics for FIT and comparison samples at baseline after propensity score matching
Full size table

Child maltreatment re-reports within 6 months Approximately 17% of parents/caregivers in the FIT intervention group and approximately 24% of parents/caregivers in the comparison group were reported for alleged child maltreatment for the second time within 6 months of the initial child maltreatment report. Both bivariate and multivariate Cox regression analyses were conducted to examine the effect of receiving FIT on the risk of child maltreatment re-reports within 6 months. The results of the analysis, where parent/caregiver demographic characteristics were included, indicated a significant effect of FIT (see Table 2). Parents/caregivers who received the FIT intervention were significantly less likely to have a subsequent child maltreatment report compared to their counterparts in the comparison group. Accordingly, participation in the FIT program reduced the hazard rate by 45%. No parent/caregiver demographic characteristics were associated with child maltreatment re-reports within 6 months.

Table 2 Cox regression results: factors associated with child maltreatment re-reports within 6 months and 12 months
Full size table

Child maltreatment re-reports within 12 months Approximately 27% of parents/caregivers in the FIT intervention group and approximately 35% of parents/caregivers in the comparison group were reported for alleged child maltreatment for the second time within the 12 months of the initial child maltreatment report. Both bivariate and multivariate Cox regression analyses were conducted to examine the effect of receiving FIT on the risk of child maltreatment re-reports within 12 months. The results of the analysis, where parent/caregiver demographic characteristics were included, indicated that there is a significant FIT effect. Parents/caregivers who received the FIT intervention were significantly less likely to have a subsequent child maltreatment report within 12 months of the initial event compared to their counterparts in the comparison group. Thus, parents/caregivers who did not participate in the FIT program increase the hazard rate of a second report within 12 months by 33%. When the effect of parent/caregiver demographic characteristics was examined, results indicated that parents/caregivers who were White were 50% more likely to have a subsequent child maltreatment report within 12 months after the initial child maltreatment investigation (see Table 2).

Recurrence of verified child maltreatment within 6 months Rates of recurrence of verified child maltreatment within 6 months after the initial verified maltreatment were similar for both groups. Approximately 4% of parents/caregivers in the intervention and comparison groups experienced recurrence of child maltreatment within 6 months of the initial incident. When the effect of receiving FIT on recurrence of maltreatment was examined, the results of the Cox regression analysis indicated no significant difference between the groups (see Table 3). No parent/caregiver demographic characteristics were associated with recurrence of verified child maltreatment within 6 months.

Table 3 Cox regression results: factors associated with recurrence of verified child maltreatment within 6 months and 12 months
Full size table

Recurrence of verified child maltreatment within 12 months Rates of recurrence of verified child maltreatment within 12 months after the initial verified maltreatment were similar for both groups. Approximately 6.9% of parents/caregivers in the intervention group and 6.4% in the comparison group experienced recurrence of child maltreatment within 12 months of the initial incident. When the effect of receiving FIT on recurrence of child maltreatment was examined, the results of the Cox regression analysis indicated no significant difference between the groups (see Table 3). No parent/caregiver demographic characteristics were associated with recurrence of verified child maltreatment within 12 months.

Reentry into out-of-home care Note that the sample for our analyses of reentry was restricted to only those who exited out-of-home care. We provided sample sizes for each group in the tables. There were 8.6% of parents/caregivers in the FIT group whose children reentered out-of-home care within 12 months after discharge from out-of-home placement. The proportion of children who reentered out-of-home care in the comparison group was smaller—4.6%. However, the results of the Cox regression analysis indicated that there is no significant difference between the groups (see Table 4). When the effect of parent/caregiver demographic characteristics was examined, age was found to be significantly associated with reentry into out-of-home care. That is, younger parents/caregivers were more likely to have children who were placed in out-of-home care after discharge. Specifically, an increase in year of age would decrease the hazard rate of re-entry by 3%.

Table 4 Cox regression results: factors associated with reentry into out-of-home care
Full size table

Permanency The sample for our analyses for permanency was restricted to only those children who entered out-of-home care. We provided sample sizes for each group in the tables. Children of parents/caregivers who received FIT achieved permanency faster and at a greater rate compared to their counterparts. There were 25.2% of parents/caregivers in the FIT group whose children exited out-of-home care for permanency reasons within 12 months after removal. The proportion of children who achieved permanency within 12 months after initial removal in the comparison group was smaller—22.5%. The results of the Cox regression analysis indicated that receiving FIT had a significant positive effect on achieving timely permanency. Findings have shown that even controlling for parent/caregiver demographic characteristics, participation in the FIT intervention increased the hazard ratio by 15% compared to the children of parents/caregivers who did not receive this intervention (see Table 5).

Table 5 Cox regression results: the effect of family intensive treatment (FIT) and demographic characteristics on achieving permanency and reunification with original caregiver within 12 months of the latest removal
Full size table

Reunification with original caregiver The sample for our analyses for reunification was restricted to only those children who entered out-of-home care. We provided sample sizes for each group in the tables. When the proportions of reunified children were compared between the FIT group and the comparison group, no significant difference was observed. There were 15.8% of children reunified whose parents/caregivers received the FIT intervention and 15.7% of children reunified whose parents/caregivers did not receive such intervention. Results of both bivariate and multivariate Cox regression analyses confirmed that there was no significant effect of FIT on reunification within 12 months of the latest removal (see Table 5).

Caregiver Protective Capacities Emotional Subscale To examine the association between receiving FIT and the score on the emotional subscale of the CPC measure, multiple linear regression was conducted. Because the score on the emotional subscale at pre-assessment was not included in the propensity score matching, this score was included in the model to control for potential initial differences between the groups. The results of multiple linear regression indicated that even controlling for parent/caregiver demographic characteristics and initial differences between the groups on the emotional subscale score, parents/caregivers in the FIT group have a higher score on the second assessment conducted on the date closest to the discharge date from the program (see Table 6).

Table 6 Summary of multiple regression analysis for the emotional, behavioral, and cognitive subscales score of the caregiver protective capacities measure among parents who received family intensive treatment (FIT)
Full size table

Caregiver Protective Capacities—Behavioral Subscale Although no statistically significant difference was found when the mean scores on the behavioral subscale of the CPC for the FIT group and comparison group were examined, the p value (i.e., p = 0.056) indicated that the effect of FIT approached statistical significance (see Table 6). Results also have shown that higher scores on the behavioral subscale at the baseline assessment were associated with higher scores at the subsequent assessment.

Caregiver Protective Capacities—Cognitive Subscale When the effect of receiving FIT on the cognitive subscale was examined, no significant effect of FIT was observed. As might be expected, higher scores on the cognitive subscale at the baseline assessment were associated with higher scores at the subsequent assessment (see Table 6).

Caregiver Protective Capacities—Total score When all caregiver demographic characteristics and the initial total score on CPC were included in the multiple linear regression analysis model, receiving FIT was significantly associated with the total score on CPC at post-assessment (see Table 7). A significant positive association between FIT and a higher total score on the CPC suggested that FIT participants significantly improved their overall caregiver protective capacities. Similar to the associations between FIT and the subscales of the CPC, a higher total score at baseline was associated with a higher total score on CPC at subsequent assessment, R2 = 0.49, F(7, 416) = 167.54, p = 0.001.

Table 7 Summary of multiple regression analysis for the total score of the caregiver protective capacities measure among parents who received family intensive treatment (FIT)
Full size table

Functional Assessment of Mental Health and Addiction (FAMHA) score Because this measure was administered to the FIT recipients only, the score on FAMHA at baseline was compared to the score on FAMHA at discharge. The results of the paired-samples t-test demonstrated a significant increase in the total score on FAMHA, indicating that caregivers’ functioning in multiple domains significantly improved over time, t(475) = − 10.1, p < 0.001.

Adult-Adolescent Parenting Inventory-2 (AAPI-2) score This measure was also administered only to parents/caregivers who received FIT. The total score at baseline was compared to the total score at the subsequent assessment. The results of the paired-samples t-test revealed a significant increase in the total score on AAPI-2, t(848) = − 8.85, p < 0.001 suggesting that parents/caregivers’ optimal attitudes including attitudes toward expectations of children, empathy towards children’s needs, use of corporal punishment as a means of discipline, parent–child roles, and children’s power and independence significantly improved over time.

Daily Living Activities (DLA) (DLA, 2001) This measure was administered to the FIT recipients only; therefore, the score on DLA at baseline was compared to the score on DLA at discharge. The results of the paired-samples t-test revealed a significant increase in the total score on DLA, t(816) =  − 15.57, p < 0.001, indicating that caregiver level of functioning in daily living activities improved over time.

Discussion

The main goals of this evaluation study were to examine outcomes for child welfare involved caregivers who received the FIT intervention and to possibly identify areas for program refinement. The results of this evaluation are mostly supportive of the FIT intervention. Findings based on the four cohorts (SFY 2016–17 through 2019–20) indicated that compared to a group of similar parents/caregivers receiving child welfare services, parents/caregivers who received FIT were less likely to have new allegations of maltreatment within 6 months and 12 months of the initial incident. Although according to Cohen’s original conventions the effect size was small, this value should be interpreted in the context of research and its practical or clinical value (Durlak, 2009). Considerations should be given to the type of outcomes assessed—some are more difficult to change than others, type of intervention, results of the previous studies, and clinical or practical significance (van der Put et al., 2018). Researchers have noted that individual-level interventions, interventions that focus on the outcomes that are hard to change (e.g., involvement in the child welfare related to substance misuse), and interventions that include a comparison group that had access to similar services typically have a small effect size (Kraft, 2020). In light of these suggestions and findings from previous studies of child welfare interventions that found overall small effect sizes (van der Put et al., 2018), the results of our study suggest that this model of service is a promising and valuable strategy to improve behavioral health related outcomes for child welfare involved caregivers and outcomes for their children.

Contrary to our expectations, findings indicated that there is no significant effect of FIT receipt on recurrence of verified maltreatment. Similarly, no significant differences were found when the rates for reentry were examined. One possible explanation for this finding is that often caregivers with verified maltreatment are involved with multiple systems, and multiple efforts to provide services to these individuals may confound the effect of FIT. Another explanation is that positive behavior change may be more challenging to observe among higher risk caregivers, those whose maltreatment was verified during the first incident. For example, a study by Connell et al. (2007) suggested that verified cases of physical abuse were more likely to reoccur compared to unsubstantiated cases or those involving neglect. It should also be noted that the rates of recurrence of verified maltreatment and reentry into out-of-home placement occur at low base rates (i.e., less than 10%) and therefore it is more difficult to demonstrate a change in these outcomes. Although a considerable advantage of FIT is the simultaneous provision of substance abuse treatment and parenting interventions, a program modification that includes a careful assessment of the severity of client behavioral health and other client problems before enrollment into the program may further enhance participant outcomes. Assessments based on standardized measures would allow for identifying the most at risk caregivers and tailoring service provision based on the type of parental mental health difficulty(ies) and the level of severity. Ultimately, there is reason to hope that tailoring both the type and the intensity of services will have a positive impact on program effectiveness.

This evaluation study also has shown that receiving FIT was associated with better permanency outcomes. Although no significant effect of FIT was found on timely reunification, children, whose parents/caregivers participated in the FIT program, were more likely to achieve other forms of timely permanency outcomes, such as guardianship or placement with relatives. One plausible explanation for this finding is that parents with substance misuse problems typically have complex needs. These parents, therefore, often have a substantially greater number of case plan requirements and a much longer (i.e., longer than 12 months) treatment and recovery period (D'Andrade & Chambers, 2012).

Consistent with the previous evaluation of the FIT intervention (Robst et al., 2019), this evaluation has shown that receiving FIT has a significant effect on improvement of caregiver protective capacities. More specifically, participation in the FIT program predicted improvement in parental emotional protective capacity and overall protective capacity and showed a positive tendency in improvement of parental behaviors related to their protective role. Similarly, the assessments that were conducted only on individuals who received the FIT intervention, including FAMHA, DLA, and AAPI, parents/caregivers demonstrated significant improvements over time in all assessed domains. Overall, this study lends support for the effectiveness of the FIT intervention. Although the effect of the FIT program on different outcomes varies, receiving FIT does seem to help families in a wide range of issues and substantially reduces the risk for further adversity among child welfare involved parents with substance misuse issues.

Limitations of the study should be noted. First, the study relies on administrative data. Therefore, validity of the records and reliability of reporting across agencies are limited by the quality and consistency of the data entry. Second, this study was limited by the use of outcome measures available via administrative data sets and the use of measures for which psychometric properties were not yet examined. For example, psychometric validation data to establish the factor structure and test–retest reliability of the CPC assessment has not yet been conducted. Third, a quasi-experimental design was utilized, and while it allows for controlling a great number of parent/caregiver characteristics, in contrast to an experimental design with random assignment to the intervention and the comparison group, it does not control for unobservable characteristics that potentially can affect the outcomes. Fourth, the findings do not account for the effects of the FIT intervention on various subgroups of parents/caregivers. There may be discrete, identifiable subgroups of individuals who received FIT and for whom the participation in the FIT program has a differential effect. In addition, not all parents could be found in the FASAMS data set and therefore, information about these parents’ mental health and substance use diagnoses were not available. Finally, the findings from this study may not generalize beyond Florida due to the unique child welfare system, in which provision of child protection services was transferred to multiple community-based agencies. Given the uniqueness of Florida’s child welfare system, it will be important to examine an intervention similar to FIT in other states.

Study Implications

First, the results suggest that providing additional supports and services for child welfare involved caregivers with substance misuse issues may help them both with behavior change and their functional improvement that subsequently contribute to further prevention of involvement with the child protection system. Second, because findings have shown that participation in FIT did not affect recurrence of verified maltreatment and reentry into out-of-home care, this suggests that the program might be modified to include specific interventions that focus on caregivers with higher level of maltreatment severity. These caregivers often have multiple co-occurring problems that consequently result in multiple maltreatment types and higher level of maltreatment severity (Lin et al., 2020). In this situation, child safety and other outcomes would need to be closely monitored. Although treating substance misuse problems is critical, addressing caregivers’ co-occurring needs and focusing on caregiver engagement and treatment completion may reduce chronicity for these higher risk parents/caregivers (Degarmo et al., 2013). Because expediting reunification for caregivers with substance misuse issues may be challenging, one possible strategy to shorten child stay in care is to initiate a concurrent planning when caseworkers are considering multiple options for permanency such as guardianship or placement with relatives at the same time and utilize reunification with the original family at a future date.

Conclusion

The findings from this study support the importance of providing intensive services for child welfare involved families experiencing substance use issues. It is also important to place an increased emphasis on keeping families engaged in services to ensure service completion. Keeping parents/caregivers with substance misuse issues involved in treatment may help improve family functioning and child welfare outcomes. Overall, given the overwhelming evidence of case severity when child maltreatment is combined with substance abuse issues, FIT and other parent/caregiver support services play a major role in both child safety and family well-being.