The review of the evidence below is limited to randomized control trials (RCTs) of Family Connects and Healthy Steps in local settings. Comprehensive screening and referral programs have not yet been studied as a statewide policy. Additional quasi-experimental studies of Healthy Steps have been conducted but are not included in this review. RCTs are the most methodologically rigorous study designs and allow for the strongest causal conclusions of the impact of a program on child and family outcomes, so these studies alone are considered in our assessment of the overall effectiveness of comprehensive screening and referral programs. Subgroup analyses and long-term findings from follow-up interviews with the original RCT families led to multiple publications from the same intervention. For the purposes of our assessment, studies that measure the impact of the same intervention on the same sample are treated as a single example of effectiveness, regardless of the number of distinct publications.
The research discussed here meets our standards of evidence for being methodologically strong and allowing for causal inference, unless otherwise noted. Each strong causal study reviewed has been assigned a letter, and a complete list of causal studies can be found at the end of this review, along with more details about our standards of evidence and review method. The findings from each strong causal study reviewed align with one of our eight policy goals from Table 1. The Evidence of Effectiveness table below displays the findings associated with comprehensive screening and referral programs (beneficial, null,i or detrimental) for each of the strong studies (A through I) in the causal studies reference list, as well as our conclusions about the overall impact on each studied policy goal. The assessment of the overall impact for each studied policy goal weighs the timing of publication and relative strength of each study, as well as the size and direction of all measured indicators.
Table 2: Evidence of Effectiveness for Comprehensive Screening and Referral Programs by Policy Goal
|Policy Goal||Indicator||Beneficial Impacts||Null Impacts||Detrimental Impacts||Overall Impact on Goal|
|Access to Needed Services||Knowledge of Community Resources||F||Positive|
|Use of Community Resources||B, D|
|Parental Health and Emotional Wellbeing||Completion of Recommended Postpartum Visits||D||Mixed|
|Maternal Depression||B, D|
|Maternal Substance Use||B|
|Emergency Department Visits||D|
|Nurturing and Responsive Child-Parent Relationships||Positive Parenting Behaviors and Caregiving||B, D, E, F, G, I||Mostly Null|
|Discipline Practices||H||F, G, H^|
|Home Environment Quality||B|
|Warmth and Nurturance||I||I^|
|Child Attachment Scores||I|
|Father-Infant Relationship Quality||D|
|Nurturing and Responsive Child Care in Safe Settings||Nonparental Care Use||B+, D||Mostly Null|
|Optimal Child Health and Development||Total Infant Emergency Care Use||A*, B, C**||D, F, G||Mixed|
|Emergency Department Visits for Accidents, Injuries, or Maltreatment||C|
|Child Protective Services Investigations||D|
|Timely Pediatric Appointments||F||D|
|Behavior Problems/Social Skills||G, I||F|
^Impacts on discipline were not consistent across assessment points.
+Among the small sample of those who did use nonparental care, quality of care was higher.
*Larger effects among “nonminority” and Medicaid insured families
**Treatment effect significant for “nonminority” families only
–Significant impact on safe sleep practices only
Access to Needed Services
Findings from three RCTs show that comprehensive screening and referral programs have a positive impact on connecting families to needed community resources, though the effect sizes are relatively small. The original RCT of Family Connects found that six months after the intervention, treatment families had accessed 0.9 more community resources,B and a recently published second RCT similarly found 0.7 more total community connections at six months among treatment families.D Healthy Steps has also been shown to positively impact referrals to needed services, with 3.5 higher odds of being informed about community resources among families who received the intervention programming.F
Parental Health and Emotional Wellbeing
Family Connects has been shown to have mixed impacts on maternal mental and physical health. Participating mothers had lower odds (0.65 OR) of reporting clinical anxiety, but statistically insignificant differences in the odds of depression or substance use.B More recent RCT findings similarly showed a null impact on maternal depression and completion of recommended postpartum visits, as well as a small but significant increase of 0.21 more emergency department visits among Family Connects mothers.D
Nurturing and Responsive Child-Parent Relationships
Overall, comprehensive screening and referral programs have not been shown to have a significant impact on parenting behaviors or child-parent relationships. Each of the three main RCT evaluations found a null impact of program participation on positive parenting behaviors, such as following routines or regularly reading to one’s child. Follow-up studies of Healthy Steps participant families at child ages 3 and 5.5 years showed no significant impact on parent responses to child misbehavior.F,G Additional observations of a subsample of Healthy Steps families concluded that participants had higher scores on inductive (positive) discipline techniques at child age 16 to 18 months, but the effect diminished by child age 34 to 37 months; no significant differences in punitive discipline techniques were found.H Subgroup analyses showed that these beneficial impacts on inductive discipline were only significant among White mothers and mothers with incomes under 200 percent of the federal poverty level, relative to their respective control counterparts.H Maternal warmth and nurturance was no different at child age 16 to 18 months, though NCASTii scores were 1.53 points higher on a scale of 73 possible points among treatment families at child age 34 to 37 months, likely due in part to skewed sample attrition issues.I Contrary to these null findings, the six month follow-up study of Family Connects did find a small but statistically significant increase of 0.21 points on an 18-point home environment quality scaleiii among treatment families.B
Nurturing and Responsive Child Care in Safe Settings
Both the original and more recent RCT of Family Connects found no statistically significant impact of program participation on use of nonparental care.B,D However, among those parents that did use nonparental care, out-of-home care quality was rated 0.66 points higher on a 5-point rating scale compared to control families.B
Optimal Child Health and Development
Comprehensive screening and referral programs have been shown to have mixed impacts on optimal child health and development measures. One major outcome assessed in evaluations of Family Connects was total infant emergency care use, which included overnight hospital stays and emergency department visits (less infant emergency care use suggests that families are using primary care instead of emergency department care for nonemergency needs). An initial report of findings 12 months after the intervention found 50 percent less total emergency care use among treatment families, with effects being larger among infants with more birth risks, infants with Medicaid or no insurance coverage compared to private insurance, and “nonminority”iv families.A An additional report of findings at child ages 6 months and 24 months found 0.91 fewer overall emergency episodes and 37 percent less total infant emergency care use, respectively, but the beneficial treatment effects were only significant among “nonminority” infants at 24 months.C The second RCT of Family Connects and the evaluation of Healthy Steps both found no significant impact on emergency care use.D,F,G
Additional measures of child safety, physical health, and behavior similarly showed mixed results. Family Connects was not found to have a statistically significant impact on accident- or maltreatment-related emergency department visitsC or Child Protective Services investigations.DHealthy Steps was significantly associated with lower odds (0.76 OR) of using incorrect sleep positioning for infants but had no significant impact on any other safety or feeding practices, including odds of breastfeeding.E Odds of timely vaccinations and pediatric appointments were 1.3 and 2.3 times higher, respectively, among Healthy Steps treatment families at child age 30 to 33 months,F but Family Connects had a null impact on timing of pediatric appointments.D Further, Healthy Steps was found to have no significant effect on mother-reported child behavioral problems at child ages 16 to 18 months, 34 to 37 months,I and 5.5 years,G though reporting of aggressive behavior was significantly higher among treatment families at child age 30 to 33 months.F
- An impact is considered statistically significant if p<0.05.
- The Nursing Child Assessment by Satellite Training (NCAST) assessment measures sensitive interactions and communication patterns between mothers and children.
- This study used the Responsivity and Acceptance subscales of the Home Observation for Measurement of the Environment (HOME) assessment.
- “Nonminority” reflects the authors’ language; this term is not defined in the study.