Earlier this year, we posed the question: should on-site subrecipient monitoring reviews be required? We talked about the benefits and challenges of in-person monitoring, particularly during the pandemic, and the lessons we’ve learned from our Early Childhood Birth-to-Three Monitoring team about remote alternatives.
Today, we’re taking a closer look at how remote options can improve the monitoring process for both the Monitors (called Assessors in this case) and the people they’re observing.
What is Early Childhood Birth-to-Three Monitoring?
The Early Childhood Block Grant provides funding to improve the quality of childcare within state systems. In Illinois, early childhood programs can apply for these state grant funds to support activities such as home visiting, doula services, center-based support, family childcare, and preschool services. The Vander Weele Group’s Birth-to-Three Monitoring team reviews programs like these to ensure that both compliance and performance goals are met.
Grant recipients and their programs are assessed using several kinds of tools, such as:
detailed compliance rubrics designed by the state;
quality-measuring tools, such as home visiting assessments and classroom observations; and
interview-based quality assessment tools.
The Assessors complete comprehensive reports that provide valuable feedback to grantee program leaders and their staff to improve program quality, enhance service delivery, guide professional development, and promote the best possible outcomes for children and families.
This is where remote monitoring tools can make a world of difference. Let’s take a look at a real-world example: virtual monitoring of home visiting programs.
The Benefits of Observing from a Distance
As you can imagine, families participating in early childhood programs face multiple stressors long before monitoring enters the process. These families may include young or first-time parents; they may be experiencing poverty or housing insecurity, or their child may have a disability. Whatever the circumstances, they’ve already had to trust a stranger to evaluate their situation and offer feedback. Despite knowing the service provider—such as a home visitor—is there to help, that’s an intimidating prospect. Adding in another observer, like a Monitor, on top of that—even if they’ve had time to build trust and get comfortable with their service provider—has the potential to seriously disrupt normal family dynamics.
It's not often much easier for the home visitors who provide services to these families. Being assessed on your work can be nerve-wracking, which might lead you to make mistakes you normally wouldn’t or behave in ways that aren’t typical of your interactions with clients. In either case, the problem is the same: if things aren’t ‘business as usual’, Monitors can’t accurately assess service quality or provide meaningful support to programs and practitioners.
One of the easiest ways to avoid scenarios like these is for Assessors to review recorded observations rather than conducting them in person. Time and time again, our Assessors—along with Monitors around the country—have found that removing the stress of a stranger in the room makes both home visitors and families more comfortable. It’s much easier to ignore a video camera sitting in the corner, freeing everyone up to communicate and interact more naturally. This provides a more realistic picture of a typical visit, allowing both Monitors and practitioners to do their jobs more effectively.
According to one member of our team, “Even if you try to be invisible, you change the dynamic just by being physically present. It feels more invasive; it can cause more anxiety. It can change how much the family is willing to share, even how the child behaves—maybe they’re anxious about a new person wanting their attention, possibly even crying, or maybe they’re friendly and want to sit on your lap. Either way, it’s not an honest representation of what a visit is usually like.”
Another benefit of recorded observations impacts the Assessors themselves. When you’re observing a visit in real time, it’s easy to miss something—part of a conversation, context for a specific interaction. Having the option to pause and re-watch footage multiple times, and even to produce transcripts to review verbal communication, leads to more accurate scoring. As one Assessor shared, “Things happen so quickly, it can be hard to track what’s going on. With the recordings, when you can’t hear things, or only saw something out of the corner of your eye the first time, you can go back and check: ‘Is this what she really said or did?’ It allows you to be more accurate.”
In addition, not being in the room in the first place prevents Assessors from interacting with the home visitors. While this might not sound like a good thing, it’s an important safeguard; Assessors aren’t permitted to engage with practitioners while they’re working. An Assessor who previously served as a home visiting supervisor explained, “As a home visiting supervisor, if I wonder what I missed, I can ask follow-up questions. Assessors can’t do that.” Another added, “The distance is helpful; I can’t ask, ‘Did I hear that correctly?’ Also, when you’re a supervisor, you can provide technical assistance, but as Assessors, that’s not our role.”
Beyond Quality Assurance: Monitoring During a Pandemic
Perhaps the most important lesson the Early Childhood Birth-to-Three Monitoring team has learned isn’t about how remote monitoring improves the validity and reliability of their observations, but how it can be used to support programs operating in crisis situations.
During the COVID-19 pandemic, the use of recorded footage allowed Assessors to continue monitoring activities without disruption, preventing gaps in mandatory oversight. It enabled them to contribute to the wellbeing of vulnerable children and families at a time when they faced even greater hardship than usual. Critically, it also reduced the risk of COVID exposure and transmission, protecting Assessors, home visitors, and the larger community.
Early Childhood Monitoring program manager Laura Abbruzzese summed it up best, saying, “The ability to do most of our monitoring virtually—including conducting home visit observations using recorded visits—has really streamlined the monitoring process. It’s made it more efficient and improved our practices, which will only improve our program and make monitoring more effective in the future.”