Slow Dance With Stakeholders: Setting the Pace in North Carolina, Part One

00:00:01.51  >> You're listening to "A Date with Data," with your host, Amy Bitterman.

00:00:07.34  >> Hey. It's Amy, and I'm so excited to be hosting "A Date with Data." I'll be chatting with state and district special education staff who, just like you, are dealing with IDEA Data every day.

00:00:19.50  >> "A Date with Data" is brought to you by the IDEA Data Center.

00:00:24.43  >> Welcome to "A Date with Data." Today, I am joined by Kelley Blas, Part B Data Manager, and Lauren Houlihan, the SSIP Coordinator, who are both with the North Carolina Department of Public Instruction. We know that states are required to engage stakeholders throughout the entire SPP/APR process, and Kelley and Lauren are here to talk about how they've been ensuring stakeholders are meaningfully part of the process on an ongoing basis. Thank you, both, so much for being here, and I'd like to start out, if you could each say a little bit about yourselves and your role. Kelley, do you want to go first?

00:01:03.36  >> Sure. So again, I am considered right now the Part B Data Manager. I've also taken on the role as the SPP/APR Coordinator. I have been with the Department of Public Instruction for going on 17 years now, and about 15 of that was solely working as the Part B Data Manager, and we had a separate SPP/APR Coordinator, and she retired about 2 years ago, right when the measurement table changed, and I had a huge learning curve in studying these targets and baselines for the new measurement table, but it has really been an adventure for me, so happy to be here. Thank you for inviting us.

00:01:48.90  >> Yeah, thank you. Lauren, how about you?

00:01:51.71  >> Yeah. I have been with the Department of Public Instruction since 2007, so 16 years, about the same amount of time and similar journeys in terms of rural transitions here at the department. I started as the Consultant for Occupational Therapy, and then our SSIP Coordinator at the time, Matt Hoskins, became our Assistant Director. I moved into the SSIP Coordinator role about the same time that Kelley moved into the SPP/APR Coordinator role, and so we have definitely learned and kind of grown up together in our understanding and our coordination of our state's efforts around the SPP/APR.

00:02:37.19  >> Yeah, it must be nice to kind of have someone to go through that with in a similar way.

00:02:41.95  >> Well, and I'll say not just someone, but I have Kelley Blas.

00:02:45.79  >> Yes.

00:02:46.03  [ Chatter ]

00:02:46.03  >> Say no more.

00:02:47.18  >> She's just in it, and an incredible, incredible ... one of the more productive people I've ever met.

00:02:54.79  >> Oh, that's wonderful. Great. Thank you. So just to get started, can you talk about how you've been engaging stakeholders in the SPP/APR process?

00:03:09.33  >> Sure, I'll start out with that. Obviously, our journey began with the new measurement table, and so right away, we knew that we were going to need a lot of organization and support as we started this journey to reset the baselines and targets for our SPP/APR [Indistinct]. So we immediately contacted our IDC person, Rachel, and we started off by using the ... I think it was called the Planning Template, and maybe you can correct me if I'm wrong, but there was an outline for us that allowed us to kind of plan what we were going to do in that process, and part of that was setting our time line and how we were going to hold these meetings to involve our stakeholders in resetting our baseline brackets, and so that was a very helpful tool, very helpful meeting for us in deciding how we were going to do it, and I'll let Lauren kind of help with the outline of those meetings and how we brought people together.

00:04:19.82  >> Yeah, it seems like a really long time ago, but I guess it was just really 2020. I think part of ... That was also at the beginning. Our journey started sort of with COVID too, and so just thinking about how to do all this work in a virtual space. It was a different kind of coordination. So we really ... That summer, fall of 2020 and on through until we got really through 2021 because this was for the FY 2021 submission, right?

00:05:00.07  >> Mm-hmm.

00:05:00.48  >> We're 2 years into the new measurement table. And so we held all of our stakeholder meetings to develop the new SPP online. We did a pretty wide invitation, wide-open invitation. We worked with lots of our different advocacy organizations and our local contacts with our LEAs, just to make the information about when the meetings were, and we just tried to cast a really broad net.

00:05:37.86  >> Mm-hmm.

00:05:39.32  >> We had ASL interpretation and Spanish interpretation for all of those meetings. We did multiple kind of repeated sessions on individual indicators. We recorded all of those sessions, posted all of the recordings, kind of kept that information in front of our partner organizations and our LEAs.

00:06:02.33  >> Mm-hmm.

00:06:03.37  >> I think one of the things that ... Well, one is, we didn't know what we were doing, and we had no idea what it meant to just throw wide the invitation, right, to ... There were several of those meetings where we had 75 or 80 people on the call.

00:06:23.77  >> Wow.

00:06:23.91  >> I think one of the things that I've said in the past, some of the maybe confidence or maybe just foolishness to do that was ... Our leadership in the Office of Exceptional Children, several years ago, invested in facilitator training for most of the OEC consultants, so we've been through pretty rigorous facilitation training so that we can conduct focus groups and large meetings and move groups through decision-making processes, through analyses of problems or situations. We have structured protocols for that, and I think just being equipped in that way really set us up for some surprising success and maybe taking some risks in terms of that open invitation that we might not have been so comfortable trying that if we had not had that training.

00:07:31.95  >> Wow. That's really cool that you all have that training.

00:07:34.78  >> Yup.

00:07:35.02  >> It seems like, duh, everyone should have that, especially given, yeah, the stakeholder engagement work and how much you would need to be facilitating those meetings and making the most of people's times and learning those strategies to really engage and get productive conversations going. So that's really cool.

00:07:57.62  >> Yup, yup.

00:07:58.62  >> And what are some ways that you've, kind of during the meetings when you've brought the stakeholders in, that you've helped to build their capacity to really be able to provide that meaningful and useful feedback?

00:08:15.26  >> I'll start off on this one again and then toss it over to Lauren, but one of the things that we did right off the bat was, as soon as we got the measurement table, we began setting up data displays for each indicator, and so what that looked like was, we would take at least 5 years of historical data and look at that, and then we would show them what it would look like with the change. For example, in Tier 1, the whole dataset changed, so we showed them what it looked like from before and what it would look like after, and we did some backtracking with the after as well, just to try to give them some meat to what they were looking at in a full understanding of what that indicator was about and what the change was about, and we went really, really slow, so I don't think we had more than two indicators at time per month, so we went really slow. We talked through all of the changes, and we also provided them with some options for their baseline and their targets. We didn't just say it has to be one of these. We also offered for them to have some conversation and provide some feedback if there was an option that we didn't think of. So we really tried to engage them in thinking about what the meaning of the indicator was, how it impacted our schools and our districts and our state data and then also allowed them to think about how they would set their targets, and so that seemed to be very helpful, I think. There was, and we may get into this in challenges, but I'll just go ahead and say it. I think at times, we may have provided too much data.

00:10:03.43  >> Mm-hmm.

00:10:06.44  >> And there's times where we felt and got feedback that they were a little lost in what we provided, so when that would happen, we've been trying to go back and analyze it. Lauren, what do you think?

00:10:19.91  >> Yeah. I just think, and a couple of things to add to building stakeholder capacity: We definitely had this kind of whatever it was, 16-month schedule of how we were going to move with our stakeholders through these indicators. We had a plan. And I remember pretty distinctly, I think it was summer, July of '21, we got to Indicator 3, and just the particular group of folks that we had, they're like, "You're asking us to make decisions about stuff that we don't understand well enough, and we just got to slow down." There was just some ...

00:10:58.12  >> Mm-hmm.

00:10:58.61  >> And because ... What, there are 16 different baselines and targets with Indicator 3 now.

00:11:04.18  >> Yeah.

00:11:05.59  >> And we were like, okay. We just need to slow down. We're going to take another month. We're going to back up. We're going to explain what each of these different parts of Indicator 3 mean. We're going to give folks more information ahead of time, and we kind of ... We revised our schedule. We let people know we were responding to the request, and so I think just listening and being able to ... being willing to adjust empowered those participants to just kind of stay with us and to not be threatened by people's sort of emotion. I think that was another ... We just created space and made adjustments.

00:11:46.95  >> Mm-hmm.

00:11:47.78  >> The other thing I'll say in terms of stakeholder capacity is particularly for those stakeholders who just showed up month after month after month. WE extended invitations to lots of those folks to join our what are now SPP workgroups. We have four different workgroups: data literacy, stakeholder and family engagement, research and informed practices, which we ... and then our systems coherence, which we sort of modeled after the National Center of Systemic Improvement, one of their frameworks.

00:12:25.31  >> Mm-hmm.

00:12:25.66  >> Those workgroups aren't just DPI employees, and they now have external stakeholders that participate in them, and in fact, we have coleaders that are external stakeholders on a couple of those workgroups.

00:12:38.90  >> Hmm.

00:12:38.99  >> And so folks really, I think, continue to deepen their understanding of the Indicators and the work in terms of improvement science that goes along with actually seeing those data improve over time, so we're really excited about that level of ... And our capacity has been built because of that level of engagement on workgroups.

00:13:05.98  >> Yes, for sure.

00:13:07.54  >> Yeah.

00:13:08.41  >> That's such a perfect example of how really taking those stakeholders and bringing them in, not just at that target setting stage but carrying through to talking about the data and the results and improvement strategies and all those other elements, so it sounds like that was a really great strategy to bring in folks who had taken part in that target setting and bring them in to these ongoing continuous workgroups now.

00:13:38.72  >> To access podcast resources, submit questions related to today's episode, or if you have ideas for future topics, we'd love to hear from you. The links are in the episode content, or connect with us via the Podcast page on the IDC website at ideadata.org.