Busy as a Beehive: Utah Addresses General Supervision, Part One

00:00:01.52  >> You're listening to "A Date with Data" with your host, Amy Bitterman.
 

00:00:07.34  >> Hey, it's Amy, and I'm so excited to be hosting "A Date with Data." I'll be chatting with state and district special education staff who, just like you, are dealing with IDEA data every day.
 

00:00:19.50  >> "A Date with Data" is brought to you by the IDEA Data Center.
 

00:00:24.61  >> Hello, welcome to "A Date with Data" with Differentiated Monitoring and Support, or DMS 2.0 in full swing, and the general supervision Q and A that came out from OSEP back in July of 2023. General supervision is a high priority area for all states. And on this episode, we have joining me Leah Voorhies, who is the Assistant Superintendent of Student Support and also the State Director of Special Education, and also LauraLee Gillespie, who is the Special Education Coordinator of the Utah Program Improvement Planning System, and they are both with the Utah State Board of Education. They are going to be sharing with us the story of their state's general supervision system, how they use it to monitor and support compliance with federal and state requirements in local education agencies across the state of Utah. Welcome, Leah and LauraLee. Thank you so much for being here.
 

00:01:24.09  >> Thank you.
 

00:01:25.10  >> Thank you for having us.
 

00:01:26.52  >> Of course! Can you start out just really briefly introducing yourselves and saying a little bit about your role?
 

00:01:33.38  >> I can begin. I'm Leah Voorhies. I've been the Assistant Superintendent and State Director for going on 8 years now. Before that, I was the Special Education Coordinator, so like the assistant director in Utah, for 6 years. And I have been working on trying to improve and implement general supervision systems in either LEAs or the state for ... This is my 22nd year.
 

00:02:08.98  >> Mmm, you've a long history of it.
 

00:02:13.10  >> And I'm LauraLee Gillespie. I've been at the State 6 years now. My work has been around program improvement and compliance for the most part, the entire time I've been at the State. Previous to the State, I was at the Disability Law Center, which is the protection advocacy for the state of Utah, and representing parents of students with disabilities in special education.
 

00:02:39.38  >> Great. Well, I guess I want to start out just, if you could, talk about your more data-focused areas of your general supervision system, like the SPP/APR, data on results and processes, LEA determinations, just describe those areas to me and what you're doing in terms of monitoring and compliance.
 

00:03:00.75  >> I will start this part off. So we focus on data a lot ...
 

00:03:05.28  >> Mm-hmm.
 

00:03:05.55  >> ... through lots of different avenues. We focus on the indicator data as outlined in the annual performance report. We also look at state-level data and local education agency-level data as we work with local education agencies to improve outcomes and ensure compliance. Just for clarification, in Utah, charter schools and districts are all public schools, so we just refer to all of it as local education agencies. Some areas that we prioritized in Utah have included post-secondary transition planning. That's our ... It's just been an area of focus that we've needed to really focus in on, really looking closely at the data and trying to improve the outcomes for students with disabilities. We've done a lot with inclusive practices and effective instruction. We're also starting to do some educational benefit reviews and looking at the data of what we're finding from LEAs across 3 years. We're just starting into that. We're in our baby phase of educational benefit, and we just want to look ... We want to educate our local education agencies on the importance of monitoring progress over time as well as developing meaningful and rigorous individualized education programs. So again, looking at indicator data across the board, looking at specific local data as well as state data to try to really get a whole picture. Currently, our LEAs annual performance determinations are based on the 16 indicators, so 1 through 16 as they are placed on the annual performance report.
 

00:04:51.99  >> Mm-hmm.
 

00:04:52.21  >> We go through and bring those together and develop a score, and they get a determination, but we also do a results-driven accountability or a risk score as well as that APR determination. And the risk score includes 1 through 14 of the indicators, but we also take into account fiscal data, timely and accurate submissions of data, the scoring on the completeness of the program improvement plan, if they've had corrective actions plans.
 

00:05:26.08  >> Mm-hmm.
 

00:05:26.37  >> And part of the reason for that is to help them to ... to help us to see how we can best support them. So the higher the risk level is, the more support the LEA is offered through the state board of education. I can jump into a little bit further about our monitoring, but I want to know are there any questions at this ...
 

00:05:49.08  >> Yeah, I was just ... I was going to ask that, yeah.
 

00:05:51.34  >> Yeah.
 

00:05:51.46  >> In terms of the risk score and how that feeds into the monitoring and how you decide which districts would ... or LEAs will get monitored and when?
 

00:06:01.73  >> Yeah, so we've gone through some overhaul in the last 3 years which has been a really great change for us. We switched to ... I want to call it a 6-year cycle, but it's not a cycle as in, I'm going to come to your LEA, and you can just say, "Oh, I'm off the hook now for 6 years."
 

00:06:21.70  >> Mm-hmm.
 

00:06:21.95  >> The way we look at it is, we tell the local education agencies that we are going to come at least once every 6 years. We may come more. We may come less, and this is where the risk score comes in. So the risk determination is considered as well as several other factors. So the higher the risk, the more likely we are going to be coming to them.
 

00:06:44.42  >> Mm-hmm.
 

00:06:44.78  >> But we also look at other pieces as well. We are really focusing in on our targeted support and improvement schools, or our TSI schools. We look at how much dispute resolution they've had.
 

00:06:57.33  >> Mm-hmm.

00:06:57.88  >> We have a hotline here at the State Board of Education, so we pull that into the determination of who we're going to be monitoring during the year. If there's fiscal concerns, we take that into consideration, and that's sometimes a specific population. So our youth in custody, our youth in care student populations are always a focused part of ...
 

00:07:22.77  >> Hmm.
 

00:07:22.89  >> ... who we do full monitoring with, but it may increase the likelihood that we're going to come a little sooner rather than later.
 

00:07:30.25  >> Hmm.
 

00:07:30.35  >> So again, it's a 6-year cycle, but the cycle is, you can be moved up quicker depending ...
 

00:07:37.06  >> Mm-hmm.
 

00:07:37.36  >> ... on the risks and needs that we see based on your data and other factors.
 

00:07:43.71  >> A very holistic approach, it sounds like ...
 

00:07:46.39  >> Mm-hmm.
 

00:07:46.60  >> ... looking at comprehensively all factors to make those types of decisions.
 

00:07:52.58  >> Absolutely, and that's our full monitoring. And, of course, we do additional monitoring for fiscal ...
 

00:07:58.57  >> Mm-hmm.
 

00:07:58.79  >> ... Indicators 4, 9, 10, 11, 12, 13, and our significant cognitive disabilities are 1 percent. And those aren't based as much on risk. They're based more on data and information that we have on file regarding those specific indicators, determines who we look at.
 

00:08:18.61  >> Great. And through all these different areas that you're focusing on, can you talk about how stakeholders are involved in either making some of the decisions around what these different processes may look like, reviewing results, giving feedback? How do you bring them in?
 

00:08:38.01  >> So this is Leah, and stakeholder feedback is important to us because it's required.
 

00:08:46.44  >> Mm-hmm.
 

00:08:46.65  >> It's also important to us because we're a small enough state that we actually know all of our major stakeholders. We participate with them in all sorts of activities, not just in gathering feedback related to the activities that we do.
 

00:09:06.44  >> Mm-hmm.
 

00:09:07.17  >> So we have established as an entire agency, not just in special education but as an entire education agency, feedback gathering standard operating procedures. So ...
 

00:09:21.53  >> Hmm.
 

00:09:21.66  >> ... For all of the decisions that our elected State Board of Education makes, we have a process to gather feedback. And then in special education, we ... Because we know all of the frequent players when it comes to students with disabilities and individuals with disabilities, we interact with them frequently. So I meet monthly with the director of our parent training and information center.
 

00:09:55.51  >> Mm-hmm.
 

00:09:56.76  >> My team members meet monthly with the PTA, with the board of the parent training and information system. We meet regularly with our Disability Law Center. We have a legislative coalition for people with disabilities that we meet with. I serve as the chair of a statutorily required committee on policy, state policy related to students and individuals with disabilities. All the state agencies in this state are required by statute to be ... participate in that committee, as well as all the nonprofit organizations that serve individuals with disabilities that receive any state funding.
 

00:10:41.83  >> Hmm.
 

00:10:42.30  >> And then we have a robust focus group and follow-up survey system that we use. So our first strategy is that we go to our stakeholders. We don't ...
 

00:10:56.50  >> Mm-hmm.
 

00:10:57.23  >> ... require that they come to us. And so we go to their meetings. We participate in their meetings as regular members, even voting members most of the time, and then we hold focus groups with them when there is something specific that we need them to comment on, and then we have a follow-up survey. And the focus group, the follow-up survey process is an agency process. That's not just special ed.
 

00:11:27.92  >> Yeah, I like that idea a lot. I think states do seem to see more success sometimes going to the stakeholders rather than adding something else, another meeting or time on them, but hitting them when they're already gathering or meeting for other purposes.
 

00:11:47.06  >> And if I tried to roll something out that they hadn't had ... that they hadn't provided feedback for, it wouldn't go very well.
 

00:11:58.15  >> You'd hear about it?
 

00:11:59.59  >> We have created ... They're my friends. They're my team members, friends. We have ... Luckily, I've been the State Director long enough, and my team has been ... LauraLee has been with us for long enough that we have a longstanding team. We have a cohesive team. They know us. They know us personally. We know them personally. It's not just that we respect each other. It's that we're friends, and it would damage our friendships, and that is not okay with us or with them ...
 

00:12:36.08  >> Mm-hmm.
 

00:12:36.70  >> ... for that matter.
 

00:12:37.75  >> Yeah, so building those relationships ...
 

00:12:39.94  >> Yeah.
 

00:12:40.64  >> ... is very key. What are some of the aspects of your general supervision work that you're especially proud of, that you really want to highlight?
 

00:12:51.33  >> There's a lot of things. It's been a ... As Leah said, the stability has led to some opportunities in terms of general supervision that have worked well. In no particular order, when this question was asked, I started to think about these things. One thing that we do in terms of monitoring that I'm really proud of is, we've really worked hard to, as one of our colleagues says, "Practice supports compliance, and compliance supports practice." And we've really tried to ensure that that is evidenced by the work that we do at the State. So in other words, when we go out and do monitoring, our specialists, our content specialists in effective instruction and inclusion and behavior, they come right along with us on those monitoring visits. They conduct the interviews, and what I see almost every time we go on a monitoring visit is that we start out with this very nervous feeling ...
 

00:13:57.23  >> Mm-hmm.
 

00:13:58.42  >> ... amongst the staff, and that doesn't always go away. But a lot of staff build relationships with our specialists during those monitoring visits and actually follow up with them and have conversations with them about their concerns. And so that ... Again, it's that building a relationship, and our monitoring is all on-site, and that's very ...
 

00:14:21.90  >> Okay.
 

00:14:22.07  >> ... intentional because we want ... We really want to build those relationships with those teams. Just as an example, our assessment specialist, she had a very in-depth conversation with one of our rural school districts, and they've continued to communicate with each other based on the monitoring visit.
 

00:14:42.35  >> Hmm.
 

00:14:42.47  >> And Indicator 13 is another really great example. We have this amazing coach for Indicator 13. She does just an incredible job, and as they go through and do the file reviews, she's right there. And they're doing the file reviews and they're checking compliance, but she is right there giving them tips and ...
 

00:15:04.20  >> Mmm.
 

00:15:04.38  >> ... things that they can use as they move forward. And again, that, to me, is powerful in terms of trying to really improve outcomes for students and think about it through a different lens. We've been doing a big focus on inclusion here ...
 

00:15:24.33  >> Mm-hmm.
 

00:15:24.55  >> ... in the state of Utah, and I think that's starting to meld into some of our monitoring and our general supervision. A lot of the technical assistance documents, we can immediately pull those as we go out and are monitoring or helping an at-risk school to ... and again connecting them to folks at the state who are specialists in these fields. Another piece, again I'm going to back to educational benefit. We're newbies to this. There's other states out there that have tried it, but our educational benefit specialist recently was with a school and went through that process, and the power that was instilled in the actual LEA, not by the state, not us telling you what to do, but the LEA, the staff there realizing and recognizing what they needed to do to really improve progress for students with disabilities was ... It's just phenomenal. And the last thing, I think, is the internal thing that we do as a state. We, for the last 4 years, have done general supervision sessions. So this isn't so much so much for the LEAs on the outside but really for our staff on the inside, and this was ... Again, I have to give Leah credit for because she had this wonderful idea that we need to ensure that all of our staff understand general supervision ...
 

00:16:50.03  >> Mm-hmm.
 

00:16:50.69  >> ... that golden requirement and why we exist as a state agency. And it's led to ... We just meet once a quarter, and we train out on different components on the general supervision pieces. We talk about rules. 2301 has been a big focus this year [Indistinct] and will be next year again ...
 

00:17:13.24  >> Mm-hmm.
 

00:17:13.47  >> ... because it's the new thing that is related to general supervision. So I kind of belled on, but I have a lot of things that I'm very proud of that have been happening. Leah, I don't know if you have any additional thoughts.
 

00:17:28.68  >> No, that's a good overview of things that we are proud of. I think where you started, LauraLee, with the fact that we have a team that has been together for a long time and that, as a result of that, we've been able to be very intentional about our continuous improvement process.
 

00:17:56.80  >> Mm-hmm.
 

00:17:57.03  >> And we've been able to be intentional with our LEAs and with our stakeholders about our continuous improvement process and support each other, and with the IETs that are training and preparing new special education providers. So that's ... I think that's definitely one of the things that we're proud of is, because we have been around a while, we have established relationships that are making it easier for us to improve our processes, and hopefully that leads to improved outcomes for students.
 

00:18:33.94  >> Yes. Well, it sounds like you have a lot in place to be proud of, for sure, and that sustainability seems like a major part of that because of building those relationships and maintaining those relationships and that trust, so kudos.
 

00:18:49.69  >> I think Leah brings up a really good point about the institutes of higher education and efforts to really prepare teachers. It's looking at it again from a whole picture, right, of ... It's not just, what can we do with the LEAs? It's, what can we do with those who are coming into this field to improve that as well?
 

00:19:09.41  >> Thank you so much, Leah and LauraLee, for sharing with us all of the great things you have going on in Utah around the data elements of your general supervision system. Please make sure to listen to our next episode of "A Date with Data," when we will be hearing more about your general supervision system and all the wonderful things you have going on.
 

00:19:36.65  >> To access podcast resources, submit questions related to today's episode or if you have ideas for future topics, we'd love to hear from you. The links are in the episode content, or connect with us via the podcast page on the IDC website at ideadata.org.