By the IDC Buzz Staff
Meet the NRBA App
Driving directions. Film reviews. Dinner reservations. These days, there’s an app for just about anything you can think of. And now, thanks to IDC, there’s an app for nonresponse bias analysis, too. Designed to help states examine their SPP/APR Indicator 8 and Indicator 14 data through nonresponse bias analysis, our new NRBA App allows users to conduct reproducible analyses tailored to their survey’s data collection method. More, this state-of-the-art resource’s point-and-click interface allows users to find the answers to questions like
- What are our response rates, and do they differ across subgroups?
- Are some subgroups in the population overrepresented or underrepresented in our respondent data?
- How do survey outcomes differ across subgroups?
- Can statistical adjustments reduce nonresponse bias in our data?
Read on to learn more about the development and purpose of this remarkable new tool.
There’s…Not an App for That?
Let’s flashback to the announcement of the updated requirement for the FFY 2020 SPP/APR. Beginning with that submission, states are required to measure nonresponse bias—systematic error in a survey estimate that causes it to be too high or too low—for any indicator that gathered data using survey methods. For Part B data, this specifically refers to Indicator 8 and Indicator 14.
Unfortunately, most existing survey analysis tools featured only a “plug-and-go” interface that afforded users precious little control. The demands of the new SPP/APR requirements, however, necessitated greater flexibility: Different states would need to conduct different calculations at different times. Clearly, states needed a more refined tool, one powerful enough to allow them to control how they assessed their survey data but adaptable enough to cover many use cases.
The Making of the NRBA App
Enter IDC TA Specialist and survey methods expert Tamara Nimkoff. Together with Westat statistician and survey methodologist Benjamin Schneider, who just so happens to also specialize in nonresponse bias analysis, Tamara began the process of creating a different kind of IDC tool, something that wedded the unique data needs of special education to the complex statistical analyses inherent in survey evaluation, a true next-generation software program that would become the NRBA App.
Collaborating with Westat software engineers, the team developed the NRBA App using the open-source R Shiny web development framework, which provides users with a guided point-and-click interface and state-of-the-art statistical methods. After a routine download, users access the app within their preferred web browser—Google Chrome, for example—while RStudio runs the computations in the background. Prompts in the app guide users to import their data, identify the data-collection method, enter additional survey details (if applicable), and proceed to analysis.
Getting Started With the NRBA App
Interested? If so, we strongly encourage you to contact your IDC State Liaison. Though the app is available to download through the R site (and though we make available a reference guide), the complicated nature of this resource might require a bit of getting used to. An experienced hand can lead you through the process of installation and navigation to help you get started. For this and any other questions you might have, your IDC State Liaison can be found here. You can also email us at IDEAdata@westat.com.