You may have encountered the saying “what gets measured gets managed,” which has become prevalent in the realm of education amidst the evolving landscape of assessment tools, big data, and the classification of schools worldwide.
As superintendents, we are constantly immersed in a cacophony of measurements: Which schools excel? Which district boasts the highest-paid staff? Which town allocates the most resources to security? The answers to these inquiries influence decisions on recruitment, retention, property values, taxes, and various other aspects.
However, discrepancies arise when school communities, administrators, and ranking publications employ divergent criteria to define excellence. This quandary confronted us during our tenure as district leaders in Madison Public Schools.
To address this challenge, we embarked on a journey to assess different ranking systems, survey stakeholders, and analyze student performance metrics within Madison Public Schools. In 2018, we initiated the practice of sharing our findings with board members and the school community through an annual “State of the Schools Address.” Subsequently, in 2022, we took a step further by presenting our insights at the New Jersey School Boards Association’s Workshop, garnering an audience of over 200 board members, administrators, and superintendents.
This article aims to showcase our endeavors so that school board members and district leaders can effectively navigate the realm of rankings and rating systems to articulate “school quality” to their stakeholders. The complexity of this topic indicates that a simple path forward does not exist, rendering the adage “what gets managed gets measured” inadequate for school or board leadership.
Distinction Between Rankings and Ratings Systems
Before delving into specific publications or datasets, it is crucial to grasp the fundamental framework of how these systems operate. This entails understanding the disparities and resemblances between rankings systems and ratings systems at a foundational level.
Ratings systems typically assign numerical scores to entities based on a scale where a higher score signifies superior performance. You are likely acquainted with ratings systems in consumer goods (e.g., Amazon), restaurants (e.g., Yelp), or travel (e.g., Orbitz), where a higher rating influences decision-making. Conversely, rankings systems assign an order value based on specific metrics, with lower values indicating higher rankings. In the education sector, publications like U.S. News & World Report, Niche.com, and SchoolDigger.com evaluate schools using diverse criteria such as test scores, college readiness, teacher retention, and student progress.
Data Sources
Each ranking or rating system relies on various publicly available data sources to formulate quality assessments. These systems access data reported to state and federal governments, encompassing a wide array of metrics such as state assessment results, attendance records, suspension data, demographic information, graduation rates, staff data, budget allocations, surveys, and more.
Exploration of Specific Ratings and Rankings Systems
The subsequent section scrutinizes several ratings and rankings systems commonly discussed in New Jersey, each characterized by unique methodologies, areas of focus, and considerations.
NJDOE Ratings:
The NJDOE ratings system assigns summative scores to schools based on criteria like NJSLA proficiency, growth, graduation rates, and chronic absenteeism. It employs percentile rank to compare schools within grade bands, aiming to provide a straightforward evaluation of schools in New Jersey. However, the system’s reliance on specific metrics may oversimplify the student experience within a school.
GreatSchools.org Ratings:
GreatSchools.org provides percentile ranks for schools based on test scores, student progress, college readiness, and equity. The platform offers transparency in methodology but incorporates user-generated reviews, potentially introducing biases.
U.S. News & World Report Rankings:
This system assesses districts and schools based on factors like college readiness, math and reading proficiency, and underserved student performance. While transparent, it focuses solely on high schools, potentially overlooking alternative paths after high school.
Niche.com Rankings:
Niche.com incorporates survey data alongside various factors, adding a qualitative dimension but also subjectivity. The fixed category weights may not fully capture evolving priorities in education.
SchoolDigger.com Rankings:
SchoolDigger.com focuses solely on student performance on state assessments, providing percentile ranks over time. However, its exclusive reliance on standardized test scores may overlook other aspects of school success.
Key Considerations
Navigating the array of factors used in ratings and rankings systems underscores the complexity of defining “school quality.” Recognizing that these systems are developed by third parties with varying interests, understanding the variability across systems, acknowledging winners and losers created by rankings, addressing data lag, and fostering community engagement are essential aspects in interpreting and utilizing these metrics effectively.
In conclusion, while the adage “what gets measured gets managed” holds merit, the intricacies of school rankings and ratings systems necessitate cautious interpretation and consideration of the unique needs and expectations of school communities. Transparency, data quality discussions, and bias mitigation are vital for district leaders and board members to navigate these systems effectively.