Module 2A: State’s System of Annual Meaningful Differentiation (AMD)
This webpage is part of the Evaluating State Accountability Systems Under ESEA tool, which is designed to help state educational agency (SEA) staff reflect on how the state’s accountability system achieves its intended purposes and build confidence in the state’s accountability system design decisions and implementation activities. Please visit the tool landing page to learn more about this tool and how to navigate these modules.
All states have developed or revised their accountability systems in response to requirements in the Elementary and Secondary Education Act of 1965 (ESEA), as amended by the Every Student Succeeds Act (ESSA). States’ accountability systems include multiple indicators, each of which illuminates a different facet of school performance or quality, though to many stakeholders the outcome of central interest is the summative rating determined annually for each school to differentiate its overall performance and quality from other schools. (Note: Final determinations are not required but are used in many states.) Annual determinations, whether summative or strictly at the indicator level, are used to identify schools that need support and improvement, help identify support strategies to improve outcomes for all students and/or student subgroups, and communicate performance to parents, educators, policymakers, and other stakeholders. The focus of this module is the methodology for the state’s system of annual meaningful differentiation (AMD) (see Figure 1 below).
Figure 1. State’s System of AMD.
All states are required to implement a system of AMD for all public elementary and secondary schools. States vary in their approaches to this. Some states use summative ratings (e.g., A–F, 1 to 5 stars) based on overall school scores, usually in a range of 0 to 100, that combine results across all indicators (e.g., through a weighted-average index score based on weights assigned to each of the five indicators).1 In many states these overall scores drive the identification of schools identified for comprehensive support and improvement (CSI) (i.e., the lowest performing 5 percent of Title I schools), schools identified for targeted support and improvement (TSI), and additional targeted support and improvement (ATSI) schools. Other states use dashboards that juxtapose summative ratings with individual indicator results, while some states rely on dashboard results exclusively in combination with business rules that outline how schools are identified for CSI, TSI, and ATSI. Regardless of approach, the AMD system should reflect the state’s theory of action and the policy objectives of the state’s accountability system.2 Figure 2 highlights examples of how some states differentiate between schools and how they communicate overall performance to the public.3
Figure 2. Examples of state approaches to State’s System of AMD.
| Summative Rating Provided | No Summative Rating Provided | |
|---|---|---|
Summative Rating |
Component Ratings |
No Summative Rating Provided |
The last section of this module (see below for more information) differentiates between whether a state uses a summative rating (e.g., an index-based approach) or a non-summative rating approach (e.g., a dashboard approach).4
This module includes three sets of self-reflection prompts that are intended to address the following concepts for the state’s system of AMD component within the broader state’s accountability system (Table 1). These three sets of prompts are not intended to be discrete; instead, they are intended to work together to help you answer questions in the following sections of this module.
Table 1. Overview of Module 2A: State’s System of AMD
| Section | What is it? | Why is it important? | How will it be used? |
|---|---|---|---|
| Section 1. Articulate the Rationale Behind the State’s System of AMD | A description of why the state’s system of AMD is designed the way it is | Understanding why the rationale of the state’s system of AMD is critical for developing a message that can be used for multiple audiences to describe the “what” and “why.” In addition, this serves as an opportunity to ensure you have a design against which you can check operations. | The rationale for the state’s system of AMD asks you to describe the expected policy objective, behavioral intent, and results of the state’s system of AMD. This rationale can be used as a point of comparison for activities associated with differentiating schools. This also will help you understand how well your rationale is reflected in the results of the state’s system of AMD. |
| Section 2. Consider Stakeholder Perceptions of the Rationale Behind the State’s System of AMD | A reflection on whether stakeholders understand the rationale behind the state’s system of AMD design and the underlying assumptions regarding the connections that support the state’s system of AMD’s rationale | Not all aspects of the state’s accountability systems carry the same level of risk. Determining what assumptions or connections require more exploration can help minimize misunderstanding and help prioritize resources to identify opportunities for revision or evidence collection. | The stakeholder perceptions of the rationale section asks you to think about the questions you’ve considered and information you’ve collected based on your state’s design. If there are particular areas that you would like stakeholders to better understand, that may be an indication of where you |
| Section 3. Assess Confidence in Operations and Results of the State’s System of AMD | Based on your rationale and potential risk, the opportunity to examine your level of confidence that design decisions are sound and evidence supports your assumptions | Determining your overall confidence in the results and presentation of the state’s system of AMD can help you collect appropriate evidence, make system revisions, or develop outreach materials. | The confidence in operations and results will help you identify potential evidence that can confirm the state’s system of AMD’s rationale and design. The rationale also can be used as a point of comparison for design decisions, and the risk assessment can be used to focus attention on key confidence claims. Please note that you will need to choose a specific set of reflections based on whether your state uses a summative rating approach or a dashboard approach. |
To get started, click on the link below:
[Begin Module 2A: State’s System of AMD]
1 States must assign “substantial” weight to each of the academic achievement as measured by proficiency, other academic, graduation rate, and English language proficiency progress indicators, and assign, in the aggregate to these four indicators, “much greater weight” than the weight assigned to the indicator or indicators of school quality or student success. [Back]
2 For more information, see Establishing Performance Standards for School Accountability Systems (link is external) from the Council of Chief State School Officers. Please note: The inclusion of links to resources and examples do not reflect their importance, nor are they intended to represent or be an endorsement by the U.S. Department of Education (ED) of any views expressed, or materials provided. ED does not control or guarantee the accuracy, relevance, timeliness, or completeness of any outside information included in this document. [Back]
3 These examples reflect how some states differentiate school performance in approved consolidated state plans. [Back]
4 There may be multiple ways in which states can meet the ESEA requirements to annually meaningfully differentiate between and identify schools. This module focuses on the summative rating and dashboard approaches based on trends in state approaches at the time of publication. [Back]
