Arizona Assessment Letter
June 12, 2001
Honorable Jaime Molera
Superintendent of Public Instruction
Arizona Department of Education
1535 West Jefferson Street, Bin 2
Phoenix, Arizona 85007
Dear Superintendent Molera:
On May 2, I discussed the status of Arizona’s assessment system with your predecessor, Ms. Lisa Graham Keegan. As I mentioned during the phone call with Ms. Keegan, the Department submitted information provided by your state on Arizona’s assessment system to a group of external reviewers with strong expertise and experience in the design of State assessment systems. The external reviewers found much to commend in Arizona’s system, but also raised several issues related to the substance and timeline for completion of some of the Title I requirements.
The purpose of this letter is to seek clarification and further information on these issues in order to ensure a complete and accurate understanding of Arizona’s final Title I assessment system. During my conversation with Ms. Keegan, she indicated that Arizona has readily available documentation that will address the issues I raised. This is an opportunity to provide additional information before we complete our review and the approval process for your State. Please note, however, that much of the information we are requesting will need to be peer reviewed; the peer review process may raise additional questions that the State will need to answer.
Below is a summary of issues that we believe need clarification and additional documentation.
Arizona’s final Title I assessment system
Title I requires that, for the purposes of school accountability, States annually administer assessments to students in at least reading/language arts and math, and in three grade spans (3-5, 6-9, and 10-12). State assessment systems must also include multiple measures that assess higher-order thinking skills and understanding. Title I also requires that the final State assessment be the primary measure of local educational agency (LEA) and school progress.
It is our understanding that Arizona made a significant change in your Title I final assessment system after you submitted assessment documentation for our review. According to Arizona’s original submission, the State’s accountability system was based solely on extracted items from the Stanford Achievement Test, 9th Edition (SAT-9). Subsequently, State staff informed us that Proposition 301, a ballot proposition passed in November 2000, would make the Arizona Instrument to Measure Standards (AIMS) the primary evidentiary basis for district and school accountability. However, during our phone call, Ms. Keegan stated that Arizona will use both the SAT -9 and the AIMS as the State final Title I assessment system. Ms. Keegan called it a “dual assessment system.”
We request that you provide us with a comprehensive description of your complete assessment system. I note that the information we currently have only partially documents the two components of Arizona’s assessment system. For example, the original submission provided partial inclusion data for the SAT-9, but no inclusion data for AIMS. That submission also discussed the use of extracted items from the SAT-9 as the only measure of school and district progress. The original submission did not discuss how the components of the full assessment system Ms. Keegan mentioned are used together to meet all of the Title I requirements for a final assessment system. If both SAT-9 and AIMS are used to meet the Title I requirements, we would need complete evidence on both tests for each of the issues described below. The State also needs to provide evidence showing how this full assessment system is used for accountability purposes.
Performance standards and alignment
Title I required that, by the 1997-98 school year, States develop challenging State content and student performance standards. Performance standards must describe three levels of performance (partially proficient, proficient, and advanced), and should be aligned with the content standards for each of the three required grade spans (3-5, 6-9, and 10-12). Title I further requires that State assessments be aligned with challenging content and performance standards.
Arizona has provided partial evidence addressing performance standards; to date you have also provided some materials documenting performance standards for AIMS at the third, fifth, and eighth-grade levels. Our understanding is that the performance standards for the high-school level AIMS had not yet been developed.
In order to meet Title I requirements, Arizona needs to demonstrate that it has developed the performance standards for all the required grade spans, and that those standards are aligned with your full assessment system. The assessment system should measure achievement as defined by the performance standards. The State must also demonstrate that the complete assessment system covers the range and cognitive complexity of the State’s content and performance standards.
Arizona should review the information originally submitted to determine whether it is still current. As evidence of alignment between the State standards and the SAT-9, Arizona submitted a copy of Harcourt Brace’s response to the Arizona Department of Education’s request for proposals for a standardized achievement test. Since this document was completed before the implementation of the SAT-9, the State may have more recent information.
Inclusion
Title I requires that State assessment systems provide for the participation of all students in the grades assessed and use their scores to evaluate school progress. The statute requires reasonable accommodations for students with diverse learning needs and provision of assessments in the language and form most likely to yield accurate and reliable information about students’ knowledge.
Arizona has submitted partial data addressing inclusion of all students, in particular students with disabilities (SWD) and limited English proficient (LEP) students, in the State assessment system. The evidence provided to date shows that one of the components of the assessment system (AIMS) is more inclusive than the other (SAT-9). Nevertheless, participation rates indicate that large numbers of students are not participating in the assessment or accountability system.
Given that one test is more inclusive than the other, Arizona needs to submit evidence showing how the entire assessment and accountability system meets the Title I requirements for inclusion of all students. Arizona must also submit participation data in the assessment and accountability systems for school year 2000-2001.
We also understand that your interim Title I accountability system does not include the results of out-of-level tests or modified assessments. Title I requires that the accountability system incorporate the results of alternate assessments, out-of-level tests, and modified assessments. For out-of-level assessments, results must be reported against standards for the grade in which the student is enrolled. Please submit to us evidence that your accountability system includes the results from such assessments.
Technical quality
Title I requires that State assessment be used for purposes for which such assessments are valid and reliable, and be consistent with professional and technical standards for such assessments. States must demonstrate that their tests are valid and reliable for school accountability purposes.
Arizona has not submitted the technical manuals for the SAT-9 or the third, fifth, and eighth-grade AIMS. We have obtained the technical manual for the high-school AIMS through your Department’s website. However, in order to complete our peer review of this portion of Arizona’s assessment system, Arizona needs to submit the remaining technical manuals.
Reporting
Title I requires that States provide individual student reports on the achievement of performance standards, and report the results for the State, local educational agencies, and schools disaggregated by gender, racial and ethnic group, English proficiency, migrant status, students with disabilities as compared to non-disabled students, and economically disadvantaged as compared to non-disadvantaged students.
Arizona’s original submission did not include disaggregated data in State-, district-, or school-level reports. Subsequently, the State submitted samples of one State-, district-, and school-level report for AIMS results that show partial disaggregation. Title I requires that Arizona provide evidence that student results are based on the complete assessment system, and that these results include disaggregated data for the categories required by Title I. Therefore, please submit evidence that your State reports the results of the complete assessments system as required by the statute.
As I discussed with former Superintendent Keegan, I do not believe that Arizona has far to go in fulfilling the Title I requirements, but our initial judgement was that it could not meet all the remaining requirement by the end of the current school year. I therefore recommended to her that Arizona apply for a short-term “timeline waiver” in order to give the State some additional months to attain compliance. Ms. Keegan responded that she believed your agency could submit sufficient additional evidence to merit full approval, and I invited her to do so. If that is still the Arizona Department of Education’s plan, I am asking that you provide us that evidence or clarification in the areas identified within 30 days.
We appreciate the time and effort you have put into the Title I assessment review and we look forward to working with you and other Arizona officials to help complete this process and clarify any outstanding issues. Upon review of the additional evidence, I will contact you to discuss the status and next steps. Please do not hesitate to contact me if you need further information.
Sincerely,
Thomas M. Corwin
Acting Deputy Assistant Secretary
