Alabama Assessment Letter 2
November 19, 2001
Honorable Edward R. Richardson
Superintendent of Education
Alabama Department of Education
Gordon Persons Office Building
50 North Ripley Street
P.O. Box 302102
Montgomery, Alabama 36130
Dear Superintendent Richardson:
It was a pleasure meeting with your staff on July 12. We appreciate Alabama’s continued efforts to provide evidence of meeting the Title I assessment requirements. However, after reviewing the additional evidence submitted by Alabama, we find that most aspects of your State’s assessment system still do not meet the requirements of sections 1111(b) and 1116(a) of Title I of the Elementary and Secondary Act of 1965. As you know, Congress adopted these requirements as part of the major overhaul of Title I in 1994. The statute required each State to implement a system of challenging content and performance standards, aligned assessments, and school accountability by the 2000-2001 school year.
I know that we both share the goal of having, as soon as possible, a full and functioning assessment system that supports education improvement in Alabama and that meets the Title I requirements. These requirements are intended to ensure that every school and school district participating in Title I is focused on improving results for all students, as defined by the State’s own academic standards. We applaud your willingness to make significant changes in your current assessment system to meet the Title I requirements and to address the needs of all the children in Alabama. We will continue to work cooperatively with Alabama to fulfill the remaining requirements in a timely fashion.
The following paragraphs convey our understanding of where Alabama stands with regard to meeting the various assessment requirements under Title I.
Final Assessment System: Title I requires that, for the purposes of school accountability, States administer assessments yearly to students in at least reading/language arts and math in three grade spans (grades 3-5, 6-9 and 10-12). Title I also requires that State assessments be aligned with State content and performance standards, and that the assessment systems use multiple measures that assess higher-order thinking skills and understanding.
Page 2 – Honorable Edward R. Richardson
Alabama partially meets these requirements. Alabama meets the requirement of at least one grade assessed in each of the three grade spans and also the two subject areas because the State uses the SAT-9 scores in reading and mathematics in grades 3-11 for accountability purposes. After the passage of its amended assessment statute, Alabama immediately included the use of a writing assessment in grades 5 and 7 and the use of the Alabama High School Graduation Exam (AHSGE) in the Alabama Student Assessment Program and accountability system for 2001 and 2002. Alabama intends to use criterion-referenced tests in at least reading and language arts in grades 2, 3, 5, and 7. However, your State has not yet developed these assessments or submitted a timeline to show when each of these proposed components will be implemented.
Performance Standards: Title I requires that each State submit evidence of performance standards that describe three levels of performance aligned with content standards for each of the three required grade spans by the 1997-98 school year.
Alabama has not met this requirement. Your State has developed performance descriptors, but has not developed cut scores to define performance standards on an aligned assessment. Alabama developed performance standards for the AHSGE in the summer of 2000 and your State Board of Education adopted these in July 2000. However, the performance standards for the AHSGE do not meet the Title I requirements since they have only two levels of performance, minimally passing and not passing.
Alignment: Title I requires that final assessments be aligned with content and performance standards in at least math and reading/language arts, as well as any other subject area in which a State has adopted standards.
Alabama does not meet this requirement. The test publishing company used by your State completed an analysis of the alignment of the SAT-9 and the Alabama Course of Study (COS) and identified a number of gaps between the assessments and the standards. Alabama has not provided a description of how it will address these gaps. Moreover, this alignment analysis did not provide information on the depth and match of the SAT-9 with the State’s performance standards, or on the cognitive complexity of the SAT-9 compared with Alabama’s content standards.
You included the standards, objectives, eligible content, and course of study for the AHSGE in your submission. However, your documents did not articulate the alignment between the high school assessment objectives and the high school COS. Also, you did not provide a description of the process for ensuring or evaluating alignment of the high school assessment with its own objectives.
Alabama has incorporated the Alabama Direct Assessment of Writing in grades 5 and 7 into your system. However, you have not provided evidence of how the writing assessment contributes to the alignment in the lower two grade spans or evidence of how your State will evaluate overall alignment in these two grade spans. You have not provided a description of your approach for ensuring alignment, data on the depth and match of the your assessments with your content and performance standards, information on the cognitive complexity of your assessments, or information on the alignment of the proposed criterion-referenced assessments.
Page 3 – Honorable Edward R. Richardson
Technical Quality: Title I requires that the State assessments be used for purposes for which such assessments are valid and reliable, and be consistent with relevant, nationally recognized professional and technical standards for such assessments.
Alabama partially meets this requirement. The technical quality of the SAT-9, as used for the purposes for which it was designed, appears to be adequate. You have provided documentation that the test is consistent with professional standards.
You provided a technical report for the AHSGE, which we reviewed and accepted. However, you have not provided a technical manual for the Alabama Direct Assessment of Writing in grades 5 and 7. We received some statistical information about the writing examination, but we could not accept it as complete technical information. You have not provided technical information on the proposed criterion-referenced assessment.
Inclusion of All Students in Assessments and Accountability: Title I requires that States’ final assessments provide for the participation of all students in the grades being assessed. The statute specifically requires the inclusion of limited English proficient (LEP) students in final assessments and makes clear that States must assess LEP students, to the extent practicable, in the language and form most likely to yield accurate and reliable information on what they know and can do in subjects other than English. Furthermore, Title I requires States to provide reasonable adaptations and accommodations for students with diverse learning needs, including LEP students and students with disabilities (SWD).
Alabama partially meets these requirements. Your State did not supply adequate participation data for LEP students and the overall population. You did not provide evidence that the accommodations associated with the norm-referenced tests (NRTs) and the high school writing assessments yield valid results for SWDs, or detailed information regarding any accommodations that you plan for the Alabama Direct Assessment of Writing in grades 5 and 7 and the technical quality of those accommodated assessments.
Alabama uses an alternate assessment for certain students with disabilities. However, your reporting system does not report summary data for the Alabama Alternate Assessment for local educational agencies (LEAs) with fewer than 10 such students. Your State has not submitted information on how these data will be used in the accountability system, or technical quality information on the instrument and the scores generated from it. Your State has not provided evidence of an alternate assessment for grades 9-12.
Page 4 – Honorable Edward R. Richardson
Alabama has not provided evidence of a comprehensive policy on assessment guidelines and accommodations for LEP students or a plan for implementing the LEP inclusion policies and for monitoring LEA compliance with the inclusion policies.
Reporting: Title I also requires that assessments provide individual student interpretive and descriptive reports that let parents know how well their students are meeting the performance standards set by the State and that all participating LEAs produce individual school performance profiles for all their participating schools.
Alabama provides individual reports of student achievement on the SAT-9. Since the State does not have cut scores for performance standards, individual student reports are now based on raw scores, national percentile ranks, and national normal curve equivalent (NCE) scores, which do not allow for an assessment of student performance relative to the State performance standards.
Alabama provided individual reports of student achievement on the AHSGE. These reports give data by objective, item total, items correct, and mastery in language, science, and math. Since your system defines performance standards only for minimally passing or not passing, it does not allow a determination of the full range of performance.
Reporting of Disaggregrated Data:Title I requires that assessment results be disaggregated within each State, local educational agency, and school. The statute specifically requires the reporting of results by gender, major racial and ethnic groups, English proficiency status, and migrant status. It also requires that students with disabilities be compared to non-disabled students, and economically disadvantaged students be compared to students who are not economically disadvantaged.
Alabama does not meet this requirement. Your system does not report the performance of economically disadvantaged students versus the performance of non-economically disadvantaged students, or disaggregrated scores by race/ethnicity and LEP status at the State, LEA, or school levels. Your school profiles do not show how groups of students are performing against performance standards aligned with content standards, as required by the statute. Moreover, your system does not include the results of the alternate assessment in its reporting. Also, you have not provided information on how you disseminate these Title I performance reports and communicate information to all stakeholders. You have not provided evidence that you report data from the AHSGE by school or district, nor have you provided us with any disaggregated reports.
Page 5 – Honorable Edward R. Richardson
Alabama needs to develop a plan and a timeline to address the following issues identified in the review of your final assessment submission:
- Evidence supporting the proposed Alabama Student Assessment Program that includes information on the financial capacity of Alabama to complete this system so that it meets the requirements of Title I, including performance standards, alignment, technical quality, inclusion of all students, reporting, and use in the State’s accountability system.
- Evidence on how you will include multiple measures in your State’s current assessment system, especially in mathematics especially in your current mathematics assessment and in all components of your proposed criterion-referenced assessment, and an explanation of the effect of including multiple measures on the validity, reliability, and fairness of your assessments.
- Evidence of performance standards having three performance levels, with cut scores for all components of the assessment system, and the process you will use to determine that these performance standards are aligned with content standards and performance descriptors for all components of the assessment system incorporated into your State’s accountability system.
- Evidence of participation rates for each grade assessed, each subject (reading and math), and, for SWD and LEP populations, the total enrollment, number assessed, and number exempted. The number assessed should be broken down by types of assessment accommodation (regular, standard accommodations, non-standard accommodations, and alternate) for all components of the State assessment system that you will include in the accountability system.
- Evidence of an approved comprehensive policy on assessment guidelines and accommodations for LEP students, clear guidance to LEAs and schools related to the use of language proficiency tests for the LEP team decisions on accommodations for assessments, and a plan for implementing the new LEP inclusion policies and for monitoring LEA compliance with those policies.
- Evidence on the process used to incorporate data for SWD and LEP students into the assessment and accountability systems.
Page 6 – Honorable Edward R. Richardson
- Evidence regarding the extent to which all components of the Alabama assessment program contribute to the alignment of the content and performance standards, especially the contribution of the writing component and the relationship of the SAT-9 to the proposed criterion-referenced tests; a description of your State’s approach for ensuring alignment; and information on the cognitive complexity of all of the Alabama assessments.
- Evidence of a technical manual for the writing component and technical information on all the proposed components when they are available.
- Evidence to show how Alabama will disaggregate its performance data in grade spans 3-5 and 6-9 by economically disadvantaged students versus non-economically disadvantaged, race/ethnicity, and LEP status at the State, LEA, and school levels and on how Alabama will disaggregate its performance data by all the required categories at the high school level.
- Evidence on how your agency will provide individual student reports and State, LEA, and school profiles by student performance standards and how it will report and disseminate student performance information to the necessary stakeholders at the LEA and school levels.
Because of the above issues, Alabama must enter into a compliance agreement with the Department of Education in order to remain eligible to receive Title I funds. A compliance agreement is a statutory remedy authorized by section 457 of the General Education Provisions Act for situations in which a State or local educational agency cannot meet statutory requirements within the timeframe specified by law. Its purpose is to bring a grantee into full compliance with applicable requirements as soon as feasible, but within three years.
This Department and your State will need to agree on the components of the compliance agreement, including a detailed plan and specific timeline for how Alabama will accomplish the steps necessary to bring the State into compliance. In addition, before entering into a compliance agreement, a State must hold a hearing to demonstrate that full compliance is not feasible until a future date, and the Department must publish findings of noncompliance and the substance of the compliance agreement in the Federal Register.
Page 7 – Honorable Edward R. Richardson
We are prepared to begin immediately to work with you to discuss the issues raised in this letter and to develop the details of the compliance agreement. We appreciate your willingness to look at the parts of your assessment system that you have not used for accountability purposes, and we appreciate the positive action that you have already taken in starting to plan your State’s new standards-based assessment system that will be aligned with your State standards.
If you would like to talk further about this, please do not hesitate to call me.
Sincerely,
Susan B. Neuman, Ed.D
Enclosure
