California Assessment Letter
January 19, 2001
Honorable Delaine A. Eastin
Superintendent of Public Instruction
California Department of Education
721 Capitol Mall
Sacramento, California 95814
Dear Superintendent Eastin:
I am pleased to have had an opportunity to work with you and your staff and with the Governor’s office over much of the last year to determine the progress that California is making toward meeting the Title I requirements for standards, assessments and accountability. I know that we all share the goal of having, as soon as possible, a full and functional assessment system that supports education improvement in California and that meets the Title I requirements. These requirements are intended to ensure that every school and school district participating in Title I is focused on improving results for all students, as defined by the State’s own academic standards.
After reviewing the assessment evidence and timelines submitted by California in May, and additional evidence that was reviewed in September 2000, it is clear that California is substantially out of compliance with sections 1111 and 1116 of Title I of the Elementary and Secondary Act of 1965. In order to remain eligible to receive Title I funds, California must enter into a compliance agreement with the U.S. Department of Education specifying the steps the State will take, and the timeline for completing them, in order to fully meet the Title I requirements.
We summarize below the major Title I requirements, our judgment of California’s compliance status, and the steps that must be addressed in order to meet the requirements. Many of the specific instances of noncompliance discussed below result from California’s failure to complete development and implementation of assessments that are aligned to State content and performance standards, and use the results of these tests to hold schools accountable for the performance of all students. Consequently, California can comply with many of these requirements if it follows through on its plan to develop and implement a standards-based assessment in a timely fashion, and if it pays careful attention to the specific Title I requirements in the process.
Page 2 – Honorable Delaine A. Eastin
However, based on our recent conversations, there are a number of issues in which it is not yet clear that California intends to fully comply with the Title I requirements. These include, for example, providing the full range of accommodations for the standards-based assessment that will maximize its validity for limited English proficient students, and providing an alternate assessment for students with disabilities who are unable to participate in the regular assessment. For these issues, California will remain out of compliance – and place Title I funding in jeopardy — unless it alters its current plans or provides more complete and compelling evidence that its current plans in fact meet the Title I requirements.
Final Assessment System: Title I requires that for the purposes of school accountability, States administer assessments yearly to students in at least reading/language arts and math, in three grade spans (between grades 3-5, 6-9 and 10-12). Title I also requires that State assessments be aligned to State content and performance standards, and that the assessment systems use multiple measures that assess higher order thinking skills and understanding.
California does not meet these requirements. Given the information you have provided to us, California’s assessment system cannot meet these Title I requirements until 2002 at the earliest, assuming that the work currently being planned is completed on the timetable you have laid out. Because the current plans call for assessments in mathematics, language arts, science and history/social sciences to be administered in grades 9-11 in 2002, the fact that the High School Exit Examination (HSEE) will not be included in the accountability system until 2004 does not further delay the prospects for meeting the Title I requirements.
California submitted for review information on the Stanford Achievement Test, Ninth Edition (SAT-9), and the California Standards-Based Test (which is the augmented
SAT-9) that will be implemented by 2002 in English language arts and mathematics in grades 2-11 and science and history/social science in grades 9-11. While valuable, the SAT-9 is not aligned State standards.
Based on the series of conversations we have had, we understand that California’s plan for a new standards-based assessment is still evolving, though the commitment among state education policymakers to standards-based assessment as the primary means for school accountability is growing. The current plan, as we understand it, will involve augmenting the SAT-9 with additional items aligned with State content standards as the basis for the standards-based assessments in reading and math to be initially administered
Page 3 – Honorable Delaine A. Eastin
in 2002. In subsequent years, however, the State will administer a standards-based test (hereafter referred to as “stand-alone test”) that is completely separate from the nationally normed test already being administered. Further, we understand that the California legislature is expected to take this issue up in the coming legislative session, as the underlying state testing statute will sunset and need to be reauthorized. We hope that this reauthorization process will keep California on the fastest possible track to develop and implement a high-quality standards-based assessment, as required by Title I.
Given the information submitted to the Department for review, California does not yet meet the requirement for multiple measures that assess higher order thinking skills and understanding. Title I does not require any particular mix of test item formats. It simply requires that the State clearly demonstrate that the mix of item formats it has chosen is aligned to State content standards and adequately measures the higher order skills incorporated into the State’s content standards. The assessment components that are being added may help California fully meet this requirement, but we could not make a judgment on that issue based on the information available to us.
Performance Standards: Title I required that each State submit evidence of performance standards that describe three levels of performance aligned to content standards for each of the three required grade spans by the 1997-98 school year.
California does not meet this requirement, and the earliest it can be met is Spring 2002. The performance standards in English-language arts will be set using data from the Spring 2000 administration and will be applied to the Spring 2001 administration of the assessments. Performance standards for mathematics will be set using data from the Spring 2001 administration of the California Standards-Based Tests. The mathematics performance standards will be applied to the Spring 2002 assessment.
Alignment: Title I requires that final assessments be aligned with content and performance standards in at least math and reading/language arts, as well as any other subject area in which a State has adopted standards.
California does not fully meet this requirement. Even the augmentation of the SAT-9 with additional items has left identified gaps between the content standards and the current assessments. California will need a clear plan to address these gaps.
Technical Quality: Title I requires that the State assessments be used for purposes for which such assessments are valid and reliable, and be consistent with relevant, nationally recognized professional and technical standards for such assessments.
Page 4 – Honorable Delaine A. Eastin
California does not meet this requirement because it did not provide evidence of technical quality for most of the components of its assessment system. The technical manual for the SAT-9 provided appropriate documentation for that portion of the assessment system. However, the reliability data provided for the augmented items were not sufficient. Information on the technical quality of the components of the California Standard-Based Assessments or the “stand-alone tests” is required as each is implemented. Information on technical quality and California’s plans for correcting any problems is particularly important given that the data provided for the augmentation items suggest low reliability for the math tests in grades 9-11.
The California narrative indicated that content review panels evaluate each item before it is used. However, no evidence was provided that this process ensures fairness and accessibility of the assessments. Moreover, no evidence was provided on the way the State will ensure that the assessment results are comparable for different schools and for different years, nor of systematic plans for reviewing and improving the assessments.
Inclusion of All Students in Assessments, Reporting and Accountability: Title I requires that final assessments provide for the participation of all students in the grades being assessed. Title I specifically requires the inclusion of limited English proficient (LEP) students in final assessments and makes clear that States must assess LEP students, to the extent practicable, in the language and form most likely to yield accurate and reliable information on what they know and can do in subjects other than English. Furthermore, Title I requires States to provide reasonable adaptations and accommodations for students with diverse learning needs, including LEP students and students with disabilities.
California does not yet meet these requirements. We recognize that California requires all students who have been in school for at least one year, regardless of English language proficiency, to be assessed with the SAT-9, and that this practice has been the topic of some controversy. However, because the SAT-9 is not aligned with California standards, it cannot be the primary test for Title I accountability purposes. Therefore, our primary focus is on the inclusion policies, practices and test validity for limited English proficient students and students with disabilities in any standard-based assessment to be developed, not the SAT-9 alone.
As we discussed, the State Board of Education recently adopted a policy that provides a broad range of accommodations in order to maximize the validity of the High School Exit Exam for limited English proficient students and students with disabilities. As you pointed out, these steps are particularly important to take for a high stakes test, and we agree. However, you indicated that the State Board does not plan to incorporate the same range of accommodations into the California Standard-Based Tests that will be
Page 5 – Honorable Delaine A. Eastin
administered for school accountability at earlier grade levels. Consequently, we are concerned that these tests used for Title I purposes may have limited validity for LEP students and students with disabilities even though the procedures needed to increase validity are readily apparent and available for use. Further, in California’s submission to the Department, no information was provided to establish the validity of the accommodations currently provided to California students, or planned for use with the standards-based assessment.
Title I requires all students, including limited English proficient students and students with disabilities, to be included in the State’s accountability system; the performance of these students may not be excluded from the determination of whether a school is making adequate progress toward helping its students meet State standards. Our interpretation of the participation data that you provided indicated that more than 900,000 students are excluded from your current system of accountability because scores on the SAT-9 were not valid, students were exempt, and students who attended small or charter schools were excluded. Simply put, the results of every student who takes an assessment must be used to determine school accountability. For the purposes of measuring school progress under Title I, results for students who participate in the standard administration of the State assessment, participate with accommodations, or take an alternate assessment must be reported and integrated into the State’s mechanism for evaluating schools. Furthermore, the number of students exempted from testing must be reported and should be integrated in the State’s accountability system.
California does not meet this requirement, in part, because of the relatively high rate of excluding students who have been tested from school accountability, and principally because the standards-based test has not been reported or used in the accountability system.
In addition, little information was provided on the California alternate assessment. No evidence was provided on the development of, participation in, success of, or reporting and use in the accountability program for the alternate assessment. Based on our conversations and evidence submitted, it is unclear what California has decided to develop as an alternate assessment for students with disabilities who are unable to participate in the standard assessment, though this is required both by Individuals With Disabilities Education Act (IDEA) and by Title I.
Further, no evidence was provided describing how performance of schools with fewer than 100 students will be evaluated as required by the alternative accountability system that is being developed for small schools and for implementation in the 2001-2002 school year.
Page 6 – Honorable Delaine A. Eastin
Title I requires that each state shall demonstrate that the state has developed or adopted a set of high quality, yearly student assessments that will be used as the primary means of determining the adequate yearly performance of each local educational agency and school served by Title I. Currently, California uses only the results from the SAT-9 for accountability purposes, not the standards-based assessment to be developed. Yet, Title I requires the use of standards-based assessments as the primary measure for determining adequate yearly progress for each school. No information was provided about how the proposed standards-based assessments and the SAT-9 will be weighted in the accountability system.
Title I also requires that assessments provide individual student interpretive and descriptive reports that let parents know how well their students are meeting the performance standards set by the State and that all participating LEAs produce individual school performance profiles for all their participating schools. California will not be able to meet this requirement until performance standards are set in 2002.
Assessment results are also required to be disaggregated within each State, local educational agency, and school. The Title I statute spells out the categories for reporting results by gender, major racial and ethnic groups, English proficiency status, and migrant status. It also requires that students with disabilities be compared to non-disabled students, and economically disadvantaged students be compared to students who are not economically disadvantaged. California disaggregates and reports by all required categories except for migrant status, and therefore does not yet fully meet this requirement.
Because of all of the above issues, California is required to enter into a compliance agreement with the Department of Education in order to remain eligible to receive Title I funds. A compliance agreement is a statutory remedy authorized by § 457 of the General Education Provisions Act for situations in which a state or local education agency cannot meet statutory requirements within the timeframe specified by law. Its purpose is to bring a grantee into full compliance with applicable requirements as soon as feasible, but no longer than three years. Attached is a summary of the issues based on the materials that were reviewed that the compliance agreement must address. This list may change as California clarifies its plans for the development and implementation of the standards-based assessments and responds to the issues raised.
Page 7 – Honorable Delaine A. Eastin
The Department and the State will need to agree on the components of the compliance agreement, and publish the final agreement. A detailed plan and specific timeline for how all of the steps necessary to bring the State into compliance will be accomplished must be submitted as a part of the negotiated agreement. Before entering into a compliance agreement:
- The State must be provided a hearing at which to demonstrate that full compliance is not feasible until a future date.
- Findings of noncompliance and the substance of the compliance agreement must be published in the Federal Register.
When the required steps have been completed, evidence that the assessment system meets the Title I requirements will be resubmitted for review by a panel of Peer Reviewers.
We are prepared to begin immediately to work with you to discuss the issues raised in this letter and to develop the details of the compliance agreement. We believe it is possible to complete work on this agreement within sixty days, and in any event, it is essential to complete this work by the spring administration of your assessment. While other issues related to California’s compliance with the requirements of federal education programs have at times remained unresolved for long periods of time, this issue is of such overriding importance that it must be resolved in a timely fashion. Otherwise, the Department is prepared to consider the full range of enforcement options at its disposal, including withholding a portion of Title 1 Part A program funds.
We look forward to working with California to support a high quality assessment system. If you would like to discuss this further, please do not hesitate to call Mitzi Beach, Group Leader for Compensatory Education Programs at 202-260-0823.
Sincerely,
Michael Cohen
Enclosure
OF/RF/JK1/19/2001
SUMMARY OF TITLE I REQUIREMENTS TO BE ADDRESSED
Based on what has been reviewed to date, California will need to provide or develop the following in order to meet the Title I requirements. This list may change as California clarifies its plans for the development and implementation of the standards-based assessments and responds to the issues raised.
The following issues are ones that California may be able to resolve in a timely fashion:
- Follow through on the development and implementation of all of the components of the final assessment system.
- Provide evidence as to how the final assessment system will meet the requirement for multiple measures that assess higher order thinking skills.
- Provide evidence of performance standards that describe three levels of performance that are aligned to content standards.
- Provide evidence about alignment and technical quality of the new standards-based assessments in English language arts and mathematics.
- Provide evidence describing how the performance of schools with fewer than 100 students is being evaluated using an alternative accountability system.
- Provide evidence that individual reports that contain information on the attainment of student performance standards are provided.
- Provide evidence that LEAs are completing school and district profiles that show student performance related to mastery of standards that include disaggregated results by all the required categories including migrant status.
It is not clear that California intends to fully comply with the following issues:
- Adopt accommodation policies for the standards-based assessment that increase validity and maximum participation.
- Provide evidence of the development of, participation in, success of, reporting and use of the accountability program for the California Special Education alternate assessment program.
- Incorporate into the accountability system data for students who take the alternate assessment and students administered assessments with non-standard accommodation.
- Participation of all students in the accountability system
- The standards-based assessments will be the primary basis for determining LEA and school progress when the standard-based assessment system is fully implemented.
