Florida Assessment Letter dated June 11, 2008
June 11, 2008
The Honorable Eric J. Smith
Commissioner
Florida Department of Education
325 West Gaines Street
Tallahassee, Florida 32399
Dear Commissioner Smith:
I am writing regarding the U.S. Department of Education’s (Department) recent review of Florida’s new alternate assessment based on alternate academic achievement standards for students with the most significant cognitive disabilities, the Florida Alternate Assessment (FAA). The FAA is part of Florida’s standards and assessment system under Title I of the Elementary and Secondary Education Act of 1965 (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB), that we approved on June 27, 2007. We appreciate the efforts required to prepare for the peer review and hope that the process provides useful feedback that will support Florida’s efforts to monitor student progress toward meeting challenging academic standards.
External peer reviewers and Department staff evaluated Florida’s submission and concluded that the evidence received indicates that the FAA and, thus, Florida’s standards and assessment system, do not meet all the statutory and regulatory requirements of section 1111(b)(1) and (3) of the ESEA. Specifically, we have concerns with the technical quality of the FAA, the alternate academic achievement standards, and the coherence of the FAA. I know that my staff has discussed the results of this review with your staff. The complete list of evidence needed to address our concerns is enclosed with this letter.
Because the FAA is being administered for the first time in spring 2008, we know that Florida is unable to provide additional evidence regarding the compliance of the FAA with ESEA requirements. Therefore, we are not assigning an approval status to Florida’s assessment system at this time. Because the FAA has not fully met the ESEA requirements, however, we will place a condition on your fiscal year 2008 Title I, Part A grant award.
To ensure that all remaining work occurs in a timely manner, I request that, within two weeks of your receipt of this letter, you provide my staff with a detailed timeline for how and when Florida will satisfy the remaining ESEA requirements. As part of that timeline, please indicate when you will submit evidence, which we would like to receive as soon as it becomes available. We will review that evidence and schedule an additional peer review as necessary.
Given that Florida’s assessment system was approved in 2006-07 and the state has previously demonstrated its ability to meet all standards and assessment requirements, Florida remains eligible to participate in Departmental pilots, for one year only (i.e., pilots that will be implemented in connection with AYP determinations based on the 2007-08 assessments). If the state’s assessment system as administered in 2008-09 is not fully compliant, Florida will be required to suspend participation in any Departmental pilot until its system is fully approved.
We look forward to working with Florida to support its development and implementation of a high-quality standards and assessment system. If you would like to discuss this further, please do not hesitate to contact Sharon Hall (Sharon.Hall@ed.gov) or Patrick Rooney (Patrick.Rooney@ed.gov) of my staff.
Sincerely,
Kerri L. Briggs, Ph.D.
Enclosure
cc: Governor Charlie Crist
Cornelia Orr
SUMMARY OF ADDITIONAL EVIDENCE THAT FLORIDA MUST SUBMIT TO MEET ESEA REQUIREMENTS FOR THE FLORIDA STANDARDS AND ASSESSMENT SYSTEM
2.0 – ACADEMIC ACHIEVEMENT STANDARDS
- Documentation of the alternate academic achievement standards, including cut scores, performance level descriptors for each of the performance levels (basic, proficient, and advanced) within each access point (participatory, supported, and independent) for reading and mathematics.
- Evidence that alternate academic achievement standards have been formally adopted by the state.
- Documentation of the academic achievement standards-setting panels, including evidence of the inclusion of individuals with knowledge of special education and individuals with content expertise for all content areas and grades.
3.0 – FULL ASSESSMENT SYSTEM
- 1. Evidence of a coherent assessment system, including the design of the current alternate assessment based on alternate academic achievement standards.
4.0 – TECHNICAL QUALITY
- 1. The completed technical manual for the FAA, including evidence of validity and reliability, as described in the Standards for Educational and Psychological Testing, and the standards-setting process and results.
- Evidence that, in its consideration of validity, Florida considered how the access limitations inherent in its test design might pose a significant constraint on the meaningfulness of assessment scores.
- The final standards-setting report.
- Evidence that students’ responses are scored with fidelity to the rubric and scored accurately.
- A plan to regularly monitor the administration and scoring procedures.
- Evidence related to the quality of score analysis and reporting processes.
- A process for examining and improving the quality of Florida’s alternate assessment system over time.
5.0 – ALIGNMENT
- Evidence of alignment among the state’s academic content standards, alternate academic achievement standards, and alternate assessments, including:
- a. Evidence that addresses the comprehensiveness, cognitive complexity, and balance aspects of alignment quality among the state’s standards and its alternate assessment;
- Evidence that addresses alignment quality given the fact that construct coverage differs across access points within each item;
- Evidence that addresses alignment quality given the fact that different students take different sets of items within a test form;
- Evidence that addresses the degree to which the assessment reflects the range of achievement defined in the performance level descriptors and the access points;
- Specific recommendations for improving alignment quality; and
- The state’s plan for addressing any identified gaps and weaknesses in alignment.
7.0 – REPORTING
- Evidence that the state’s reporting system facilitates appropriate, credible, and defensible interpretation and use of its alternate assessment data.
- Evidence that, for each content area and grade tested, the state provides a summary report that includes the number of students enrolled or number tested/not tested as evidence that all students are tested.
- Evidence that the state reports participation and assessment results for all students and for each of the required subgroups in its reports at the school, district, and state levels.
- Evidence that the state has provided for the production of individual interpretive, descriptive, and [non-clinical] diagnostic reports that indicate relative strengths and instructional needs:
- a. Evidence that these individual student reports express results in terms of the State’s alternate academic achievement standards rather than numerical values such as scale scores or percentiles;
- Evidence that these individual student reports provide information for parents, teachers, and principals to help them understand and address a student’s specific academic needs. This information must be displayed in a format and language that is understandable to parents, teachers, and principals, for example, through the use of descriptors that describe what students know and can do at different performance levels. The reports must be accompanied by interpretive guidance for these audiences; and
- Evidence that the state ensures that these individual student reports will be delivered to parents, teachers, and principals as soon as possible after the alternate assessment is administered.
- Evidence that the state has ensured that student-level assessment data are maintained securely to protect student confidentiality.
- Evidence that the state has provided for the production of itemized score analyses by subdomains or standards (item-by-item reports not required) so that parents, teachers, and principals can interpret and address the specific academic needs of students.
