Oregon Assessment Letter

January 25, 2006

Honorable Susan Castillo
Superintendent of Public Instruction
Oregon Department of Education
255 Capitol Street NE
Public Service Building
Salem, Oregon 97310

Dear Superintendent Castillo:

Thank you for submitting Oregon’s assessment materials for review under the standards and assessment requirements of the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). We appreciate the efforts required to prepare for the peer review and hope that the process provides useful feedback that will support your State’s efforts to monitor student progress toward challenging standards.

External peer reviewers and U.S. Department of Education (ED) staff evaluated Oregon’s submission and found, based on the evidence received, that it did not meet most of the statutory and regulatory requirements of Section 1111(b)(3) of ESEA. Additional evidence is needed to show how Oregon meets the following critical elements of the NCLB standards and assessment peer review guidance:

Academic Achievement Standards (Element 2.0): Oregon does not meet this requirement. It appears that current academic achievement standards were established before the academic content standards. It is essential that the academic achievement standards be developed based on the current (existing) grade-level content standards.

Limiting the alternate assessments to K-2 content standards must be revised to reflect the changes in academic content that would be expected across grades to ensure appropriate challenge for older and more capable students. Oregon must conduct and document standard-setting studies for both its regular and alternate assessments.

Full Assessment System (Element 3.0): Oregon’s assessment system includes reading/literature and mathematics at grades 3-8 and the Certificate of Initial Mastery (CIM) at grades 9-12, in addition to alternate assessments. The four content area tests are referred to as “Knowledge and Skills Tests”. Technology Enhanced Student Assessment (TESA) is an adaptive, web-based approach that is available to schools in place of the paper/pencil Knowledge and Skills Tests. Additional evidence must be submitted to support the comparability of test scores (e.g., plain language and regular test forms) and consistency of achievement level definitions (e.g., paper/pencil and adaptive modes).

Technical Quality (Element 4.0): Oregon does not meet this requirement. Remaining technical quality issues include: (1) standard setting for regular and alternate assessments; (2) comparability of paper-and-pencil and computer-adaptive tests between school years; (3) reliability and validity of alternate assessments; (4) comparability of adaptive tests to each other and to paper/pencil versions at the achievement levels and grades; and (5) document that only academic achievement scores not life skills count toward AYP. When possible, it is suggested that the additional evidence be organized and submitted as technical manuals.

Alignment(Element 5.0): Oregon does not meet this requirement. Additional evidence is needed related to the alignment of Knowledge and Skills Tests with the academic content standards and the re-established academic achievement standards. Also evidence is needed to support the alignment/linkages of the Oregon alternate assessments to the State’s academic content standards and the re-established academic achievement standards.

Reporting (Element 7.0): Oregon partially meets this requirement. The State met most of the requirements under this section; however, performance level descriptions were not located on the student/parent reports and no parent reports were submitted for the extended assessments or CLRAS. Alternate assessment performance ratings were not tied to achievement levels used for NCLB reporting.

Because Oregon’s standards and assessment system meets some, but not all, of ESEA’s statutory and regulatory requirements, and because, based on consultations with your staff, ED expects that Oregon can take the necessary steps to come into full compliance, the status of the Oregon assessment system is Final Review Pending. In this status, a State must clearly articulate to ED how it will meet the remaining requirements and be able to fully implement its standards and assessment system by the end of the 2005-2006 school year.

More detailed information of the additional evidence that Oregon must submit to demonstrate compliance with NCLB is given beginning on page four. Oregon must submit to ED as soon as possible its plan for coming into full compliance and the timeframe for submitting additional evidence of compliance. When the required additional evidence has been submitted, it will be subject to peer and ED staff review.

A second peer review will need to be scheduled consistent with the previously announced dates. Enclosed with this letter are detailed comments from the peer review team that evaluated the Oregon assessment materials. This panel of standards and assessments experts review and discuss a State’s submission of evidence and prepare a consensus report that is documented as the Peer Notes. I hope you will find the reviewers’ comments and suggestions helpful.

We look forward to working with Oregon to support a high-quality assessment system. If you would like to discuss this further, please do not hesitate to call David J. Harmon (202-205-3554) or Catherine Freeman (202-401-3058) of my staff.

Sincerely,

Henry L. Johnson

Enclosures

cc: Doug Kosty

Summary of Additional Evidence that Must be Submitted to Meet NCLB Requirements for the Oregon Assessment System

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

Additional evidence including approved, re-established academic achievement standards that show alignment to the State’s grade level content standards with technical and stakeholder participation.
Additional evidence including approved, re-established alternate academic achievement standards appropriately linked to Oregon’s content standards.
Additional evidence that all students are tested on academic content standards, not just on life skills.
Document Oregon State Board of Education adoption of academic performance descriptors in science.

3.0 – FULL ASSESSMENT SYSTEM

Document the consistency in achievement level definitions for adaptive and paper/pencil modes.
Additional evidence supporting the comparability of tests based on the extended content standards and the Extended Career and Life Role Assessment (CLRAS).
Additional evidence supporting the comparability of the plain language and regular test forms.
Additional evidence supporting the comparability of the Juried assessments and the Knowledge and Skills Tests.
Additional evidence supporting the comparability of Spanish and Russian side-by-side translations with English versions.

4.0 – TECHNICAL QUALITY

Additional evidence for each assessment, including alternate assessments, that documents the standard setting process with descriptions of the selection of judges, methodology employed, and final results.
Additional evidence that adaptive tests are comparable to each other and paper/pencil versions at the achievement levels, restricted to grade-level content, and matched to detailed grade level test blueprints.
Additional evidence that documents consistency of strand content among the paper-and-pencil and computer-adaptive versions that includes detailed test form construction rules and test maps.
Additional evidence supporting the comparability of paper-and-pencil and computer-adaptive test difficulties between school years.
Documentation that CLRAS (life skills) scores do not count for AYP either alone or in combination with extended assessment scores.
Documentation that supports the reliability and validity of alternate assessments.

5.0 – ALIGNMENT

Document the alignment of 3-8 and high school assessments in reading/language arts and mathematics with academic content standards and with the re-established academic achievement standards.
Document the alignment/linkages of the Oregon alternate assessments to the State’s academic content standards and to re-established academic achievement standards.

7.0 – REPORTING

Document that performance level descriptions appear on all student/parent reports.
Document the existence of parent reports for extended assessments and for CLRAS.
Document that alternate assessment performance ratings are tied to NCLB achievement levels used for reporting.

Return to state-by-state listing

Top