Montana Assessment Letter

May 18, 2006

Honorable Linda McCulloch
Superintendent of Public Instruction
Montana Office of Public Instruction
1227 11th Avenue
Helena, MT 59620-2501

Dear Superintendent McCulloch:

Thank you for submitting Montana’s assessment materials for review under the standards and assessment requirements of the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). We appreciate the efforts required to prepare for the peer review and hope that the process provides useful feedback that will support Montana’s efforts to monitor student progress toward challenging standards.

External peer reviewers and U.S. Department of Education (ED) staff evaluated Montana’s submission and found, based on the evidence received, that it did not meet all the statutory and regulatory requirements of Section 1111(b)(3) of the ESEA. I know that my staff has discussed the results of this review with your staff. However, I want to take this opportunity to enumerate the evidence that Montana must provide in order to have a fully compliant standards and assessment system under NCLB. That evidence is listed on the last pages of this letter.

I urge you to submit any available evidence demonstrating how Montana’s system meets the standards and assessment requirements as soon as possible. I also request that, as soon as possible, you provide us a plan with a detailed timeline for how Montana will meet any remaining requirements for which evidence is not currently available. After reviewing those materials, I will then determine the final approval status for Montana’s standards and assessment system.

Enclosed with this letter are detailed comments from the peer review team that evaluated Montana’s assessment materials. The peer reviewers, experts in the areas of standards and assessment, review and discuss a State’s submission of evidence and prepare a consensus report. I hope you will find the reviewers’ comments and suggestions helpful. I remind you of our offer to provide you further technical assistance at your request.

We look forward to working with Montana to support a high-quality assessment system. If you would like to discuss this further or would like to request technical assistance, please do not hesitate to call Carlos Martínez (202-260-2493) or Catherine Freeman (202-401-3058) of my staff.

Sincerely,

Henry L. Johnson

Enclosures

cc: Governor Brian Schweitzer

Summary of Additional Evidence that Montana Must Submit to Meet ESEA Requirements for the Montana Comprehensive Assessment System (MontCAS)

1.0 – ACADEMIC CONTENT STANDARDS

• Evidence that it has approved content and skill expectations for each grade, 3-8 and 10.
• Evidence of how its standards were developed to ensure that they contain rigorous content and encourage the teaching of advanced skills or provide evidence of standards analysis (such as Depth of Knowledge analysis) that address the rigor and challenge of the content expectations in reading and mathematics in grades 3, 5, 6, and 7 and the content expectations in science in grades 4, 8, and 10.
• The state should provide evidence that the committees of stakeholders involved in the standards development process adequately represented the diversity of the State’s student population.

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

• Documentation of completed achievement descriptors for grades 3, 5, 6, and 7.
• Evidence that it has alternate academic achievement standards (achievement level descriptors and cut scores) that can be applied to each tested grade.
• Evidence of the development of alternate achievement descriptors in science for the alternate assessment.
• A specific plan and timeline for setting and approving cut scores in reading and mathematics on the general assessments and the alternate assessments in grades 3, 5, 6, and 7.

3.0- FULL ASSESSMENT SYSTEM

• Evidence requested in section 1.0 (Content Standards) regarding rigor and challenge, and section 5.0 (Alignment) would satisfy this section.

4.0 – TECHNICAL QUALITY

• Test blueprints with sufficient detail to support the creation of substantively comparable test forms that are also aligned with the content and skill aspects of its standards.
• A plan for evaluating how the results from the statewide assessments are used to evaluate and improve curriculum and instruction, evaluating the degree to which CRT and CRT-Alt results converge with measures of similar constructs and diverge from measures of different constructs, and for identifying and evaluating unintended consequences associated with its statewide assessment system.
• Evidence that it has evaluated the inter-rater agreement and consistency for constructed-response items on the CRT and CRT-Alt. This evidence should include a clear description of the scoring processes for constructed-response items on the CRT.
• A clear articulation of how accommodations must be aligned with instructional approaches a student experiences in the classroom and how the state will monitor the use of accommodations at the time of testing.
• A clarification of how translated and “Sheltered English” versions of its tests are developed and administered, the eligibility criteria for these test versions, and evidence that these test versions yield results that are comparable to those from other test versions. In addition, the state should submit a plan for evaluating the meaning of scores from translated and “Sheltered English” versions of the tests and for ensuring the comparability of these scores with scores from non-accommodated test conditions.

5.0 – ALIGNMENT

• A plan for addressing the gaps and weaknesses alignment cited in the 2005 external alignment study; and
• An impartial study that evaluates the quality of alignment among its reading and mathematics standards and assessments in grades 3, 5, 6, and 7, and among its science standards and assessments in grades 4, 8, and 10. These plans should address the degree to which its assessments yield scores that adequately reflect the full range of performance defined in the performance level descriptors.
• Evidence that the CRT-Alt extended content standards are linked to content standards at each grade level, 3-8 and 10, and will yield scores aligned with grade-specific expectations applicable to each tested grade.

6.0 – INCLUSION

• Provide enrollment and number tested data for each student group within each grade and content area.
• Separate lists of allowable accommodations for students with IEPs, students with Section 504 plans, and English language learners to help ensure that accommodations are aligned with students’ specific, individual, academic needs. These lists must provide a clear distinction among which accommodations are allowable for students with IEPs, are allowable for students with Section 504 plans, and are allowable for English language learners.

7.0 – REPORTING

• Provide participation data for each grade level and content area as addressed in section 6.0.

Return to state-by-state listing