Montana Assessment Letter

October 3, 2006

The Honorable Linda McCulloch
Superintendent of Public Instruction
Montana Office of Public Instruction
1227 11th Avenue
Helena, Montana 59620-2501

Dear Superintendent McCulloch:

Thank you for submitting additional assessment materials for peer review under the standards and assessment requirements of the Elementary and Secondary Education Act of 1965 (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). We appreciate the efforts required to prepare for the peer review and hope that the process provides useful feedback that will support Montana’ efforts to monitor student progress toward challenging standards.

In my letter to you on June 22, 2006, I enumerated the fundamental components as well as a number of additional technical issues that had not met the standards and assessment requirements of the ESEA after the first peer review in February 2006. Specifically, the Department could not approve Montana’ standards and assessment system due to fundamental concerns with the development of general and alternate content and achievement standards, technical quality, including inter-rater agreement and appropriate accommodations, and alignment of the State assessments to grade-level content standards and the alternate assessment based on alternate achievement standards (the alternate Criterion-Referenced Tests, or CRT-Alt). The peer reviewers and Department staff evaluated Montana’s additional evidence and found, based on the evidence received, that it still does not meet all the statutory and regulatory requirements of Section 1111(b)(1) and (3) of the ESEA. Specifically, the peer review of this evidence suggests that there remain concerns regarding the technical quality, including inter-rater agreement and appropriate use of accommodations, and alignment of the State assessments, including the CRT-Alt, to grade-level content standards. However, Montana’s new evidence is sufficient to address concerns with the development of general and alternate content and achievement standards.

Because Montana’s additional submission of evidence did not resolve all outstanding issues, the status of its standards and assessment system remains Approval Pending. However, due to the fact that the State was able to show cause that it had satisfactorily addressed one of the three fundamental components listed in my letter of June 22, 2006, and because Montana has joined the LEP Partnership, the Department will not withhold Title I, Part A administrative funds for fiscal year 2006. Please note, however, that Montana must address all outstanding issues in order to have a fully compliant standards and assessment system under the ESEA by the end of the 2006-07 school year. Please refer to the enclosure for a detailed list of the evidence Montana must still submit to meet the requirements for an approved standards and assessment system.

Because the status of Montana’s standards and assessment system is still Approval Pending, Montana continues to be under Mandatory Oversight, as authorized under 34 C.F.R. §80.12. Under this status, we placed specific conditions on Montana’ fiscal year 2006 Title I, Part A grant award. In addition, Montana must provide, not later than 20 business days from receipt of this letter, a plan and detailed timeline for how it will meet the remaining requirements to come into full compliance by the end of the 2006-07 school year. Beginning in November 2006, Montana must also provide bi-monthly reports on its progress implementing the plan. If, at any time, Montana does not meet the timeline set forth in its plan, the Department will initiate proceedings, pursuant to Section 1111(g)(2) of the ESEA, to withhold 15 percent of Montana’s fiscal year 2006 Title I, Part A administrative funds, which will then revert to local educational agencies in Montana.

I appreciate the steps Montana has taken toward meeting the requirements of the ESEA, and I know you are anxious to receive full approval of your standards and assessment system. We are committed to helping you get there. I also appreciate your willingness to join the LEP Partnership; we believe that this is the first step in providing Montana assistance in appropriately assessing the growing population of English language learners. We also remain available to provide technical assistance regarding other issues that you identify. We will schedule an additional peer review when you have evidence available to further evaluate your system. If you have any questions or would like to discuss this further, please do not hesitate to contact Tim Vansickle (Timothy.Vansickle@ed.gov) or Patrick Rooney (Patrick.Rooney@ed.gov) of my staff.

Sincerely,

Henry L. Johnson

Enclosure

cc: Governor Brian Schweitzer
Nancy Coopersmith
Judy Snow

<h2Summary of Additional Evidence that Montana Must Submit to Meet ESEA Requirements for the Montana Assessment System

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

1. Evidence of the development of alternate achievement descriptors in science for the alternate assessment.

3.0 – FULL ASSESSMENT SYSTEM

1. Evidence requested in section 5.0 (Alignment) would satisfy this section.

4.0 – TECHNICAL QUALITY

1. Test blueprints with sufficient detail to support the creation of substantively comparable test forms that are also aligned with the content and skill aspects of its standards.
2. Evidence that it has evaluated the inter-rater agreement for constructed-response items on the CRT-Alt.
3. A clear articulation of how accommodations must be aligned with instructional approaches a student experiences in the classroom and how the state will monitor the use of accommodations at the time of testing.
4. A clarification of how translated and “Sheltered English” versions of its tests are administered and the eligibility criteria for these test versions.

5.0 – ALIGNMENT

1. A plan for addressing the gaps and weaknesses in alignment cited in the 2005 external alignment study.
2. An impartial study that evaluates the quality of alignment among its reading and mathematics standards and assessments in grades 3, 5, 6, and 7, and among its science standards and assessments in grades 4, 8, and 10. These plans should address the degree to which its assessments yield scores that adequately reflect the full range of performance defined in the performance level descriptors.
3. Evidence that the CRT-Alt extended content standards are linked to content standards at each grade level, 3-8 and 10, and will yield scores aligned with grade-specific expectations applicable to each tested grade.
4. Evidence of how its content standards were developed to ensure that they contain rigorous content and encourage the teaching of advanced skills or provide evidence of standards analysis (such as Depth of Knowledge analysis) that address the rigor and challenge of the content expectations in reading and mathematics in grades 3, 5, 6, and 7 and the content expectations in science in grades 4, 8, and 10.

6.0 – INCLUSION

1. Provide enrollment data and number tested for each student group within each grade and content area.
2. Separate lists of allowable accommodations for students with individualized education programs (IEPs), students with Section 504 plans, and English language learners (ELLs) to help ensure that accommodations are aligned with students’ specific, individual academic needs. These lists must provide a clear distinction among which accommodations are allowable for students with IEPs, students with Section 504 plans, and ELLs.

7.0 – REPORTING

1. Provide participation data for each grade level and content area as addressed in section 6.0.

Return to state-by-state listing