Montana Assessment Letter
July 6, 2001
Honorable Linda McCulloch
Superintendent of Public Instruction
Montana Office of Public Instruction
1227 11th Avenue
Helena, MT 59620-2501
Dear Superintendent McCulloch:
It was a pleasure speaking with you and your staff regarding the findings from this Department’s review of Montana’s final assessment system under the requirements of Title I of the Elementary and Secondary Education Act. We appreciate your efforts to prepare for the final review and hope that the process provides useful feedback that will support your State’s monitoring of student progress toward challenging standards.
Working for the Department, a group of external reviewers with strong expertise and experience in the design of State assessment systems reviewed the documentation your agency provided on Montana’s assessment system against the requirements of the Title I statutes. The reviewers raised a number of issues related to the substance and timeline for completion of your State system. Their comments are enclosed.
The statute requires each State to implement a system of challenging content and performance standards, aligned assessments, and school accountability for the 2000-2001 school year. As we discussed, a number of aspects of the Montana assessment system that still do not meet the requirement of sections 1111(b)(3) and 1116(a) of Title I. Montana must, therefore, do the following to meet Title I requirements.
- Provide evidence that Montana’s performance standards are aligned with your State content standards, and that a broad base of stakeholders was involved in the development of the performance standards. The State must have performance standards in place as part of the final assessment system and must report student performance against these standards.
- Complete the development of the second phase of the Montana assessment system addressing multiple measures that assess higher order thinking skills and the portions of the State standards that are not currently being assessed. Montana must describe the design of this phase of the assessment system, including the content to be assessed, the processes by which the system is to be created, the nature of the scores to be produced, and how the scores will be aggregated for decision making at the school, district, and State levels. Montana administered the Iowa Test of Basic Skills (ITBS) and the Iowa Test of Educational Development (ITED) in grades 4, 8, and 11, in spring 2001. The State also administered an Alternate Assessment Scale for students with disabilities (SWD) and limited English proficient (LEP) students for the first time in 2000-2001. However, since ITBS and ITED were the only tests available at the time of our review, the alternate assessments and any newly developed local assessments will have to be peer reviewed for alignment with Montana’s content and performance standards and for technical quality.
- Provide evidence of further objective alignment studies completed by teachers and other experts knowledgeable about Montana’s content standards and submit the results for peer review. Montana previously submitted for peer review evidence of a study done by the contractor of the alignment between the ITBS and ITED and Montana’s content standards.
- For the Alternate Assessment Scale, Montana must provide evidence of technical quality, the timeline for implementation, and the role of the Scale in the State’s accountability system. Further, we note that your plans call for both LEP students and SWD to use the same alternate assessment. The peer reviewers will seek evidence that this strategy is successful and appropriate.
- Provide complete participation data for SWD and LEP students so that Montana’s inclusion practices relating to assessment, reporting, and accountability can be evaluated. In addition, your agency will need to monitor the local application of inclusion policies to ensure that all students have access to the assessments and submit to this Department statewide participation rates for all tested grades and subgroups of students.
- Provide data showing that all assessments used in your State for Title I accountability meet commonly accepted professional standards for technical quality consistent with the uses made of the results. Montana provided evidence of technical quality only for the norm-referenced test portion of the State assessment system, and this evidence was in the form of a prepublication technical manual by the vendor; we will need to receive documentation of the technical quality of locally administered assessments and the alternate assessment. You will also need to develop and carry out evaluations the technical adequacy of all components of the system on a comprehensive basis.
- Develop and disseminate annual school reports that display assessment results for all students, disaggregated by gender, major racial/ethnic groups, LEP status, migrant status, students with disabilities as compared to non-disabled, and economically disadvantaged students compared to non-disadvantaged. You must submit your report format for review since the reporting of disaggregated data has not previously been part of the reporting format for Montana. Finally, these school reports must be shared with teachers and other educators, parents and the community.
- Upon completion of the development of performance standards, individual student interpretive and descriptive reports must be generated and disseminated to parents to inform them how well their students are meeting those performance standards.
- Provide the Department with your definition of “full academic year” for including students in determining adequate yearly progress.
Because of the scope and significance of the areas in which Montana has not met Title I statutory requirements, we have determined that, in order to maintain eligibility for Title I funding, Montana must enter into a compliance agreement with the Department of Education. A compliance agreement is a statutory remedy authorized by § 457 of the General Education Provisions Act for situations in which a State or local agency cannot meet statutory requirements within the legally specified timeframe. Its purpose is to enable a grantee to remain eligible to receive funding while coming into full compliance with applicable requirements as soon as feasible but within three years.
Although it is our understanding that the Montana legislature has not approved any funds to support the completion of the State’s final assessment system, we are ready to work with you to reach an accord on the terms of the compliance agreement as soon as possible. After we have agreed to tentative terms, the Department must hold a hearing to explore why full compliance with the Title I final assessment requirements is not feasible until a later date. The State, affected students and their parents, and other interested parties may participate. If, after the hearing, it is still evident that full compliance is not feasible until a later date, this office will make written findings to that effect and will publish those findings, along with the substance of the compliance agreements, in the Federal Register.
If you would like to discuss this further, please do not hesitate to call me.
Sincerely,
 
Thomas M. Corwin
Acting Deputy Assistant Secretary
Enclosure
