Idaho Assessment Letter

June 8, 2001

Honorable Marilyn Howard
Superintendent of Public Instruction
Idaho Department of Education
Len B. Jordan Office Building
650 West State Street
P.O. Box 83720
Boise, Idaho 83720

Dear Superintendent Howard:

It was a pleasure speaking with you and Tom Farley regarding the progress that Idaho is making toward meeting the Title I requirements for standards, assessments, and accountability. I know that we all share the goal of having, as soon as possible, a full and functional assessment system that supports education improvement in Idaho and that meets the Title I requirements. These requirements are intended to ensure that every school and school district participating in Title I is focused on enabling all students to meet the State’s academic standards.

After peer-reviewing the assessment evidence and timelines submitted by Idaho last October, we have determined that Idaho is substantially out of compliance with sections 1111 and 1116 of Title I. In order to remain eligible to receive Title I funds, Idaho must enter into a compliance agreement with the U.S. Department of Education, specifying the steps the State will take to fully meet the Title I requirements and the timeline for completing those steps.

Based on the peer review and our recent conversations, additional clarity is needed regarding Idaho’s plans to fully comply with the Title I requirements. We will need, for example, additional information on the process that Idaho has used to establish content and performance standards, the alignment and technical quality of the components of the Idaho assessment system, the status of Idaho’s plan to include all limited English proficient students in the State assessment system, and the State’s use of student performance data for all students in determining school accountability. Questions also exist regarding Idaho’s intent to disaggregate student performance on Idaho assessments by the elements required by the Title I statute at the State, district, and school levels. Finally, Idaho needs to clarify the components of its final assessment system and the contribution of that system to a school’s or district’s Adequate Yearly Progress (AYP).

Page 2 – Honorable Marilyn Howard

We summarize below the major Title I requirements and our judgment of Idaho’s compliance status. Steps that must be addressed in order to meet the requirements are detailed in the summary attached to this letter.

Content Standards: Title I requires that each State submit evidence of content standards that reflect what students should know and be able to do by the 1997-98 school year.

Idaho has not yet met this requirement. During our conversation, you indicated that Idaho’s content standards were recently approved by the Idaho legislature. Documentation of legislative approval and submission of evidence regarding the broad-based involvement of special educators, English language proficiency experts, university personnel, and others representing Idaho’s citizens, in the development and review of content standards is needed for Idaho to receive approval of its standards.

Performance Standards: Title I requires each State to submit evidence of performance standards that describe three levels of performance aligned with the State’s content standards for each of the three required grade spans by the 1997-98 school year.

Idaho has not yet met this requirement. You have informed the Department that Idaho will develop new assessments based on the State’s content standards. Those assessments should serve as the basis for the establishment of performance standards. A plan and timeline for completing performance standards must be provided to the Department as a part of the compliance agreement.

Final Assessment System: Title I requires that, for the purposes of school accountability, States administer assessments yearly to students in at least reading/language arts and math, in three grade spans (grades 3-5, 6-9 and 10-12). Title I also requires that assessment systems use multiple measures that assess higher-order thinking skills and understanding.

Idaho does not meet these requirements. Based on the evidence submitted for the peer review, Idaho appears to have a mixed assessment system that includes both State and local assessments. Your submission focused on the assessments administered by the State without providing details on the guidance provided to local educational agencies (LEAs) or how your agency monitors the locally developed assessments in terms of their alignment with standards, technical quality, and reporting requirements. In addition, information is needed on the cognitive complexity of the Idaho assessment items and the extent to which Idaho assessments assess student knowledge using multiple measures. It is not clear what role, if any, the Direct Mathematics and Direct Writing assessments play in the Idaho assessment system and in accountability. ??

Page 3 – Honorable Marilyn Howard

Based on our conversation, we also understand that the Idaho assessment system is still evolving. The current plan, as we understand it, will involve development of additional State-level assessments that are based on Idaho content standards. The assessment components that are being added may help Idaho fully meet this requirement, but we could not make a judgment on that issue based on the information available to us.

As part of the compliance agreement, Idaho must provide a complete description of the assessment system it will implement as well as a plan and timeline for implementation.

Alignment: Title I requires final assessments to be aligned with content and performance standards in at least math and reading/language arts, as well as any other subject area in which a State has adopted standards. Idaho provided no objective evidence regarding the alignment between the components of the Idaho assessment program and Idaho standards or on the cognitive complexity of the Idaho assessments.

As part of the compliance agreement, Idaho must provide a complete description of its plan for documenting that its final assessment system is aligned with Idaho content and performance standards and a timeline for implementation.

Technical Quality: Title I requires that the State assessments be used for purposes for which those assessments are valid and reliable, and be consistent with relevant, nationally recognized professional and technical standards for such assessments. The October 1 submission provided no technical quality or fairness information on the Idaho assessment system or how multiple measures are used to provide access to the Idaho assessment system for all children.

Idaho did not provide information regarding how the State will provide guidance or monitor the technical quality of assessments chosen by LEAs that are used for Title I accountability purposes. Moreover, Idaho provided no evidence on the way the State will ensure that local assessment results are comparable for different schools and for different years, or of systematic plans for reviewing and improving the local assessments.

As part of the compliance agreement, Idaho must provide a complete description of its plan for documenting the technical quality of its final assessment system and a timeline for implementation.

Inclusion of All Students in Assessments, Reporting, and Accountability: Title I requires that final assessments provide for the participation of all students in the grades being assessed. Title I specifically requires the inclusion of limited English proficient (LEP) students in final assessments and makes clear that States must assess LEP students, to the extent practicable, in the language and form most likely to yield accurate and reliable information on what they know and can do in subjects other than English. Furthermore, Title I requires States to provide reasonable adaptations and

Page 4 – Honorable Marilyn Howard

accommodations for students with diverse learning needs, including LEP students and students with disabilities (SWD).

Idaho does not yet meet these requirements. Idaho provided no evidence that its proposed new LEP policy has been approved by the State Board of Education. In addition, Idaho needs a plan for implementing its new LEP inclusion policies, including appropriate guidance on accommodations, and for monitoring LEA compliance with the new inclusion policies once they have been approved.

Idaho’s plans regarding the development of an alternate assessment for students with disabilities who are unable to take the standard assessment with or without accommodations is unclear.

Idaho also did not provide participation rate data for SWD and LEP students, or for students in general, and did not provide information on how the performance data for SWD and LEP students will be incorporated into the reporting of results for assessment and accountability purposes.

Idaho must submit a plan and timeline that includes LEP students and SWD in all Idaho assessments and report the results for accountability purposes on all assessments that comprise the Idaho assessment system, including ITBS/ITED assessments, direct assessments in writing and mathematics, and all local assessments. Idaho also needs to provide a plan for determining the progress of small schools, describing how the performance of schools with fewer than 100 students will be evaluated.

Reporting: Title I also requires that assessments provide individual student interpretive and descriptive reports that let parents know how well their children are meeting the performance standards set by the State and that all participating LEAs produce individual school performance profiles for all their participating schools. While Idaho will not be able to meet this requirement until performance standards are implemented, the State needs to ensure that the reports will be produced.

Assessment results also must be disaggregated within each State, local educational agency, and school. The Title I statute requires reporting results by gender, major racial and ethnic group, English proficiency status, and migrant status. It also requires that students with disabilities be compared to non-disabled students, and economically disadvantaged students be compared to students who are not economically disadvantaged. Idaho does not disaggregate by any of these elements at the school or district level. However, it does disaggregate five of the six elements at the State level. Idaho must develop procedures for incorporating the remaining element (disadvantaged versus non-disadvantaged) in State-level reports and for ensuring that local- and school-level reports include all the required disaggregations.

Page 5 – Honorable Marilyn Howard

Because of all of the above issues, Idaho must enter into a compliance agreement with the Department of Education in order to remain eligible to receive Title I funds. A compliance agreement is a statutory remedy authorized by ??457 of the General Education Provisions Act for situations in which a State or local educational agency cannot meet statutory requirements within the timeframe specified by law. Its purpose is to bring a grantee into full compliance with applicable requirements as soon as feasible, but no longer than three years. Attached is a summary of the issues, based on the materials that we reviewed, that the compliance agreement must address. This list may change as Idaho clarifies its plans for the development and implementation of the standards-based assessments and responds to the issues raised.

The Department and the State will need to agree on the components of the compliance agreement, including a detailed plan and specific timeline for all of the steps necessary to bring the State into compliance. Before entering into a compliance agreement, the Department will provide Idaho with a hearing at which to demonstrate that full compliance is not feasible until a future date and will publish findings of noncompliance and the substance of the compliance agreement in the Federal Register.

When the required steps have been completed, we will resubmit evidence that the assessment system meets the Title I requirements to a panel of peer reviewers for review.

We are prepared to begin immediately to work with you to discuss the issues raised in this letter and to develop the details of the compliance agreement. We believe it is possible to complete work on this agreement within 60 days.

We look forward to working with Idaho to support a high-quality assessment system. If you would like to discuss this further, please do not hesitate to call Mitzi Beach, Group Leader for Compensatory Education Programs, at 202-260-0823.

Sincerely,

Thomas M. Corwin
Acting Deputy Assistant Secretary

Enclosure

SUMMARY OF TITLE I REQUIREMENTS TO BE ADDRESSED

Based on the material submitted to date, Idaho will need to provide or develop the following in order to meet the Title I requirements. This list may change as Idaho clarifies its plans for the development and implementation of the standards-based assessments and responds to the issues raised.

Idaho must provide clearer information on its full assessment system. The role of local assessments in the Idaho full assessment system is unclear. It appears that an LEA may substitute the non-standard components of the State assessment system for assessments that are locally determined and that those assessments can count for up to 50 percent in the determination of AYP. What is the role of the local assessments in the State assessment system? How does the State monitor the alignment and technical quality of the local assessments as they are included in accountability?
Idaho permits local data to account for 50 percent of the weight in determining school and district accountability. Title I requires each State plan to demonstrate that the State has developed or adopted a set of high-quality, yearly student assessments that will be used as the primary means of determining the yearly performance of each local educational agency and school served under Title I, Part A. Idaho needs to provide evidence that its accountability system will allow the results of the Idaho final assessment system, including local assessments where applicable, to be the primary indicators of adequate yearly progress.
Idaho must submit evidence that its content standards were developed with broad-based involvement. It also must provide evidence that performance standards have been developed and implemented and that they are aligned with Idaho’s content standards and the Idaho assessment program.
Idaho needs to provide information on the enrollment of LEP students and SWDs in the State at the assessed grade levels and provide information on the number of LEP students and SWDs who take the standard form of the Idaho assessments and the Idaho assessments with accommodations, and the number of those students exempted or excluded from the Idaho assessment program.
Idaho must publish evidence that its proposed LEP inclusion policy has been approved by the State Board of Education, along with a copy of the new policy. To implement its new policies, Idaho must develop assessment guidelines and guidance on accommodations for LEP students, and a plan for implementing the new LEP inclusion policies and for monitoring LEA compliance with the new inclusion policies when they are approved.
Idaho must submit detailed information on the alternate assessment for SWD, along with technical quality information on the instrument and the scores generated from it. We also need evidence that the accommodations associated with the norm-referenced tests and State-developed assessments yield valid results for SWD, as well as information regarding any accommodations that are planned for the Direct

2

Mathematics and Writing assessments and the technical quality of those accommodated assessments.
Idaho needs to document how it will incorporate performance data for SWD and LEP students into its reporting of results for assessment and accountability purposes.
Idaho must provide evidence regarding the extent to which the components of the Idaho Assessment Program are aligned with Idaho standards. Information on the cognitive complexity of the Idaho assessments and their alignment with the State’s content and performance standards should be provided, as well as identification of the gaps and weaknesses of the assessment system.
Idaho must provide technical information on each of the components of the Idaho Assessment Program and information on how Idaho ensures the fairness of its assessments for all students.
Idaho must provide evidence on how the multiple measures that have been incorporated in the Idaho Assessment Program affect the validity, reliability, and fairness of those assessments.
Idaho must disaggregate student performance by gender, race/ethnicity, migrant status, disability (versus non-disability), economic disadvantage (versus non-disadvantaged), and limited English proficiency status at the LEA and school levels. In addition, it must add economic disadvantage to the categories that are currently being disaggregated at the State level.
Idaho must define for LEAs which students are to be included in determining adequate yearly progress (AYP) for schools and LEAs.
Idaho needs to provide a plan for evaluating the AYP of its small schools and K-3 schools.

Return to state-by-state listing