Idaho Science Assessment Letter

September 26, 2008

The Honorable Mike Rush
Executive Director
Idaho State Board of Education
650 West State Street
P.O. Box 83720
Boise, Idaho 83720-0037

The Honorable Tom Luna
Superintendent of Public Instruction
Idaho Department of Education
Len B. Jordan Office Building
650 West State Street
P. O. Box 83720
Boise, Idaho 83720

Dear Executive Director Rush and Superintendent Luna:

I am writing regarding our review of Idaho’s science assessments under the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB).

As outlined in my letter of February 28, 2008, states had to meet four basic requirements in science for the 2007-08 school year. In particular, each state was required to: (1) have approved content standards in science; (2) administer a general and alternate science assessment in each of three grade spans; (3) include all students in those assessments; and (4) report the results of the general and alternate science assessments on state, district, and school report cards. After reviewing the evidence submitted, Idaho still needs to provide data to the Department demonstrating that all students were included in the science assessments and that results from the science general and alternate assessments are included in the state-and district-level reports. Please let us know within 10 days of receipt of this letter the date by which Idaho will have those data and reports available so that we can confirm that it has, in fact, met the basic requirements for administering science assessments in 2007-08. States that do not provide the outstanding evidence to verify that they have met the four criteria for the 2007–08 school year have not met the basic requirements of the statute will be subject to consequences, such as withholding of Title I, Part A administrative funds.

In 2008-09, Idaho must provide evidence for peer review that demonstrates full compliance of its science standards and assessments. In anticipation of that required peer review, Idaho chose to participate in an optional technical assistance peer review in May 2008. I appreciate the efforts that were required to prepare for the technical assistance peer review and hope that the process provided useful feedback to support Idaho’s efforts to monitor student progress toward meeting challenging science standards.

Based on the evidence received from Idaho, which was reviewed by the peers and Department staff, we have concluded that Idaho’s science standards and assessments do not yet meet all the statutory and regulatory requirements of section 1111(b)(1) and (3) of the ESEA. Specifically, with respect to the Idaho Standards Achievement Test (ISAT), we have concerns with technical quality and reports and, with respect to the alternate assessment based on alternate academic achievement standards for students with the most significant cognitive disabilities, the Idaho Alternate Assessment (IAA), we have concerns regarding the alternate academic achievement standards, technical quality, alignment of science IAA to grade-level science content standards, and the inclusion of all students. The complete list of evidence needed to address these concerns is enclosed with this letter. We have scheduled peer reviews for states’ science assessments for the weeks of October 25 through November 1, 2008, and March 23 through 27, 2009. All materials for review must be provided to the Department three weeks before the scheduled peer review.

Please keep in mind that science assessments represent one piece of a state’s complete standards and assessment system, which also includes general and alternate assessments for reading and mathematics. As stated in the letter to you on November 16, 2006, from former Assistant Secretary Henry L. Johnson, Idaho’s standards and assessment system is currently fully approved. To remain fully approved, Idaho must demonstrate that all components of its standards and assessment system, including general and alternate assessments for science, comply with all ESEA requirements for standards and assessment systems as administered in 2008-09.

We look forward to working with Idaho to support a high-quality standards and assessment system of which science standards and assessments are an integral part. If you would like to discuss this further, please do not hesitate to contact Sharon Hall (sharon.hall@ed.gov) or Lauren Prehoda (lauren.prehoda@ed.gov) of my staff.

Sincerely,

Kerri L. Briggs, Ph.D.

Enclosure

cc: Governor C.L. Butch Otter
Margo Healy

SUMMARY OF ADDITIONAL EVIDENCE THAT IDAHO MUST SUBMIT TO MEET ESEA REQUIREMENTS FOR SCIENCE STANDARDS AND ASSESSMENTS MUST SUBMIT TO MEET ESEA REQUIREMENTS FOR SCIENCE STANDARDS AND ASSESSMENTS

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

Idaho Standards Achievement Test (ISAT)

  1. 1. Evidence that science content experts and a demographically diverse group of stakeholders were involved in the development of the academic achievement standards. (2.6)

Idaho Alternate Assessment (IAA)

  1. 1. Evidence that, for the 2008-09 revision of IAA, the alternate academic achievement standards are aligned with the grade-level 2006 Idaho science content standards. (2.5)

3.0 – FULL ASSESSMENT SYSTEM

  1. 1. Evidence that the ISAT and IAA involve measures that assess higher-order thinking skills and student understanding of challenging content. (3.6)

4.0 – TECHNICAL QUALITY

ISAT

  1. Evidence of validity:
    • Evidence that Idaho has developed a timeline to conduct a study to ascertain whether the ISAT science produces intended and unintended consequences. (4.1)
  2. Evidence that the use of accommodations has been evaluated:
    • Evidence that scores based on accommodated administration conditions allow for valid inferences. (4.6.b)
    • A monitoring plan and timeline to conduct a study of the availability and use of accommodations during test administration. (4.6.e)

IAA

  1. Evidence of validity (4.1):
    • Description of the purposes of the revised science IAA.
    • Evidence that Idaho has developed a timeline to conduct a study to ascertain whether the IAA science produces intended and unintended consequences.
    • Evidence that IAA science item scores are consistent with domain structures for science and that scores are related to internal and external variables as intended.
    • Documentation of the standards setting process, and the plan for subsequent standards setting for the revised science IAA.
  2. Evidence of reliability (4.2):
    • Documentation of how the format used for IAA science achieves consistency over time among raters and administrators to ensure meaningful interpretation of results.
  3. Evidence that the IAA is fair and accessible to students who are eligible for participation. (4.3)
  4. Evidence of clear criteria for the administration, scoring, analysis, and reporting components of the IAA science (4.5):
    • Evidence that all materials, including IAA administration guidelines and the test coordinators’ guide reflect the current test year, have consistent requirements, and establish clear criteria for the administration, scoring, analysis, and reporting of the IAA science.

5.0 – ALIGNMENT

IAA

  1. Evidence that the IAA, including all student evidence collected for the IAA, links and aligns comprehensively with Idaho’s 2006 science (not health) academic grade level content standards and reflects a similar range of Idaho’s academic content standards. (5.1, 5.2)
  2. Documentation of ongoing procedures to maintain and improve alignment of the IAA science with the academic content standards and academic achievement standards. (5.7)

6.0 – INCLUSION

IAA

  1. Documentation that all students in the grades tested are included in the science assessments, including students with disabilities, students with limited English proficiency, economically disadvantaged students, and students from major racial and ethnic groups, migrant students, and homeless students. (6.1)
  2. Documentation that all students assigned to participate in IAA do so if they are enrolled during the test window. (6.1)

7.0 – REPORTING

ISAT

  1. 1. ISAT state summary data that facilitates appropriate, credible, and defensible interpretation and use of assessment data, including number of students enrolled and percent or number tested/not tested, participation and assessment results for all students and for each student subgroup, as well as student proficiency in relation to Idaho content and achievement standards. (7.1, 7.2)
  2. Description of how the state ensures that individual student science reports are delivered to parents. (7.3.c)
  3. Evidence that the state ensures that administrators and school-based staff maintain the confidentiality of student data within the school. (7.4)

IAA

  1. Evidence that IAA science reports for the state, district, school, and individual student demonstrate that the reporting system facilitates appropriate, credible, and defensible interpretation and use of its assessment data, including the number of students enrolled, percent or number tested/not tested, participation and assessment results for all students and for each sub-group, as well as student proficiency in relation to Idaho content and achievement standards. (7.1)
  2. Evidence that the state provided for the production of individual interpretive, descriptive, and diagnostic reports that indicates relative strengths and instructional needs. (7.3)
    • Individual student reports express results in terms of Idaho’s achievement standards rather than numerical values such as scale scores or percentiles.
    • Individual student reports provide information for parents, teachers, and principals to help them understand and address a student’s specific academic needs, in a format and language that is understandable, and are accompanied by interpretive guidance for these audiences.
    • Evidence that Idaho ensures that these individual student reports are delivered to parents, teachers, and principals as soon as possible after the assessment is administered.
  3. Evidence of steps taken by the state to ensure that administrators and teachers maintain the confidentiality of the student data within the school. (7.4)

Return to state-by-state listing