North Dakota Science Assessment Letter

October 8, 2008

The Honorable Wayne G. Sanstead
Superintendent of Public Instruction
North Dakota Department of Public Instruction
600 Boulevard Avenue, East
Bismarck, North Dakota 58505-0440

Dear Superintendent Sanstead:

I am writing regarding our review of North Dakota’s science assessments under the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB).

As outlined in my letter of February 28, 2008, States had to meet four basic requirements in science for the 2007-08 school year. In particular, each State was required to: (1) have approved content standards in science; (2) administer a regular and alternate science assessment in each of three grade spans; (3) include all students in those assessments; and (4) report the results of the regular and alternate science assessments on State, district, and school report cards. Based on the evidence submitted to date, it appears that North Dakota has met the first two of these requirements for 2007-08. However, before we can determine whether North Dakota has met the other two requirements, North Dakota needs to submit the following evidence: data showing the inclusion in the science assessments of all students in the tested grades; and sample State and district reports or report shells for results of general and alternate science assessments. Please let us know within 10 days of your receipt of this letter when North Dakota will have those data and reports available. States that do not provide the evidence necessary to demonstrate that they have met the four criteria for the 2007-08 school year have not met the basic requirements of the statute and will be subject to consequences, such as withholding of Title I, Part A administrative funds.

In 2008-09, North Dakota must provide evidence for peer review that demonstrates full compliance of its science standards and assessments. In anticipation of that required peer review, North Dakota chose to participate in an optional technical assistance peer review in May 2008. I appreciate the efforts that were required to prepare for the technical assistance peer review and hope that the process provided useful feedback to support North Dakota’s efforts to monitor student progress toward meeting challenging science standards.

Based on the evidence received from North Dakota, which was reviewed by the peers and Department staff, we have concluded that North Dakota’s science standards and assessments do not yet meet all the statutory and regulatory requirements of section 1111(b)(1) and (3) of the ESEA. The complete list of evidence needed to address these concerns is enclosed with this letter. We have scheduled peer reviews for States’ science assessments for October 25 through November 2, 2008 and March 23 through 27, 2009. All materials for review must be provided to the Department three weeks before the scheduled peer review.

Please keep in mind that science assessments represent one piece of a State’s complete standards and assessment system, which also includes regular and alternate assessments for reading/language arts and mathematics. As stated in my letter to you on June 29, 2007, North Dakota’s standards and assessment system is currently designated Full Approval. To remain fully approved, North Dakota must demonstrate that all components of its standards and assessment system as administered in 2008-09, including general and alternate assessments for science, comply with all ESEA requirements for standards and assessment systems.

We look forward to working with North Dakota to support a high-quality standards and assessment system, of which science standards and assessments are an integral part. If you would like to discuss this further, please do not hesitate to contact Collette Roney (collette.roney@ed.gov) or Abigail Rogers (abigail.rogers@ed.gov).

Sincerely,

Kerri L. Briggs, Ph.D.

Enclosure

cc: Governor John Hoeven
Greg Gallagher

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

1. A report showing separately the number and percent of those students with disabilities assessed against alternate achievement standards, those assessed on an alternate assessment against modified achievement standards, and those included in the regular assessment (including those administered with appropriate accommodations).
2. Evidence of the subject matter (i.e., science) expertise of the NDAA1 Cut score Setting Team members.

3.0 – FULL ASSESSMENT SYSTEM

1. A plan for ensuring that the science tests assess higher order thinking skills, especially at grades 8 and 11.

4.0 – TECHNICAL QUALITY

1. A plan and timeline for implementing the studies North Dakota has proposed to address validity issues (Consequential Validity Studies, June 2007).
2. Reliability statistics for the NDSA for student subpopulations and reported sub-scores.
3. Clear criteria for administering and scoring the NDAA1 to ensure that teachers consistently provide students with the same opportunities for performing tasks in multiple settings and that teachers reliably score student performance on the situational indicators.
4. Evidence of the appropriateness of the standard-setting methodology used to set cut scores for the NDAA1 (e.g., a rationale for the method employed, a description of how the performance level descriptors (PLDs) were drafted, a description of how grade-level PLDs were used to align cut scores with grade-level content standards, and a rationale for using the same cut scores across grade levels).
5. Evidence of inter-rater reliability of teacher scoring of the situational indicators for the NDAA1.

5.0 – ALIGNMENT

1. A plan and a timeline for improving the depth of knowledge assessed by the NDSA in science relative to North Dakota’s science content standards, especially at grade 11.
2. Additional evidence of the alignment of the NDAA1 to the State’s academic content standards for science (e.g., regarding the test development process, an independent alignment study), including a rationale for how each of the situational indicators reflects students’ academic content knowledge and not other non-academic factors.

7.0 – REPORTING

1. Reports showing:
   1. Participation and assessment results for science assessments for all students and for each of the required subgroups in its reports at the school, district, and State levels; and
   2. For each grade tested in science, a summary report that includes the number of students enrolled or number tested/not tested.
2. Documentation that the State’s reporting system facilitates appropriate, credible, and defensible interpretation and use of its assessment data, specifically:
   1. Evidence of interpretive cautions on school, district and State reports about the measurement error in reported scores for the NDSA where reported standard and benchmark level scores are based on a small number of items; and
   2. Evidence that NDAA1 results are only reported by achievement level by standard when they are based on achievement levels set for each standard through the achievement standards-setting process.

Return to state-by-state listing