District of Columbia Science Assessment Letter

October 16, 2008

The Honorable Deborah Gist
State Superintendent for Education
Office of the State Superintendent of Education
Government of the District of Columbia, One Judiciary Square
441 4th Street, NW, Suite 350 North
Washington, DC 20001

Dear Superintendent Gist:

I am writing regarding our review of the District of Columbia’s science assessments under the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB).

As outlined in my letter of February 28, 2008, states had to meet four basic requirements in science for the 2007-08 school year. In particular, each state was required to: (1) have approved content standards in science; (2) administer a regular and alternate science assessment in each of three grade spans; (3) include all students in those assessments; and (4) report the results of the regular and alternate science assessments on state and district report cards. Based on the evidence submitted, I have concluded that the District of Columbia did not meet these requirements for 2007-08. Specifically, the District of Columbia has not provided evidence that all high school students are included in the biology assessment. Consequently, the Department intends to withhold five percent of the District of Columbia’s fiscal year 2008 Title I, Part A administrative funds, totaling $23,051, pursuant to section 1111(g)(2) of the ESEA. The District of Columbia has the opportunity, within 20 business days of receipt of this letter, to show cause in writing why the Department should not withhold these funds. If the District of Columbia cannot show cause, the Department will withhold five percent of the District of Columbia’s fiscal year 2008 Title I, Part A administrative funds, which will then revert to Title I, Part A funds available for allocation to schools in the District of Columbia.

In 2008-09, the District of Columbia must provide evidence for peer review that demonstrates full compliance of its science standards and assessments. In anticipation of that required peer review, the District of Columbia chose to participate in an optional technical assistance peer review in May 2008. I appreciate the efforts that were required to prepare for the technical assistance peer review and hope that the process provides useful feedback that will support the District of Columbia’s efforts to monitor student progress toward meeting challenging science standards. I also recognize that this past June marked the one year anniversary of the development of the Office of the State Superintendent of Education (OSSE) and understand that you have been working to reorganize and establish an effective unit.

Based on the evidence received from the District of Columbia, which was reviewed by the peers and Department staff, I have concluded that the District of Columbia’s science assessments do not yet meet all the statutory and regulatory requirements of section 1111(b)(1) and (3) of the ESEA. Specifically, I have concerns with the academic achievement standards, full assessment system, technical quality, alignment, inclusion and reporting of the science component of the District of Columbia Comprehensive Assessment System (DC CAS) and the District of Columbia Comprehensive Assessment System Alternate (DC CAS-Alt). The enclosed list of evidence describes the information and documentation that the District of Columbia will need to provide to be able to demonstrate that its science standards and assessments satisfy all relevant ESEA requirements. We have scheduled peer reviews for states’ assessments for the weeks of October 25 through November 2, 2008, and March 23 through 27, 2009. A state must submit evidence to the Department three weeks prior to the review in which it is participating.

Please keep in mind that science assessments represent one piece of a state’s complete standards and assessment system, which also includes general and alternate assessments for reading and mathematics. As stated in my letter to you on April 25, 2008, the District of Columbia’s standards and assessment system is currently designated Approval Pending. The District of Columbia must demonstrate that all outstanding components of its standards and assessment system as administered in 2008-09, including the general and alternate assessments for reading, mathematics, and science, comply with all ESEA requirements for state standards and assessment systems. I commend OSSE for working with me and my staff to sign the Memorandum of Agreement on October 3, 2008 demonstrating the District of Columbia’s intent to come into compliance and administer a fully approved standards and assessment system for reading and mathematics in the 2008-09 school year.

We look forward to working with the District of Columbia to support a high-quality standards and assessment system, of which science standards and assessments are an integral part. If you would like to discuss this further, please do not hesitate to contact Sue Rigney (Sue.Rigney@ed.gov) or Jessica Morffi (Jessica.Morffi@ed.gov) of my staff.

Sincerely,

Kerri L. Briggs, Ph.D.

Enclosure

cc: Mayor Adrian Fenty
Bill Caritj
Kimberly Statham

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

1. The completed standards-setting manuals for both the DC CAS and DC CAS-Alt science tests that include:
   1. Qualifications of the standards-setting judges;
   2. Standard setting model and procedures, including the training programs for the judges;
   3. Final performance level descriptors; and
   4. The recommended cut scores.
2. Evidence that its state board has formally approved the achievement standards for both assessments in science.
3. Evidence that the number and percent of those students with disabilities assessed against alternate achievement standards, those assessed on an alternate assessment against grade-level standards, and those included in the regular assessment (including those administered with appropriate accommodations) are reported separately.

3.0 – FULL ASSESSMENT SYSTEM

1. Clarification of the DC CAS test design with appropriate technical documentation of equivalent forms.
2. Documentation showing that both the DC CAS and DC CAS-Alt science assessments include challenging academic content appropriate to the student population tested.

4.0 – TECHNICAL QUALITY

1. A statement of the purposes that the science assessments are designed to serve and how DC ascertains that decisions based on the results are appropriate and consistent with these intended purposes. (DC CAS and DC CAS-Alt)
2. Evidence of an evaluation of unintended negative consequences of the science assessments. (DC CAS and DC CAS-Alt)
3. Evidence that its science assessments yield reliable scores that are consistent with the structures inherent to the state’s academic content standards. (DC CAS and DC CAS-Alt)
4. Evidence of the relationships between science assessment performance and other relevant, external variables. (DC CAS)
5. Completed technical reports to document the technical quality of its science assessment system. (DC CAS and DC CAS-Alt)
6. Evidence of a policy and of the implementation of its policy for allowing only those accommodations that do not fundamentally alter the construct being assessed. The state must provide evidence that students assessed under non-standard testing conditions are considered non-participants in the assessment system.
7. Evidence that judgmental and empirical methods are used to evaluate potential bias of items on the science tests. This evidence must include a description of the process, the participants, and the outcomes of these reviews.
8. Detailed test blueprints for DC CAS grades 3 and 5 and for high school Biology that support the development of comparable test forms over time that sample all of the relevant science standards.
9. A long-term equating plan that specifies field-test design and item development/acquisition procedures to support the development of test forms that are comparable.
10. Evidence of its quality control requirements for scoring and analyzing results for both the DC CAS and DC CAS-Alt science assessments, including scoring rubrics for the DC-CAS in science.
11. Evidence of continuous procedures (including regular engagement of the TAC) for using the results of monitoring and other analysis for improving the on-going quality of its assessment system. (DC CAS and DC CAS-Alt)

5.0 – ALIGNMENT

1. For the DC CAS assessment:
   1. The rationale for the coverage of only approximately one third of science standards and clarification of whether or not the other two-thirds will be covered in future years;
   2. Item specifications used to guide item writing (item content, format and scoring criteria);
   3. Procedures and rationale for item selection; and
   4. Two-way alignment studies that verify that the operational tests are aligned with the science content standards.
2. For the DC CAS-Alt assessment:
   1. Policy and procedures for selection of prioritized skills for the DC CAS-Alt that ensures that students do not repeat the same skills over time; and
   2. Evidence of alignment for the science assessment.

6.0 – INCLUSION

1. Data confirming participation of all high school students in the biology assessment (DC CAS or DC CAS-Alt) at some point in their high school career.
2. Evidence of clarification to the field that students may not participate in the DC CAS-Alt on the basis of a 504 plan.

7.0 – REPORTING

1. Actual reports that address all requirements in this section, including:
   1. School, district and state summary reports that include results for all required reporting;
   2. subgroups as well as the number enrolled and the number tested/not;
   3. An individual student report that includes results expressed as achievement levels with appropriate explanation of the meaning of the achievement levels accompanied by a detailed interpretive guide for parents in language that is understandable to them;
   4. Evidence that test results will be delivered to parents and educators as soon as possible after the assessment is completed; and
   5. Itemized score analyses by subdomain or standards that provides useful instructional information to educators.
2. Evidence that the state ensured that student-level assessment data are maintained securely to protect student confidentiality.

Return to state-by-state listing