South Carolina Assessment
February 15, 2006
Honorable Inez M. Tenenbaum
State Superintendent of Education
South Carolina Department of Education
1429 Senate Street
Columbia, South Carolina 29201
Dear Superintendent Tenenbaum:
I am pleased to approve South Carolina’s assessment system under Title I of the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). I congratulate you on meeting this important NCLB requirement.
My decision is based on input from peer reviewers external to the U.S. Department of Education (the Department) and Department staff who reviewed and carefully considered the evidence submitted by South Carolina. I have concluded that the evidence provided demonstrates that South Carolina’s standards and assessment system satisfies the NCLB assessment requirements. As such, the South Carolina system will receive Full Approval with Recommendations. This status means that South Carolina’s standards and assessment system meets all statutory and regulatory requirements; however, some elements of the system could be improved. The Palmetto Achievement Challenges Test-Alternate (PACT-ALT) meets the essential requirements of NCLB, but we recommend South Carolina strengthen its alternate assessment.
To provide more detail, on the basis of the additional evidence provided by South Carolina following the initial peer review, the Department concludes that the State’s assessment system – that is, the reading/language arts and mathematics assessments in each of grades 3-8 and high school, as well as the alternate assessment for both these subjects – meets all statutory and regulatory requirements. The Department recommends that South Carolina continue developing and improving its alternate assessments for reading/language arts and mathematics for students with disabilities in grades 3 through 8, in particular focusing on its technical quality. I note that the State has already begun this work and given the Department a timeline for improving these alternate assessments. Also, please note that South Carolina’s science assessment for elementary and middle school (i.e., Palmetto Achievement Challenges Tests) does not yet meet the alignment requirements of NCLB. South Carolina will need to provide additional evidence for peer review before the end of the 2007-08 school year to receive approval for this assessment.
Please be aware that approval of South Carolina’s assessment system for Title I is not a determination that the system complies with Federal civil rights requirements, including Title VI of the Civil Rights Act of 1964, Title IX of the Education Amendments of 1972, Section 504 of the Rehabilitation Act of 1973, Title II of the Americans with Disabilities Act, and requirements under the Individuals with Disabilities Education Act. Finally, please remember that if South Carolina makes significant changes in its assessment system, the State must submit information about those changes to the Department for review and approval.
We have found it a pleasure working with your staff on this review. Please accept my congratulations for your State’s approved standards and assessment system under NCLB. I wish you well in your continued efforts to improve student achievement in South Carolina.
Henry L. Johnson
Additional evidence is needed to show how South Carolina meets the following critical elements of the NCLB standards and assessment peer review guidance:
1.0a – There does not seem to be sufficient evidence of involvement of education stakeholders in the development of the science content standards.
Academic content standards review process was described in a narrative format (1.5a)
Evidence is needed to demonstrate stakeholder participation/review [sample agendas, sign-in, areas of expertise/representation, with emphasis on SWD and ELL, minutes and/or comments (Advisory Groups review would strengthen this area)]
1.0b – Documentation that the ELA standards for 1998 are not significantly different from the revised standards adopted in 2002. If the 2002 ELA standards are different, then the process for developing these standards must be reviewed.
The crosswalk document of grades 3-8 is evidence of a comparative analysis (1.1b) but the relationships between 1998 and 2002 ELA content standards remain unclear.
Evidence is needed regarding the EOC or other reviewing body examining the structural modifications, alignment information from 1998 to 2002 reflecting parallel content and process standards; Evidence is needed for PreK-12.
2.0a – Documentation that the superintendent has adopted academic achievement standards for the PACT, PACT-ALT, HSAP, and the HSAP-ALT.
The evidence documents adopted academic achievement standards (2.0a).
2.0b – For the PACT and HSAP, the State needs to include educators with expertise in educating diverse student populations, including English language learners, students with disabilities, and students living in poverty in the standards development process.
The (2.1b-2.17b) evidence does not document the involvement of diverse stakeholders (SWD and ELL) in the standard setting process, initial or review.
Evidence for the HSAP is has not completed standard setting for confirmatory standard setting process without stakeholder participation. TAC recommendations are unavailable for examination (e.g., agenda, members). For the PACT, there is partial evidence suggesting review by stakeholder advisory groups (SWD and ELL) reexamining the academic achievement standards would address the issue of diversity.
2.0c – Documentation of the number and percent of students with disabilities assessed in the regular assessment (including those administered with appropriate accommodations).
The (Summary Document pg. 3) provides documented evidence.
2.0d – Documentation must be provided to show how the interpolation on the PACT grades 4-7 was done and how the interpolation is related to content.
Evidence provided of the interpolation method is not sufficiently detailed. The evidence does not provide adequate information about the relationship between interpolated cut scores and academic content standards.
Empirical evidence is needed to demonstrate that the initial standard setting method provide valid and meaningful cut scores for current administrations of the assessments.
Performance level description development in conjunction with revisiting the initial standard setting (including established “cut scores”) was not evident within the materials presented.
The performance level descriptor approach implemented in SC appears to be sufficient only as long as the various performance levels at each grade are represented by test items that meaningfully link with content standards. Evidence that alignment exists between performance descriptors, content standards, and test content is needed. Because perceived test forms change and item pools are expanded to reflect needs for greater coverage of standards and depth of knowledge use of the performance level descriptors approach based on only the initial pilot test form may be inadequate. No procedure appears to be in place for reconsidering performance level descriptors as the test evolves.
2.0e – For the PACT-ALT, descriptors that relate to the levels of competency must be developed and a standard setting procedure must be employed.
The (2.1e-2.7e, Standard Setting Report)-PACT ALT reflects evidence of student progress, self-determination, and multiple settings scoring rubric.
Evidence is needed of a standard-setting process for PACT-ALT that contains descriptors of competency levels relative to the standards.
4.0a – Technical information about all the indicators for the PACT-ALT and HSAP-ALT except for (4.5) clear criteria for the administration, scoring, analysis, and reporting components of the assessments.
The (4.1a-4.12 a) technical information does include inter-rater reliability, re-training, and simple correlations.
Evidence is needed for PACT-ALT that provides technical information on reliability, and validity. **Technical documents should parallel the regular assessment to provide clarity.
HSAP-ALT – provides documentation that addresses reliability and validity, including a bias review, sensitivity and content alignment that lacks sufficient depth.
South Carolina needs to address the areas of validity and reliability discussed in the Standards for Educational and Psychological Testing (1999).
4.0b – Technical information on reliability of the PACT science assessment.
The 4.1 b provides the documented evidence. Send ED the complete technical document on Science.
5.0a – For the PACT, reports of independent alignment studies and evidence of a systematic process for addressing any gaps or weaknesses identified in these studies.
The 5.1-5.3 does not provide a systematic process for re-aligning the system over time.
Evidence is needed for the statewide assessment including alternate assessments that address Standards and Assessments Peer Review Guidance, Section 5: Alignment (5.1-5.7).
6.0a – Report the percentage of students who are excluded from the participation data due to use of test modifications, including off-level tests.
The 6.1a-6.2a provided the documented evidence.
6.0b – Report the number of students who are receiving test accommodations.
The 6.0b provided the documented evidence.
7.0a – Clarifications how students taking modified test such as out-of-level are in the reporting system.
The 7.0 provided the documented evidence.