Kansas Assessment Letter
June 27, 2006
The Honorable Bob Corkins
Commissioner of Education
Kansas State Department of Education
120 SE 10th Avenue
Topeka, KS 66612-1182
Dear Commissioner Corkins:
Thank you for your participation in the U.S. Department of Education’s (Department) standards and assessment peer review process under Title I of the Elementary and Secondary Education Act of 1965 (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). I appreciate the efforts required to prepare for the peer review. As you know, with the implementation of NCLB’s accountability provisions, each school, district, and State is held accountable for making adequate yearly progress (AYP) towards having all students proficient by 2013–14. An assessment system that produces valid and reliable results is fundamental to a State’s accountability system.
I am writing to follow up on the letter that was sent to you on March 24, 2006. In that letter we presented the results of the peer review of the Kansas standards and assessment system and detailed the additional evidence necessary for Kansas to meet the statutory and regulatory requirements of Section 1111(b)(1) and (3) of the ESEA. The Department scheduled a second peer review for Kansas in May 2006. The additional evidence provided was not sufficient to address all of the outstanding items. The remaining evidence necessary is listed in the enclosure to this letter.
As you will recall, the Department laid out new approval categories in the letter to the Chief State School Officers on April 24, 2006. These categories better reflect where States collectively are in the process of meeting the statutory standards and assessment requirements and where each State individually stands. Based on these new categories, the current status of the Kansas standards and assessment system is Approval Pending. This status indicates that Kansas’ standards and assessment system administered in the 2005–06 school year has three or more fundamental components that are missing or that do not meet the statutory and regulatory requirements, in addition to other outstanding issues that can be addressed more immediately. These deficiencies must be resolved in a timely manner so that the standards and assessment system administered next year meets all requirements. The Department believes that Kansas can address the outstanding issues by the next administration of its assessment system, that is, by the end of the 2006–07 school year.
Kansas’ system has a number of fundamental components that warrant the designation of Approval Pending. Specifically, the Department cannot approve Kansas’ standards and assessment system due to outstanding concerns with the academic achievement standards for mathematics on the general assessment; the comparability of test formats (e.g., computer-based and paper-and-pencil); the lack of evidence regarding the challenging content of the reading assessment; and whether English language learners are being assessed on grade-level expectations in reading and mathematics. Please refer to the enclosure for a detailed list of the evidence Kansas must submit to meet the requirements for an approved standards and assessment system.
In addition to the above deficiencies, please note that the Department was not able to fully evaluate the alternate assessment with modified achievement standards tied to grade-level content standards (the Kansas Assessment of Multiple Measures, or KAMM). The Department has not published final regulations regarding this assessment; until such time, the Department cannot fairly evaluate this assessment. We have provided information on the KAMM from the peer review, which you may use as guidance, but once the Department releases regulations, Kansas will have to submit the KAMM for peer review.
Accordingly, Kansas is placed under Mandatory Oversight, pursuant to 34 C.F.R. §80.12. Under this status, there will be specific conditions placed on Kansas’ fiscal year 2006 Title I, Part A grant award. Kansas must provide, not later than 25 business days from receipt of this letter, a plan and detailed timeline for how it will meet the remaining requirements to come into full compliance by the end of the 2006–07 school year. Beginning in September 2006, Kansas must also provide bi-monthly reports on its progress implementing the plan.
Due to the number of outstanding items that Kansas needs to complete to come into compliance with NCLB, the Department intends to withhold 10 percent of the State’s fiscal year 2006 Title I, Part A administrative funds, totaling $81,754, pursuant to Section 1111(g)(2) of the ESEA. Kansas has the opportunity, within 20 business days of receipt of this letter, to show cause in writing why we should not withhold these funds. If Kansas cannot show cause, the Department will withhold 10 percent of Kansas’ fiscal year 2006 Title I, Part A administrative funds, which will then revert to local educational agencies in Kansas. Moreover, if, at any time, Kansas does not meet the timeline set forth in its plan, the Department will initiate proceedings to withhold an additional 10 percent of the State’s fiscal year 2006 Title I, Part A administrative funds.
I know you are anxious to receive full approval of your standards and assessment system and we are committed to helping you get there. Toward that end, let me reiterate my earlier offer of technical assistance. We remain available to assist you however necessary to ensure you administer a fully approved standards and assessment system. We will schedule an additional peer review when you have evidence available to further evaluate your system. If you have any questions or would like to request reconsideration of the conditions, please do not hesitate to contact Patrick Rooney (Patrick.Rooney@ed.gov) or Sue Rigney (Sue.Rigney@ed.gov) of my staff.
Henry L. Johnson
cc: Governor Kathleen Sebelius
Summary of Additional Evidence that Kansas Must Submit to Meet ESEA Requirements for the Kansas Assessment System
2.0 – ACADEMIC ACHIEVEMENT STANDARDS
- Detailed plan and timeline for revising the mathematics performance level descriptors associated with the general assessment and the performance level descriptors for the Kansas Assessment of Multiple Measures (KAMM) in all content areas to ensure appropriate differentiation across levels, grades, and tests.
- Evidence of achievement standard-setting for the regular test, the Kansas Alternate Assessment (KAA), and the KAMM with an accurate description of the procedures used, including qualifications and diversity of participants and methods for resolving discrepant results from separate standard-setting methods.
- Evidence of Board adoption of the achievement standards for reading and mathematics for the regular test, the KAA, and the KAMM.
3.0 – FULL ASSESSMENT SYSTEM
- Evidence to support the comparability of scores at the cut points between basic and proficient performance levels from the Spanish language version and the general assessment.
- Results of completed studies that demonstrate comparability at the proficient level between the regular test in paper and pencil format and in computer format.
- Evidence that the reading assessments include challenging content. The evidence submitted for mathematics (Assessment Framework Information Math) could be used as a model for this task.
4.0 – TECHNICAL QUALITY
- Completed studies that document item interrelationships and structural consistency; criterion validity; and standard setting.
- A detailed timeline for completing the data collection necessary for studies of consequential validity and for completing these studies.
- Detailed test specifications (such as those provided for the general mathematics assessment) for the general reading assessment, the KAA, and the KAMM assessments.
- Detailed test administration, training, and scoring manuals for the KAA and the KAMM.
- A description of how the State monitors and evaluates the availability of accommodations for limited English proficient (LEP) students and for students with disabilities during testing.
- Documentation, including having the vendor evaluate and produce detailed reports of the reliability data for all assessments, particularly the KAA, including score reliability at the cut points.
- Completed comprehensive technical manuals that provide technical quality documentation for the general assessments and all alternate assessments.
- Evidence that the translated mathematics assessment is comparable to the English version of this assessment and measures grade-level expectations for mathematics.
5.0 – ALIGNMENT
- For the reading test, the KAA, and the KAMM, provide test specifications that are detailed enough to support the creation of test forms that: are comparable over time; comprehensively represent the target indicators on any test form; are aligned with the content and skills aspects of its content standards; reflect the same degree and pattern of emphasis as are reflected in the State’s content standards; and reflect the knowledge and skills represented in Kansas’ achievement standards.
- Results from the external, independent alignment study scheduled for November 2006.
- Information regarding the potential inclusion of constructed-response items on test forms and how the inclusion of these items is expected to affect alignment with the target standards.
- Evidence of how the State identified its assessed indicators.
- Clarification of how the KAA is aligned comprehensively with the standards in each content area.
- Explanation of why functional indicators are represented in the extended standards document and how the inclusion of these indicators improves access to the general academic curriculum at the student’s grade level.
- Detailed description of the process, including information about the participants in the process, used to develop the Kansas Extended Standards.
- Detailed plans, including standard-setting method, and a timeline for identifying the alternate academic achievement standards on the KAA.
6.0 – INCLUSION
- Clarification of the process for determining how students participate in the assessment system, including the State’s exemption policies and guidelines for identifying students to take the KAA and the KAMM.
- Evidence that the KAMM is based on grade-level content standards.
- Evidence that all limited English proficient (LEP) students are assessed against grade-level expectations in reading and mathematics.
- Clarification that LEP students are tested in English after they have been enrolled in schools in the United States for three years.
7.0 – REPORTING
- Evidence that the State reports disaggregated participation and performance data for males and females. The State must also clarify that the participation and performance results are based on the students enrolled at the time of testing and are not adjusted for the “full academic year.”
- Evidence that parents are informed when a child’s achievement is based on alternate achievement standards.
- Sample student reports along with explanatory information provided to help parents understand the results.