Hawaii Assessment Letter
June 29, 2006
The Honorable Patricia Hamamoto
Hawaii Department of Education
P.O. Box 2360
Honolulu, Hawaii 96804
Dear Superintendent Hamamoto:
Thank you for your participation in the U.S. Department of Education’s (Department) standards and assessment peer review process under Title I of the Elementary and Secondary Education Act of 1965 (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). I appreciate the efforts required to prepare for the peer review. As you know, with the implementation of NCLB’s accountability provisions, each school, district, and State is held accountable for making adequate yearly progress (AYP) towards having all students proficient by 2013-14. An assessment system that produces valid and reliable results is fundamental to a State’s accountability system.
I am writing to follow up on the letter that was sent to you on March 24, 2006. In that letter, we presented the results of the peer review of the Hawaii standards and assessment system. A second review in June 2006 was not sufficient to resolve all of the outstanding evidence necessary for Hawaii to meet the statutory and regulatory requirements of Section 1111(b)(1) and (3) of the ESEA. The list of remaining evidence is listed in the enclosure to this letter. In addition, please refer to the peer review guidance (refer to: www.ed.govhttps://www2.ed.gov/admins/lead/account/peerreview/index.html) for a complete list of information that must be provided to meet the requirements.
As you will recall, the Department laid out new approval categories in a letter to the Chief State School Officers on April 24, 2006. These categories better reflect where States collectively are in the process of meeting the statutory standards and assessment requirements and where each State individually stands. Based on these new categories, the current status of the Hawaii standards and assessment system is Approval Pending. This status indicates that Hawaii’s standards and assessment system administered in the 2005-06 school year has three or more fundamental components that are missing or that do not meet the statutory and regulatory requirements, in addition to other outstanding issues that can be addressed more immediately. These deficiencies must be resolved in a timely manner so that the standards and assessment system administered next year meets all requirements. The Department believes that Hawaii can address the outstanding issues by the next administration of its assessment system, that is, by the end of the 2006-07 school year.
Hawaii’s system has a number of fundamental components that warrant the designation of Approval Pending. Specifically, the Department cannot approve Hawaii’s standards and assessment system due to the fact that Hawaii’s new general assessment, the Hawaii State Assessment (HSA) aligned to the Hawaii Content and Performance Standards III (HCPS III) will be administered for the first time in the 2006-07 school year. As a result, Hawaii was not able to demonstrate the technical quality and alignment of this assessment. There are also concerns regarding the inclusion of all students in the assessment system. Please refer to the enclosure for a detailed list of the evidence Hawaii must submit to meet the requirements for an approved standards and assessment system.
Accordingly, Hawaii is placed under Mandatory Oversight, pursuant to 34 C.F.R. §80.12. Under this status, there will be specific conditions placed on Hawaii’s fiscal year 2006 Title I, Part A grant award. Hawaii must provide, not later than 25 business days from receipt of this letter, a plan and detailed timeline for how it will meet the remaining requirements to come into full compliance by the end of the 2006-07 school year. Beginning in September 2006, Hawaii must also provide bi-monthly reports on its progress implementing the plan.
Due to the number of outstanding items that Hawaii needs to complete to come into compliance with NCLB, the Department intends to withhold 10 percent of the State’s fiscal year 2006 Title I, Part A administrative funds, totaling $46,179, pursuant to Section 1111(g)(2) of the ESEA. Hawaii has the opportunity, within 20 business days of receipt of this letter, to show cause in writing why we should not withhold these funds. If Hawaii cannot show cause, the Department will withhold 10 percent of Hawaii’s fiscal year 2006 Title I, Part A administrative funds, which will then revert to Title I, Part A funds available for allocation to schools in Hawaii. Moreover, if, at any time, Hawaii does not meet the timeline set forth in its plan, the Department will initiate proceedings to withhold an additional 10 percent of the State’s fiscal year 2006 Title I, Part A administrative funds.
I know you are anxious to receive full approval of your standards and assessment system and we are committed to helping you get there. Toward that end, let me reiterate my earlier offer of technical assistance. We remain available to assist you however necessary to ensure you administer a fully approved standards and assessment system. We will schedule an additional peer review when you have evidence available to further evaluate your system. If you have any questions or would like to request reconsideration of the conditions, please do not hesitate to contact Patrick Rooney (Patrick.Rooney@ed.gov) or Valeria Ford (Valeria.Ford@ed.gov) of my staff.
Henry L. Johnson
cc: Governor Linda Lingle
Summary of Additional Evidence that Hawaii Must Submit to Meet ESEA Requirements for the Hawaii Assessment System
1.0 – ACADEMIC CONTENT STANDARDS
- Documentation of involvement of stakeholders such as parents, educators of students with disabilities and English language learners, and representatives from the outer islands in the development of the Hawaii Content and Performance Standards III (HCPS III).
2.0 – ACADEMIC ACHIEVEMENT STANDARDS
- Documentation of the standard-setting process employed, qualifications of the panelists, and final recommendations for the achievement standards for the Hawaii State Assessment (HSA), the Hawaii State Alternate Assessment (HSAA), and the Hawaii Aligned Portfolio Assessment (HAPA).
- Evidence of adoption of the achievement standards for the HSA, HSAA, and HAPA by the State Board of Education.
- Evidence of science performance level descriptors, descriptions of the competencies associated with each level, and the cut scores that differentiate each level (when set).
3.0 – FULL ASSESSMENT SYSTEM
- Evidence that the operational HSA has been administered to all students in grades 3-8 and high school in 2006-07.
- Documentation of the design and scoring of the HSAA.
4.0 – TECHNICAL QUALITY
- Information regarding basic validity and reliability (i.e., consequential validity, item interrelationships and structural consistency, criterion validity and standards setting procedures) for the 2005-06 HSAA and HAPA and the 2006-07 HSA.
- Technical manuals for the 2005-06 administration of the HSAA and HAPA and the 2006-07 administration of the HSA.
- Complete reliability analyses for the 2005-06 HSAA and HAPA and the 2006-07 HSA.
- Documentation of the test development and accommodation procedures for the HSA.
- Documentation of the procedures for monitoring accommodations, administration, scoring, and reporting.
- Specific information regarding vertical and horizontal equating needed to ensure consistent meaning of scores across grades and years.
- Evidence that the accommodations permitted support valid use of the assessment results.
- Documentation of a comprehensive plan for assuring that security is in place for all assessments.
- Documentation that that accommodations yield valid scores and can be used meaningfully with scores from non-accommodated students taking the regular assessment.
5.0 – ALIGNMENT
- Documentation of the alignment of all assessments to content standards and plans to address gaps or weaknesses identified in previous alignment studies.
- Extensions of the interpretive guides/blueprints to illustrate that scoring and reporting structures are consistent with the sub-domain structures of the academic content standards.
- Documentation of the linkage of the HSAA with the general curriculum.
6.0 – INCLUSION
- Documentation of enrollment, participation, and exclusion numbers for all subgroups on all assessments.
- A description of monitoring assessment procedures for migrant or mobile students.
- Documentation of the inclusion of the following students:
- "Grade 31" students (students in an out-of-level placement authorized by a student’s Individualized Education Program (IEP) team, a hearing officer, or a judge);
- Students who are deemed by their physician to be medically unable to take part in the HSA (with or without accommodations) or the HSAA;
- Students with a disability who are placed in or referred to an out-of-State residential program by their IEP/MP team, a hearing officer, or a judge; and
- Homeschooled children with a disability who are provided special education and related services by a school.
7.0 – REPORTING
- A description of materials and procedures for guiding interpretation and use of alternate assessments.
- Sample reports used in the 2005-06 school year for the HSAA and HAPA and sample reports or mock-ups to be used in the 2006-07 school year for the HSA. These should include individual student reports for each type of test used for NCLB accountability purposes.
- Procedures for security and protection of assessment data and instruments.
- Complete chart of disaggregated reports.
- Clarification regarding how the results of the HSAA are reported to parents and schools.