July 13, 2007

The Honorable Jim McBride
Superintendent
Wyoming Department of Education
Hathaway Building
2300 Capitol Avenue
Cheyenne, Wyoming 82002-0050

Dear Superintendent McBride:

I am writing regarding our review of Wyoming’s standards and assessment system under the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). We appreciate the efforts required to prepare for the peer review and hope that the process provides useful feedback that will support Wyoming’s efforts to monitor student progress toward challenging standards.

External peer reviewers and Department staff evaluated Wyoming’s fourth peer review submission and found, based on the evidence received, that it still does not meet all the statutory and regulatory requirements of section 1111(b)(1) and (3) of the ESEA. I know that my staff has discussed the results of this review with your staff and I appreciate you taking the time to meet with me during my visit to Cheyenne on June 28. I want to take this opportunity to enumerate the evidence that Wyoming must provide in order to have a fully compliant system. Specifically, we have concerns with the technical quality and alignment of the Proficiency Assessments for Wyoming Students (PAWS) to Wyoming’s grade-level academic content standards and academic achievement standards as well as concerns with the technical quality and alignment of Wyoming’s alternate assessment based on alternate academic achievement standards (the Alternate Proficiency Assessments for Wyoming Students, or PAWS-Alt) to grade-level content standards. We appreciate the effort your staff has made to conclude the peer review process but we are concerned about the lack of comprehensive and coherent approach to documenting validity of the assessment system. The complete list of evidence needed to address these concerns is enclosed with this letter.

I know that Wyoming submitted a draft timeline for how and when Wyoming will satisfy the remaining requirements for the PAWS and additional evidence for peer review on July 3 and that additional evidence will be submitted in the coming month. Sue Rigney of my staff will review and respond under separate cover to Wyoming’s draft timeline. Please note that all evidence regarding the PAWS must be submitted no later than August 27, 2007 so that it may be peer reviewed during the Department’s review from September 17-21. Therefore, we are not assigning an approval status to Wyoming’s system at this time. Because that system is not fully approved, a condition was placed on your fiscal year 2007 Title I, Part A grant award.

If Wyoming is unable to resolve the remaining issues with the PAWS, we will take appropriate enforcement actions as outlined in the Department’s May 10, 2007, fact sheet, including the possibility of a Compliance Agreement under Section 457 of the General Education Provisions Act. For your convenience, I am enclosing a copy of that fact sheet, which is also available on the Department’s website (http://www.ed.govhttps://www2.ed.gov/admins/lead/account/statesystems.html).

We look forward to working with Wyoming to support a high-quality standards and assessment system. If you would like to discuss this further, please do not hesitate to call Sue Rigney (202-260-0931) or Patrick Rooney (202-205-8831) of my staff.

Sincerely,

Kerri L. Briggs, Ph.D.

Enclosures

cc: Governor Dave Freudenthal
Joe Simpson
Lesley Wangberg

SUMMARY OF ADDITIONAL EVIDENCE THAT WYOMING MUST SUBMIT TO MEET ESEA REQUIREMENTS FOR THE WYOMING ASSESSMENT SYSTEM

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

1. Confirmation that the PAWS-Alt is administered as intended, e.g. number and percent of non-scoreable components.
2. The scoring rubrics for all three PAWS-Alt components: the Data Collection Form (DCF), the Teacher Observation of Academic Skills (TOAS), and the Student Performance Events (SPE). The DCF rubric was provided but the scoring criteria for TOAS and SPE components were not.
3. Training materials for the accurate administration, scoring, and interpretation of results from the PAWS-Alt.
4. A description of the standards-setting process for the PAWS-Alt that includes documentation of the procedures employed, including:
   1. The standards-setting model;
   2. The materials used to train the panelists;
   3. The rationale and procedures for selecting student work samples;
   4. A sample student profile as used by the panelists; and
   5. The numerical information used to summarize the contents of the student profiles used for standard setting.
   6. An explanation of how variation in student profiles was considered in setting performance standards and drafting the final performance descriptors.

4.0 – TECHNICAL QUALITY

1. Evidence of technical quality for PAWS to address the reviewers’ concerns regarding:
   1. Procedures for the development of test forms appropriate for the design and intended purpose (reading, writing, and mathematics);
   2. Stability of results from multiple forms within year (split-half option) and across time (reading, writing, and mathematics);
   3. Consistency between test administration practices and the standards-setting process (reading, writing, and mathematics);
   4. Clear blueprints, item specifications, and test development procedures for 2006-07 (reading and writing);
   5. Implementation of a writing scale that combines with reading scale scores to form a combined English/language arts score for accountability; and
   6. Implementation of sound scaling and equating procedures across test forms and years.
2. Implementation of sound scaling and equating procedures across test forms and years.
3. Evidence of technical quality for PAWS-Alt to address the reviewers’ concerns regarding:
   1. Clear explanations of design and scoring;
   2. Documentation of scoring reliability (consistency and accuracy);
   3. Justification of score use given the threat to validity represented by the inter-component correlations that show higher correlations between content areas than between components within content areas;
   4. The low percentage of items related to assessment targets; and
   5. Data discrepancies within and between successive versions of the Technical Reports.
   6. Complete validity information for the operational PAWS and reliability and validity information for the PAWS-Alt assessments, including data supporting concurrent and consequential validity.
   7. Data confirming that the accommodations permitted on the PAWS assessment support valid use of the assessment results.
   8. A plan and commitment for ongoing analysis of the relation between test administration patterns (split-half option), test results and the implications for interpretation, and intended consequences.

5.0 – ALIGNMENT

1. The systematic procedures that will be used to address the alignment gaps documented in the ELY alignment study and the concerns about alignment stated in the technical manual.
2. Test blueprints or other documents that show alignment of the PAWS with the content standards rather than “skills.”
3. Clarification regarding the design and administration of the operational PAWS-Alt sufficient to determine whether the assessment content is aligned with grade-level content standards.

6.0 – INCLUSION

1. Official grade-by grade enrollment data (either for spring 2006 or for the date of enrollment that is used by Wyoming to calculate the participation rate for adequate yearly progress determinations) for all students. This should include official State assessment reports for the assessments administered in 2006-07 that display the number of students with disabilities tested on PAWS and PAWS-Alt.

Return to state-by-state listing

April 26, 2007

The Honorable Jim McBride
Superintendent
Wyoming Department of Education
Hathaway Building
2300 Capitol Avenue
Cheyenne, Wyoming 82002-0050

Dear Superintendent McBride:

Thank you for your participation in the U.S. Department of Education’s (Department) standards and assessment peer review process under Title I of the Elementary and Secondary Education Act of 1965 (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB).

In the letter to you on June 22, 2006, the Department enumerated the fundamental components as well as a number of additional technical issues that had not met the standards and assessment requirements of the ESEA. Specifically, the Department could not approve Wyoming’s standards and assessment system due to outstanding concerns with the alternate assessment based on alternate academic achievement standards (the Alternate Proficiency Assessments for Wyoming Students, or PAWS-Alt) and the process for standards setting for the new assessments based upon the field trial and operational assessments.

In January 2007, the peer reviewers and Department staff evaluated additional evidence submitted by Wyoming. Based on the evidence received, we have determined that Wyoming’s system still does not meet all of the statutory and regulatory requirements of Section 1111(b)(1) and (3) of the ESEA. I appreciate you taking the time last month to meet with Catherine Freeman and Sue Rigney to discuss the results of that peer review and I hope it helped you in preparing for the next peer review. I know that Wyoming submitted additional evidence that will be reviewed by the peers next week; I encourage you to bring in all available evidence for review.

Because the peer review did not resolve all outstanding issues, the status of Wyoming’s standards and assessment system remains Approval Pending. The enclosure provides a detailed list of the evidence Wyoming must still submit to meet the requirements for a fully approved standards and assessment system. Please note that Wyoming must address all outstanding issues in order to have a fully compliant standards and assessment system under the ESEA by the end of the 2006-07 school year. Wyoming continues to be under Mandatory Oversight, as authorized under 34 C.F.R. §80.12, and the condition on your Title I, Part A grant award will continue.

I appreciate the steps Wyoming has taken toward meeting the requirements of the ESEA, and I know you are anxious to receive full approval of your standards and assessment system. We are committed to helping you get there and remain available to provide technical assistance regarding issues that you identify. Toward that end, let me reiterate my earlier offer of technical assistance. We will schedule an additional peer review when you have evidence available to further evaluate your system. If you have any questions or would like to discuss this further, please do not hesitate to contact Sue Rigney (Sue.Rigney@ed.gov) or Patrick Rooney (Patrick.Rooney@ed.gov) of my staff.

Sincerely,

Kerri L. Briggs, Ph.D.
Acting Assistant Secretary

Enclosure

cc: Governor Dave Freudenthal
Joe Simpson
Lesley Wangberg

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

1. Clarification of the design elements of the Alternate Proficiency Assessments for Wyoming Students (PAWS-Alt) and evidence that it generates appropriate and accurate information about student performance.
2. Directions for teachers to administer the assessment and confirmation that the directions are implemented as intended.
3. The scoring rubrics for all PAWS-Alt components, as well as the training provided to ensure reliable scoring.
4. A description of the standards setting process for the PAWS-Alt that includes the procedures employed, the qualifications of the panelists, and an explanation of how data from the three main components is integrated into a performance standard.

4.0 – TECHNICAL QUALITY

1. A completed technical manual that includes detailed information regarding the Proficiency Assessments for Wyoming Students (PAWS) item and test development; procedures for combining results based on multiple choice and constructed response items; procedures for the combination of reading and writing results to produce a language arts proficiency score; standards setting, scaling, scoring, and equating procedures; and validity and reliability information.

5.0 – ALIGNMENT

1. The systematic procedures that will be used to address the alignment gaps documented in the ELY alignment study.
2. Test blueprints or other documents that show alignment of the PAWS with the content standards (as opposed to alignment with “skills”).
3. Clarification regarding the design and administration of the operational PAWS-Alt sufficient to determine whether the assessment content is aligned with grade-level content standards.

6.0 – INCLUSION

1. Participation data for PAWS and PAWS-Alt showing that all students enrolled are included in the State assessments. This is not the same as AYP participation rates and AYP reports will not meet this requirement.

7.0 – REPORTS

1. Sample student reports for the PAWS and PAWS-Alt with the explanatory materials provided to parents and educators.
2. Sample item analysis consistent with the description in the Department’s Standards and Assessment Guidance, dated March 2003.

Return to state-by-state listing

June 22, 2006

The Honorable Jim McBride
State Superintendent of Public Instruction
Wyoming Department of Education
2300 Capitol Ave, 2nd Floor
Hathaway Building
Cheyenne, Wyoming 82002-0050

Dear Superintendent McBride:

Thank you for your participation in the U.S. Department of Education’s (Department) standards and assessment peer review process under Title I of the Elementary and Secondary Education Act of 1965 (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). I appreciate the efforts required to prepare for the peer review. As you know, with the implementation of NCLB’s accountability provisions, each school, district, and State is held accountable for making adequate yearly progress (AYP) towards having all students proficient by 2013-14. An assessment system that produces valid and reliable results is fundamental to a State’s accountability system.

I am writing to follow up on the letter that was sent to you on May 10, 2006. In that letter we presented the results of the peer review of the Wyoming standards and assessment system and detailed the additional evidence necessary for Wyoming to meet the statutory and regulatory requirements of Section 1111(b)(1) and (3) of the ESEA. A second peer review in May 2006 was not sufficient to resolve all of the issues. The need for the remaining outstanding evidence, as listed in the enclosure to this letter, remains.

As you will recall, the Department laid out new approval categories in the letter to the Chief State School Officers on April 24, 2006. These categories better reflect where States collectively are in the process of meeting the statutory standards and assessment requirements and where each State individually stands. Based on these new categories, the current status of the Wyoming standards and assessment system is Approval Pending. This status indicates that Wyoming’s standards and assessment system administered in the 2005-06 school year has two or more fundamental components that are missing or that do not meet the statutory and regulatory requirements, in addition to other outstanding issues that can be addressed more immediately. These deficiencies must be resolved in a timely manner so that the standards and assessment system administered next year meets all requirements. The Department believes that Wyoming can address the outstanding issues by the next administration of its assessment system, that is, by the end of the 2006-07 school year.

Wyoming’s system has a number of fundamental components that warrant the designation of Approval Pending. Specifically, the Department cannot approve Wyoming’s standards and assessment system due to outstanding concerns with the alternate assessment based on alternate achievement standards (the Alternate Proficiency Assessments for Wyoming Students, or PAWS-Alt) and the process for standard setting for the new assessments based upon the field trial and operational assessments. Please refer to the enclosure for a detailed list of the evidence Wyoming must submit to meet the requirements for an approved standards and assessment system.

Accordingly, Wyoming is placed under Mandatory Oversight, pursuant to 34 C.F.R. §80.12. Under this status, there will be specific conditions placed on Wyoming’s fiscal year 2006 Title I, Part A grant award. Wyoming must provide, not later than 25 business days from receipt of this letter, a plan and detailed timeline for how it will meet the remaining requirements to come into full compliance by the end of the 2006-07 school year. Beginning in September 2006, Wyoming must also provide bi-monthly reports on its progress implementing the plan. If, at any time, Wyoming does not meet the timeline set forth in its plan, the Department will initiate proceedings, pursuant to Section 1111(g)(2) of the ESEA, to withhold 15 percent of Wyoming’s fiscal year 2006 Title I, Part A administrative funds, which will then revert to local educational agencies in Wyoming.

I know you are anxious to receive full approval of your standards and assessment system and we are committed to helping you get there. Toward that end, let me reiterate my earlier offer of technical assistance. We remain available to assist you however necessary to ensure you administer a fully approved standards and assessment system. We will schedule an additional peer review when you have evidence available to further evaluate your system. If you have any questions or would like to discuss this further, please do not hesitate to call Patrick Rooney (Patrick.Rooney@ed.gov) or Sue Rigney (Sue.Rigney@ed.gov) of my staff.

Sincerely,

Henry L. Johnson

Enclosure

cc: Governor Dave Freudenthal
Annette Bohling

Summary of Additional Evidence that Wyoming Must Submit to Meet ESEA Requirements for the Wyoming Assessment System

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

1. Final comprehensive reports from the Proficiency Assessments for Wyoming Students (PAWS) and the PAWS-Alt standard setting, including the qualifications of the panelists indicating experience with students with disabilities and students with limited English proficiency.
2. Evidence of State Board adoption of the achievement standards for the PAWS and the PAWS-Alt assessments.
3. A plan and timeline for the completion and adoption of the final academic achievement descriptors and cut scores for science assessments.

4.0 – TECHNICAL QUALITY

1. A completed technical manual that includes detailed information regarding item and test development, procedures for combining results based on multiple choice and constructed response items as well as the combination of reading and writing results to produce a language arts proficiency score, standard setting, scaling, scoring and equating procedures and validity and reliability information.
2. Reliability and validity information for the operational PAWS and PAWS-Alt assessments, including data supporting concurrent and consequential validity.
3. Data confirming that the accommodations permitted on the PAWS assessment support valid use of the assessment results.
4. Comparability evidence showing that tests administered both on-line and by paper and pencil are comparable in order to support standards set on the basis of paper/pencil administration (field trial) but applied to the online administration (operational).
5. Specific information regarding the processes for reconciling results from the 2006 Blue Ribbon Panel and the summer 2005 standard-setting participants.

5.0 – ALIGNMENT

1. The systematic procedures that will be used to address the alignment gaps documented in the Ely alignment study.
2. Final test construction blueprints for the PAWS assessment as administered in 2005-06 and future years.
3. Documentation showing that the operational PAWS-Alt assessment is aligned with grade-level content standards.

6.0 – INCLUSION

1. Participation data for the PAWS and PAWS-Alt assessments showing that all students enrolled are included in the State assessments. Please note that this is not the same as AYP participation rates; AYP reports will not meet this requirement.
2. Additional documentation for the PAWS-Alt assessment, including guidelines for eligibility for the PAWS-Alt assessment, evidence that the guidelines were disseminated to the field in a timely manner, and a complete description of the test administered in 2005-06 with an explanation of how pilot results were or will be incorporated in the final operational test.

7.0 – REPORTS

1. Sample student reports for the PAWS and PAWS-Alt with the explanatory materials provided to parents and educators.
2. Sample school, district, and State reports that include data for the migrant population.
3. Sample item analysis.
4. Evidence that the State has notified LEAs of the procedures to be used for distributing student reports to parents in a timely manner.

Return to state-by-state listing

May 10, 2006

The Honorable Jim McBride
Superintendent
Wyoming Department of Education
Hathaway Building
2300 Capitol Avenue
Cheyenne, WY 82002-0050

Dear Superintendent McBride:

Thank you for submitting Wyoming’s assessment materials for review under the standards and assessment requirements of the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). We appreciate the efforts required to prepare for the peer review and hope that the process provides useful feedback that will support Wyoming’s efforts to monitor student progress toward challenging standards.

External peer reviewers and U.S. Department of Education (ED) staff evaluated Wyoming’s submission and found, based on the evidence received, that it did not meet all the statutory and regulatory requirements of Section 1111(b)(3) of the ESEA. I know that my staff has discussed the results of this review with your staff. However, I want to take this opportunity to enumerate the evidence that Wyoming must provide in order to have a fully compliant standards and assessment system under NCLB. That evidence is listed on the last pages of this letter.

For this initial review, very few details were provided regarding the technical quality, alignment, reliability, and validity of the Wyoming assessment system, making it almost impossible to determine whether the system meets professionally accepted testing standards and NCLB requirements. I understand you have submitted additional evidence for a second round of peer review that is currently underway. Following the review, you will receive a letter detailing any additional evidence needed, if any, to demonstrate that Wyoming’s system meets the standards and assessment requirements.

Enclosed with this letter are detailed comments from the peer review team that evaluated Wyoming’s assessment materials in February. The peer reviewers, experts in the areas of standards and assessment, review and discuss a State’s submission of evidence and prepare a consensus report. I hope you will find the reviewers’ comments and suggestions helpful. I also want to remind you of our offer to provide you further technical assistance at your request.

We look forward to working with Wyoming to support a high-quality assessment system. If you would like to discuss this further or request technical assistance, please do not hesitate to call Sue Rigney (202-260-0931) or Patrick Rooney (202-205-8831) of my staff.

Sincerely,

Henry L. Johnson

Enclosure

cc: Governor Dave Freudenthal
Annette Bohling

Summary of Additional Evidence that Wyoming Must Submit to Meet ESEA Requirements for the Wyoming Assessment System

1.0 – CONTENT STANDARDS

1. Provide results from an external evaluation that documents that the language arts, mathematics, and science content standards are challenging and contain rigorous and coherent content (this might be included as part of an overall alignment study).
2. Provide documentation that diverse stakeholders were involved in development of the content standards. Specifically, provide evidence of the consideration of the applicability to all students, including students with disabilities and limited English proficient students.

ACADEMIC ACHIEVEMENT STANDARDS

1. Provide documentation of the standard setting process employed, qualifications of the panelists, and final recommendations for the Proficiency Assessments for Wyoming Students (PAWS) and the PAWS-Alt assessments.
2. Provide evidence of adoption of the achievement standards for the PAWS and the PAWS-Alt assessments by the Board.
3. Provide a plan and timeline for completion and adoption of the final academic achievement descriptors and cut scores for the science assessment.
4. Produce evidence that the standard-setting panelists reflect knowledge of students with disabilities and limited English proficient students.

FULL ASSESSMENT SYSTEM

1. Provide evidence that the operational PAWS and PAWS-Alt assessments have been administered to all students in grades 3-8 and 11.
2. Provide documentation regarding the determination of student scores.

TECHNICAL QUALITY

1. Provide, at a minimum, basic validity and reliability information (i.e., consequential validity, item interrelationships and structural consistency, criterion validity and standards setting procedures) beginning with the 2004-05 pilot.
2. Due to the nature of recent and planned changes to the PAWS assessment program, provide technical manuals for 2004-05, 2005-06, and 2006-07 assessments. This practice should continue annually until the assessment system stabilizes.
3. Provide complete reliability analyses for the 2005-06 PAWS and PAWS-Alt assessments (note: the 2004-05 Wyoming Comprehensive Assessment System (WyCAS) technical manual chapter on reliability was comprehensive, consistent with the peer review guidelines, and would serve as a good model for such evidence).
4. Provide detailed standards-setting plans and procedures. Provide a rationale for the adequacy of the proposed 2-phase process in determining AYP for the PAWS and PAWS-Alt, including:
   1. Detailed information on the materials and process used, panelist instructions, and results of the panel’s recommendations;
   2. Specific information regarding the processes for reconciling results from the 2006 Blue Ribbon Panel and the Summer 2005 standards-setting participants; and
   3. Detailed description of Wyoming’s scaling and equating design.
5. Provide quality control procedures for monitoring accommodations, administration, scoring, and reporting. This should include plans and procedures for communicating the quality control procedures to the field and the contractor.
6. Provide specific information regarding vertical and horizontal equating needed to ensure consistent meaning of scores across grades and years.
7. Provide evidence that the accommodations permitted support valid use of the assessment results.
8. Provide evidence that the State has contracted for production of a complete technical manual that will include complete data for each PAWS assessment, including the PAWS-Alt, timeline for delivery, and delivery of the complete manual as quickly as possible.

ALIGNMENT

1. Clear evidence that its assessments are aligned to content standards and that they represent the depth and breadth of the Wyoming Content Standards.
2. An overall rationale that connects the content standards, blueprints, and assessments as well as documentation that Wyoming educators and parents have access to this information.
3. Documentation of a more-detailed test blueprint and item specifications for the field test and operational test, including the rationale for item selection for the Spring 2006 assessments.
4. Evidence that the State will require the contractor conducting the external alignment study to provide specific recommendations regarding the quality of alignment.
5. Documentation describing Wyoming’s ongoing process for addressing deficiencies found by any independent alignment studies.
6. Provide results from the independent alignment study for the PAWS and PAWS-Alt assessments.

INCLUSION

1. Produce participation rates for all student groups based on the number enrolled at the time of testing.
2. Provide information regarding accommodations training.
3. Produce participation guidelines and administration procedures for the PAWS-ALT assessment.

REPORTS

1. Provide sample reports for the PAWS and PAWS-Alt assessments at the student level that present results in terms of the achievement standards, including explanatory materials provided to parents.
2. Provide sample reports for the PAWS and PAWS-Alt assessments at the State, district, and school levels that include all required disaggregation.
3. Conduct sample item analysis and provide the results of these analyses.
4. Produce evidence that the State has notified local educational agencies (LEAs) of the procedures to be used for distributing student reports to parents in a timely manner.

Return to state-by-state listing

May 5, 2006

Dr. Terry Bergeson
State Superintendent of Public Instruction
Washington Department of Public Instruction
P.O. Box 47200
Olympia, WA 98504-7200

Dear Superintendent Bergeson:

Thank you for submitting Washington’s assessment materials for review under the standards and assessment requirements of the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). We appreciate the efforts required to prepare for the peer review and hope that the process provides useful feedback that will support Washington’s efforts to monitor student progress toward challenging standards.

External peer reviewers and U.S. Department of Education (ED) staff evaluated Washington’s submission and found, based on the evidence received, that it did not meet all the statutory and regulatory requirements of Section 1111(b)(3) of the ESEA.
I know that my staff has discussed the results of this review with your staff. However, I want to take this opportunity to enumerate the evidence that Washington must provide in order to have a fully compliant standards and assessment system under NCLB. That evidence is listed on the last pages of this letter.

Also, I wanted to alert you to two issues raised during this review. First, last year, the Department approved Washington to use its writing assessment in combination with reading as an alternative option to reading alone when determining proficiency in language arts at grades 4, 7, and 10. We did so with the understanding that Washington would document with the appropriate evidence that both the reading/writing combination and the reading only option meet the required critical elements for State assessment systems. Washington did not include the writing assessment in its submission for this peer review. (Please refer to my letter dated September 1, 2005, which may be found at: http://www.ed.govhttps://www2.ed.gov/admins/lead/account/letters/). Second, the peers and staff raised a number of questions about the “Developmentally Appropriate Washington Assessment of Student Learning.” It is not clear from the information provided if this assessment is used for accountability purposes. If the results are used for adequate yearly progress (AYP) decisions, Washington must submit the appropriate evidence to demonstrate this assessment meets ESEA’s requirements.

I urge you to submit any available evidence demonstrating how Washington’s system meets the standards and assessment requirements as soon as possible. I also request that, as soon as possible, you provide us a plan with a detailed timeline for how Washington will meet any remaining requirements for which evidence is not currently available. After reviewing those materials, I will then determine the appropriate approval status for Washington’s standards and assessment system.

Enclosed with this letter are detailed comments from the peer review team that evaluated Washington’s assessment materials. The peer reviewers, experts in the areas of standards and assessment, review and discuss a State’s submission of evidence and prepare a consensus report. I hope you will find the reviewers’ comments and suggestions helpful and remind you of our offer to provide you further technical assistance at your request.

We look forward to working with Washington to support a high-quality assessment system. If you would like to discuss this further, or would like to request technical assistance, please do not hesitate to call David Harmon (202-205-3554) or Catherine Freeman (202-401-3058) of my staff.

Sincerely,

Henry L. Johnson

Enclosure

cc: Governor Christine Gregoire
Mary Alice Heuschel

Summary of Additional Evidence that Washington Must Submit to Meet ESEA Requirements for the Washington Assessment System

1.0 – ACADEMIC CONTENT STANDARDS

• Documentation that the Superintendent officially adopted the State content standards and grade-level expectations.

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

• Approved academic achievement standards for WASL in reading and mathematics at grades 3, 5, 6, and 8 with documentation of diverse stakeholder participation.
• Approved academic achievement standards for the DAW in reading and mathematics at grades 3-8 and 10 with documentation of diverse stakeholder participation, if used for AYP decisions.
• Approved alternate academic achievement standards for WAAS in reading and mathematics at grades 3-8 and 10 with documentation of diverse stakeholder participation.
• Additional guidelines for individualized educational plan (IEP) teams to use when deciding which assessment a student should take and additional documentation that parents are informed about and understand the consequences of agreeing to the use of an alternate assessment.

3.0- FULL ASSESSMENT SYSTEM

• Documentation of full WASL and WAAS implementation at grades 3, 5, 6, and 8 as planned during the 2005-06 school year and evidence that assessments at these grade levels measure challenging content.
• Documentation that the Developmentally Appropriate Washington Assessment of Student Learning (DAW) meets all ESEA requirements, if used for AYP decisions.

4.0 – TECHNICAL QUALITY

• Documentation of the validity and reliability of the WASL at grades 3, 5, 6, 7, 8, and 10.
• Documentation of the validity and reliability of the WAAS at grades 3, 5, 6, and 8.
• Technical quality documentation for the WASL and WAAS standard setting in reading and mathematics at grades 3, 5, 6, and 8.
• Documentation of the technical quality (validity, reliability, standard setting) of the DAW, if used for AYP decisions.
• Final equating study technical report for the WASL at grades 4, 7, and 10 in reading and mathematics.
• Documentation that the State monitors its assessment system and testing activities with special attention to evaluating accommodations for students with disabilities and limited English proficient (LEP) students.

5.0 – ALIGNMENT

• Plans delineating on-going and long-term processes whereby alignment between the assessment(s) and the Essential Academic Learning Requirements (EALRs) and Grade Level Expectations (GLEs) will be monitored and improved over time.
• Documentation (including detailed test blueprints and item specifications) of reading and math WASL alignment at grades 3, 5, 6, and 8 with EALRs and GLEs.
• Documentation (including detailed test blueprints and item specifications) of reading and math DAW alignment at grades 3-8 and 10 with EALRs and GLEs.
• Documentation of reading and mathematics WAAS alignment at grades 3, 5, 6, and 8 with EALRs and GLEs.

6.0 – INCLUSION

• Enrollment and number of students assessed by grade level, content area, and subgroup.
• Enrollment and number of students with disabilities assessed with the WASL, WAAS, and DAW (if used for AYP decisions) by grade level and content area.
• Guidelines for educators to determine in which assessment, WASL, WAAS, and DAW (if used for AYP decisions), a student with disabilities should participate.
• Documentation that the State is distinguishing between migrant and LEP students with regard to assessment policies and procedures.
• Documentation that procedures are in place to inform parents/guardians of students with disabilities about appropriate assessment accommodations.

7.0 – REPORTING

• Documentation that performance descriptors for the WAAS and DAW (if used for AYP decisions) are appropriately included on student assessment reports.
• Timelines for reporting assessment results to parents.

Return to state-by-state listing

December 19, 2007

Honorable Richard H. Cate
Commissioner
Vermont Department of Education
120 State Street
Montpelier, VT 05620-2501

Dear Commissioner Cate:

I am writing to follow up on our conversation on December 19, 2007, in which you requested to amend Vermont’s timeline to address the remaining concerns regarding the technical quality and alignment to grade-level content standards of the Portfolio Assessment of Alternate Grade Expectations (PAAGE), Vermont’s alternate assessment based on alternate academic achievement standards for students with the most significant cognitive disabilities. As mentioned in my September 21 letter, Vermont set an ambitious schedule to complete the remaining work on the PAAGE in the 2007-08 school year. I understand, based on our recent conversation and correspondence with your staff that Vermont will no longer be able to adhere to this timeline. As a result, Vermont will not have in place an alternate assessment or alternate academic achievement standards that meet all the statutory and regulatory requirements of Sections 1111(b)(1) and 1111(b)(3) of the ESEA and 34 C.F.R. §§ 200.1, 200.2, and 200.6 until the 2008-09 school year.

Because Vermont continues to lack an approved alternate assessment, the status of Vermont’s standards and assessment system remains Approval Pending. Under this status, the condition on Vermont’s fiscal year 2007 Title I, Part A grant award will continue. Due the time it will take Vermont to address the remaining concerns, Vermont must enter into a Compliance Agreement with the Department, as authorized by Section 457 of the General Education Provisions Act. The purpose of the agreement is to enable Vermont to remain eligible to receive funding while coming into full compliance with applicable requirements as soon as feasible but within three years. The Department and Vermont will need to agree on the components of the agreement, including a detailed plan and specific timeline for how Vermont will accomplish the steps necessary to bring its system into compliance. For example, Vermont may need to contract with outside experts or technical assistance providers knowledgeable in the areas of non-compliance. In addition, before entering into the agreement, the Department must hold a hearing to explore why full compliance with the Title I standards and assessment requirements is not feasible until a future date. The State, affected students and their parents, and other interested parties may participate in the hearing. The Department must publish findings of noncompliance and the substance of the agreement in the Federal Register.

The compliance agreement must include a budget for each year the agreement is in place that demonstrates that Vermont has committed sufficient resources to correct the areas of non-compliance. The budget must reflect that Vermont will use a reasonable portion of its Title I, Part A administrative funds, in addition to State funds and funds it receives under section 6111 of the ESEA, toward improving its assessment system. The Title I, Part A State administrative funds that are used must supplement, not supplant, the State funds dedicated for this purpose.

I appreciate the steps Vermont has taken toward meeting the requirements of the ESEA, and I know you are anxious to receive full approval of your standards and assessment system. I have asked my staff to periodically visit with you and your staff throughout the term of the compliance agreement to discuss the progress that is being made to bring your alternate academic achievement standards and alternate assessment into compliance with the ESEA requirements, as well as any other concerns or issues that may arise. If you have any questions or would like to discuss this further, please do not hesitate to contact Sharon Hall (Sharon.Hall@ed.gov) or Abigail Potts (Abigail.Potts@ed.gov) of my staff.

Sincerely,

Kerri L. Briggs, Ph.D.

Enclosure

cc: Governor Jim Douglas
Gail Taylor
Michael Hock

SUMMARY OF ADDITIONAL EVIDENCE THAT VERMONT MUST SUBMIT TO MEET ESEA REQUIREMENTS FOR THE VERMONT STANDARDS AND ASSESSMENT SYSTEM

2.0 – ACADEMIC ACHIEVEMENT STANDARDS

1. 1. Evidence of approved/adopted alternate academic achievement standards for students with the most significant cognitive disabilities in reading/language arts and mathematics for each of grades 3 through 8 and at least one grade in the 10-12 grade span.
2. Documentation of the development of academic achievement descriptors for the alternate assessment in the content area of science.
3. Evidence that the alternate academic achievement standards include for each content area:
   1. a. At least three levels of achievement, including two levels of high achievement (e.g., proficient and advanced) that determine how well students are mastering a State’s academic content standards, and a third level of achievement (e.g., basic) to provide information about the progress of lower-achieving students toward mastering the proficient and advanced levels of achievement;
   2. Descriptions of the competencies associated with each achievement level; and
   3. Assessment scores (“cut scores”) that differentiate among the achievement levels.
4. Evidence that the Board or other authority has adopted all alternate achievement standards.
5. Documentation that the State has reported separately the number and percent of those students with disabilities assessed on the alternate assessment based on alternate achievement standards, those assessed on an alternate assessment based on grade-level standards, and those included in the regular assessment (including those administered that assessment with appropriate accommodations).
6. Evidence that the State has documented the involvement of diverse stakeholders in the development of its alternate academic achievement standards.

4.0 – TECHNICAL QUALITY

1. 1. Evidence that the State has documented validity (in addition to the alignment of the alternate assessment with the content standards), as described in the Standards for Educational and Psychological Testing (AERA/APA/NCME, 1999).
2. For the alternate assessments, evidence that the State has provided documentation of the standard setting process, including a description the selection of judges, methodology employed, and final results.
3. For the alternate assessment(s), evidence that the State has considered the issue of reliability, as described in the Standards for Educational and Psychological Testing (AERA/APA/NCME, 1999).
4. Evidence that the State has established:
   1. Clear criteria for the administration, scoring, analysis, and reporting components of its alternate assessment; and
   2. A system for monitoring and improving the on-going quality of its alternate assessment.

5.0 – ALIGNMENT

1. Evidence that the Alternate Grade-Level Expectations (AGEs) and all associated tasks across grade spans submitted for the PAAGE are aligned with State academic content standards in reading and mathematics.
2. Evidence that the State has developed on-going procedures to maintain and improve alignment between the alternate assessment and standards over time, particularly if gaps have been noted.

7.0 – REPORTING

1. Evidence that the State will produce individual student alternate assessment reports in terms of the State’s revised alternate achievement standards. With respect to such individual student reports:
   1. Evidence that these individual student reports provide information for parents, teachers, and principals to help them understand and address a student’s specific academic needs. This information must be displayed in a format and language that is understandable to parents, teachers, and principals, for example, through the use of descriptors that describe what students know and can do at different performance levels. The reports must be accompanied by interpretive guidance for these audiences; and
   2. Evidence that the State ensures that these individual student reports will be delivered to parents, teachers, and principals as soon as possible after the alternate assessment is administered.

Return to state-by-state listing