FY 2002

Enhancing the Assessments of Students with Disabilities: A Proposal for a Multi-State Special Education Collaborative

Abstract

The Colorado Department of Education is submitting this proposal as the lead agency for a Special Education Assessment Collaborative. The Collaborative is comprised of states and not-for profit organizations who desire to improve how students with complex disabilities are alternately assessed in the content areas of English language arts, mathematics and science. Colorado and the Collaborate states wish to enhance assessment efforts already in place to better meet the new accountability challenges of the NCLB Act and state accountability requirements. The Collaborative will strive to improve a variety of alternate assessment strategies, by increasing their technical soundness, accessibility, efficiency and feasibility for measuring adequate yearly progress. It will then examine the nature of the information derived from the multiple measures. The assessment methods will be developed, pilot tested and analyzed in the course of this project.

Seven tasks will be undertaken. Three states will provide leadership on development projects, five other participating states will select the components in which they wish to participate. Four organizations will assist with various aspects, as described below.

1. A consensus standards framework will be developed by Measured Progress that will serve as the foundation for the improved assessment methods.

2. Colorado, with Measured Progress, will lead in developing enhanced performance assessments.

3. Iowa, with the Inclusive Large Scale Standards and Assessment group will develop instructional and assessment modules for use in portfolios.

4. Iowa will partner with Measured Progress to improve observation, interview and data collection strategies.

5. The Center for Applied Special Technologies will assist in the production of the standards frameworks and assessment methods to be more accessible through application of universal design principles.

6. Research will be conducted by Oregon to investigate the technical attributes of the individual measures and to determine the effects of multiple measures.

7. Colorado State University will conduct an evaluation of the processes and products of the project.

Improving the Achievement of English Language Learners through Authentic Proficiency Assessments

Abstract

The Minnesota Department of Children, Families & Learning (CFL) in collaboration the Nevada Department of Education, the South Carolina Department of Education and the Wyoming Department of Education propose to develop new and innovative assessment tools to measure the progress of students with limited English proficiency based on the elements of universal assessment design and proficiency-oriented assessments for second language learners. The University of Minnesota’s National Center on Educational Outcomes (NCEO) and the University of Minnesota’s Center for Advanced Research on Language Acquisition (CARLA) will also collaborate on this project to enhance the development of assessments for English language learners and to increase the validity of state assessments for all groups of students.

Information technology (IT) tools will be used to pilot methods of language assessment, develop new methods to organize, collect, score student assessment data and combine data from multiple measures to facilitate the evaluation of student progress over time. Staff development will be designed to expand teacher use of assessment results to improve instruction. These methods will be designed with the intent of expanding their use to other state assessment programs in the near future. The objectives of this project are to (1) use principles of universal assessment design to review existing assessments and develop a process for incorporating their use into the development of new assessments required by NCLB including standards based listening and speaking proficiency assessments for LEP students and standards based assessments for students in grades three through eight, and high school in reading and mathematics; (2) use IT tools to develop authentic, contextually based listening and speaking assessments, pilot innovative data collection and scoring methods; and (3) develop a web-based framework for on-going staff development to help teachers interpret scores from state assessments, and use that information to make appropriate instructional adjustments, for the purposes of school improvement within the context of language development standards and state content standards.

Proficiency on state assessments is the primary indicator in ESEA of student academic achievement. States must develop educational systems that provide opportunities for maximum access to content and high expectations of learning for all students including students with limited English proficiency. It is critical that state assessment programs use measures of progress that are accurate and fair for all groups.

Design and Development of an English Language Development Assessment

Abstract

The Nevada State Department of Education will serve as the lead state in a collaborative with 16 other states including Indiana, Iowa, Kentucky, Louisiana, Massachusetts, Michigan, Nebraska, Nevada, New Jersey, New York, Ohio, Oklahoma, Oregon, South Carolina, Texas, and West Virginia to design and develop an English language development (ELD) assessment over the period from December 2002 through September 2004. The proposed project addresses the need for states to implement an assessment, which measures the annual growth of English language development in the domains of speaking, listening, reading and writing among limited English proficient students. As a result, the project will produce intact core test forms and an item bank from which states can draw to create test forms that reflect local needs and characteristics. This new assessment will 1) measure the progress of LEP students in their development of English language proficiency (for a maximum of three years), (2) determine status with reference to English proficiency standards required at each grade level in pre-K-12, and (3) predict an LEP student’s readiness for English language assessment. In addition to the design and development of an ELD assessment, the collaborative will field test the assessment and participate in standards setting workshops and validity studies. The collaborative will also be trained on how to implement, score, and use the results of the ELD assessment.

The American Institutes of Research (AIR) will assist the Nevada State Department of Education in the Implementation of the project and be responsible for overseeing the development of the assessment framework and specifications blueprint, and for the development of all core assessment items and non-core items.

Improving Alignment Tools for Enhanced, More Accessible Assessments: Development of an Electronic, Automated Alignment Analysis for States

Abstract

The Oklahoma State Department of Education will serve as the lead state in a collaborative with 15 other states including Alabama, California, Delaware, Kansas, Louisiana, Massachusetts, Minnesota, New Jersey, North Carolina, Pennsylvania, South Carolina, Texas, Wyoming, West Virginia, and Wisconsin. The group proposes to expand and automate an alignment process for judging the match of assessments with content over the period from December 2002 through September 2004.

The proposed project addresses very specific needs for improving assessments so that they 1) will more closely match content standards, 2) will be applicable for students with disabilities, and 3) will increase capacity for linking across grades. The collaborative, organized through the Council of Chief State School Officers, will incorporate into the process, along with the automation, the modifications necessary to 1) make the process valid for assessing special populations of students and 2) expand the procedures used so they will be applicable with every-grade assessments.

The proposed project will make the alignment process system available on a CD-ROM that can be readily distributed to states, thus increasing the use of the alignment tool in assessment development and verification. Along with automating the process, the collaborative will generate the necessary decision-making rules for aligning assessments to content standards.

As a culminating activity of the project, the collaborative will host a workshop to train state assessment, curriculum, and Special Education staff in use of the alignment process. An important outcome of the project will be the enhanced capacity of state department of education staff members to improve their assessment programs.

Assisting Oklahoma in the project implementation will be the Council of Chief State School Officers and staff from higher education, including researchers from the University of Wisconsin Madison, the University of Oregon, the Human Resources Research Organization, and WestEd.

Proposal of the English Proficiency for All Students (EPAS) Consortium

Abstract

Giving all children the chance to succeed, regardless of their background, is the central purpose of our nation’s schools. As America’s diverse population continues to grow, more and more students are entering our education system with limited English proficiency. These students are faced with major, often debilitating obstacles as they begin the transition from their native language to English. The attached proposal offers a plan that will help ease this difficult journey.

The states represented in this Consortium believe that English language learner (ELL) students should participate in meaningful assessments of what they are learning, providing the state with data to help address their educational needs. The states agree that the attached proposal not only complements current federal mandates, it goes beyond the requirements of ESEA, Title I, Part A.

This proposal envisions a true collaboration between a Consortium of states, AccountabilityWorks, and Educational Testing Service for the development of scientific-based solutions to ELL-based problems. This project is designed to assist participating states in assessing and providing results for all ELL learners. The states have selected AccountabilityWorks and ETS to serve as subcontractors on the project because of their excellent and relevant qualifications to perform the work involved.

Pennsylvania will serve as the Consortium’s lead state and fiduciary agent. Grant funds will flow from the lead state to AccountabilityWorks, who will collaborate with state educators, top researchers and practitioners to analyze relevant state standards and establish content benchmarks. AccountabilityWorks will pass these benchmarks to the ELL experts from ETS, who will then collaborate with teachers in each state to develop standards-based assessments drawn from scientific research on English language acquisition for K-12.

The New England Compact, a Four-State Consortium to Enhance the Quality of Their State Assessment Systems

Abstract

This project grows out of an existing collaboration formed by the Commissioners of Education of four states, Maine, New Hampshire, Rhode Island, and Vermont. These states have been working together since fall, 2001 to discuss and address the changes to their state accountability systems under the No Child Left Behind Act (NCLB). During the year, the informal collaboration was formalized to become the New England Compact. This Compact proposes to leverage the power of its shared commitment to maintaining challenging standards and the important role that local practitioners play in helping to design a state assessment system, and human resources of the four states in order to improve the achievement of all students through the development of comprehensive academic assessment instruments, particularly technology-based assessments that are designed to meet the needs of students with disabilities, limited English proficient students and other students who are at-risk. This proposal goes beyond the requirements for the assessments described in Section 1111(b)(3) of Title I, Part A of the NCLB Act in the following ways.

First, the states will create common, priority standards from which they will create a test blueprint. The states will then be able to compare progress across the states and combine resources to develop highest quality assessments. Second, the states will conduct a series of design experiments that focus on the impact of computer-based testing and accommodations on the validity of test scores for students with and without special needs, resulting in the development of exemplars for further development. Teachers will participate in the design process. Third, a group of teachers will be trained to provide professional development to teachers in their states on how to create and use assessments that are aligned to the state’s standards and that use the same accommodations, design and alternatives for students with disabilities, limited English proficient students, and students who are at risk of academic failure.

There are six broad goals.

Goal 1. The Compact will establish a set of common, priority standards termed ‘common expectations’ in English/language arts and mathematics in grades 3-8 and high school.

Goal 2. The Compact will create a test blueprint, based on the common expectations that will be used to cooperatively develop grade-level assessments in reading and mathematics that reflect the developmental issues of young children, the learning differences among all children, and the special needs of students with disabilities and limited English proficient students.

Goal 3. The Compact will develop and validate up to four assessments that are based on the common expectations and are designed to accurately measure academic content and skills achievement by limited English proficient students and students with disabilities.

Goal 4. The Compact will build the capacity of teachers within the Compact states to engage in effective classroom assessment and uses of assessment data from both state and local sources.

Goal 5. The Compact will enhance the states’ AYP systems to make them capable of demonstrating progress within the range of each category of performance and each disaggregated student achievement category at the individual school level.

Goal 6. The Compact will disseminate its products throughout the country to other state departments of education.

The resources for this project are substantial. The project is guided by a Research Technical Advisory Committee, representing national, regional and state experts in assessment, accommodations, universal design, mathematics, reading, and standards. These experts also bring the resources of their organizations, including two leading institutions of higher education, the Center for the Study of Testing, Evaluation and Educational Policy at Boston College and the Alliance at Brown University; resource-rich non-profit organizations, including the Annenberg Institute for School Reform, CAST, Educational Development Center, Inc., the Council of Chief State School Officers, the Center for Research on Evaluation, Standards, and Student Testing, the National Center on Educational Outcomes, TERC, and WestEd; and all the regional service providers funded by the U.S. Department of Education. Both the service providers and the state departments of education are providing resources beyond the funding from the Enhanced Assessment grant.

South Carolina Taxonomy for Testing English Language Learners (TTELL)

Abstract

With increasing attention being paid to the issue of obtaining valid information about English language learners’ (ELLs) academic knowledge and skills, different forms of accommodation have been suggested in the assessment of this population. However, no research has been done to date to address how the large scale testing needs of English language learners in schools should be systematically identified and then systematically matched to specific accommodated methods. Thus, the project proposes to develop and evaluate a rigorous taxonomy model that would appropriately and systematically define and identify different types of English language learners based on their testing needs, and then provide the mechanisms that match these students to the proper accommodation methods. Furthermore, the project will also investigate the validity of the proposed decision match.

This research and development project is a collaborative undertaking between South Carolina Department of Education, the University of Maryland, Maryland State Department of Education, North Carolina Department of Public Instruction, District of Columbia Public Schools, Austin Independent School District, the American Association for the Advancement of Science, and a legal firm that specializes in federal and local educational law. The collaboration of multiple states and districts will allow the project to research the appropriateness of the work at both state and local level. This consortia will ensure that the products are technically, legally, and academically defensible, and that they are useful for educators at different levels of schooling and over a variety of assessment and accountability systems.

This project involves three phases: 1) collecting and analyzing the identification of differential assessment needs of ELLs through discovery interviews and survey information, 2) developing a taxonomy and process model to match access needs with appropriate large scale testing accommodations and develop dissemination materials, and 3) investigating the technical rigor of these tools and the overall decision-making process.

Improving the Assessment of Hispanic, Native American, and Other English Language Learners

Abstract

The following is an application for work that the Utah State Office of Education is submitting on behalf of the Mountain West Assessment Consortium. Utah and the Mountain West Assessment Consortium propose to accomplish the development of improved assessments of English language proficiency by external funding through the Enhanced Assessment Instruments Competitive Grant Program (Title VI, section 6112) as well as non-grant funds that the consortium partners will provide. The Mountain West Assessment Consortium is a group of states in the mountain west and northern plains regions. Measured Progress, Inc., a not-for-profit educational assessment organization, is also a partner in the consortium.

The goal of this collaborative effort is to develop a series of academically oriented assessments of English language proficiency at four levels (K-3; 4-6; 7-9; 10-12). This series of assessments will be designed to enable teachers to more effectively diagnose the level of English language proficiency of English language learners in their classrooms. Levels of English language proficiency (pre-emergent, emergent, intermediate, fluent, advanced) will be assessed for different language skills (listening, speaking, reading, writing, comprehension) using classroom learning activities and technical vocabulary for each major content area (English language arts, mathematics, science, social studies). Teachers will therefore be able to provide more appropriate academic instruction to English language learners. This, in turn, would lead to better progress of English language learners toward proficiency in the major content areas.

The scope of the project is divided into Phase I activities and Phase II activities. During Phase I, which will occur in the fall of 2002, the consortium will undertake on its own to define a consensus set of learning activities and technical vocabulary for the content areas; develop a consensus set of English language development standards, and develop a draft test blueprint (test framework). During Phase II, which will occur during the grant period from December 2002 through September 2004, the series of assessments will be developed. The major activities include: validating the test blueprint; developing the test items; conducting pilot tests and field tests in each state; conducting statistical analysis of item performance and test reliability; conducting validity studies with other quantitative and qualitative data; setting standards for English language proficiency cut points; and developing operational tests forms, along with ancillary test materials such as administration manuals, scoring guides and demonstration materials.

Special features of this assessment series include test content that is academically oriented; a test design that is diagnostic providing a clear link to appropriate instructional interventions; test content that is based on the target language (English) rather than on the dominant language of the English language learner so that the assessment series can be used for students of any language/cultural background; technical information available about the assessment series as a result of the extensive set of reliability and validity analyses that will be conducted; and delivery of a complete assessment package that the consortium states can use and make available to other states at the end of the grant period.

Enhanced Assessment Instruments for Limited English Proficient Students

<p.Abstract

Goal 1: To develop State standards-based assessment instruments for measuring English proficiency and literacy skills for LEP students.

Goal 2: To develop and enhance State standards-based alternate assessment with alternate performance indicators (APIs) for measuring academic performance of LEP students in math, science, and social studies.
Goal 3: To improve and promote English language acquisition and Academic achievement through technology, this project will develop, adopt, and enhance computer and technology skills with performance indicators related to English language acquisition, literacy skills development, academic content learning, and career path exploration for English language learners. Technology-based assessment is an integral part of this goal.
Goal 4: To provide training-of-trainers on assessment instruments designed for measuring English language proficiency, literacy skills, and standards- based alternate assessment (API/MECCA) for prospective trainers at local educational agency (LEA) and institution of higher education (IHE) levels:
Goal 5: To collaborate with institutions of higher education, local educational agencies, and other research institutions in the development, research, and administration of assessment instruments.
Goal 6: To disseminate information on the development and administering of assessment instruments at state and national levels.
Goal 7: To conduct research on the validity and reliability of the assessment instruments.