Data Quality Component 1: Data Collection
Data collection refers to the activities involved in identifying data to be collected and possible data sources. Data collection also includes identifying who will be collecting the data and the data collection timeline. The following considerations outline how state educational agency (SEA) staff may consider how new or revised data collection procedures play a role in the larger data quality system.
- Identifying data elements and collection sources: SEA staff are responsible for identifying the specific data elements (i.e., types of data, such as number of student behavioral incidences and description of behavioral incidences) for which to collect data. Depending on the data previously collected, the SEA may need to add a new data element, potentially including new subgroup-level data—for example, a data element to capture the new ESSA requirement related to students in foster care. SEA staff will also need to ensure that the data element will allow them to meet reporting requirements under ESEA. Once the data element(s) to be added are identified, SEA staff will need to identify the specific data to be collected and what the data sources will be.
- Defining data collection processes: SEA data collection processes are governed by a variety of policies, which may be set at the program, agency, and state levels. As new data elements are identified, SEA staff will need to determine whether updates need to be made to program, agency, or state policies and processes for collecting these new data. These updates should be done with consideration of the source(s) for the data (i.e., whether the data come from schools, districts, or partner organizations) and the responsible parties for collecting the data. SEA staff will need to ensure a process is in place for collecting the data or determine whether a new process should be established. SEA staff should also include data governance groups and local stakeholders in this decision-making process, including determining the timeline and communication strategies needed to explain any changes to current processes.
Data collection for statewide K–12 educational reporting and analysis involves numerous levels across various groups and individuals. After a data element is defined and created in the SEA system, schools and districts are responsible for entering data that are collected by the state systems. Ensuring consistent, high-quality data collection across time and a myriad of schools and districts poses many challenges:
- As data collection requirements are revised and expanded, data collection for multiple program needs has the potential to create redundancy and take time away from other important work.
- It can be a challenge for schools and districts to juggle multiple reporting systems, duplicative reporting requirements, and various data file formats for the variety of different data they must collect. A data collection system that is cumbersome for users can exacerbate other challenges.
- Schools, districts, and states may face the challenge of time constraints. If the data collected are to be used for intended purposes, the collection and reporting must be timely. If there is too great a lag between data collection and the availability of reports, the data can be regarded as invalid at worse and unhelpful at best.
SEAs can assess how new data collection processes may present or exacerbate these types of challenges and refine their data collection processes to be more effective.
SEAs can use the Data Systems Self-Reflection Checklist to consider how their current actions promote quality data systems. To download the Data Systems Self-Reflection Checklist, click here.
For more information on how Colorado is currently approaching data collection, please see the vignette below.
|Data Collection Vignette: The Colorado Data Pipeline|
To combat common data collection challenges (e.g., duplicative reporting requirements, multiple types of data file formats), the Colorado Department of Education built a data collection system called the “Data Pipeline”—a secure, browser-based common platform for data collection. This platform enables common collection and data technology across the different sources and levels of data. Implemented in 2013–14, the system was designed to reduce data redundancy and streamline the data collection process. The system accepts multiple file formats rather than just text-based files, allows for online error correction via a “format checker” instead of uploading data multiple times, and permits differentiated data access for various users, i.e., allowing different users access to certain data and protecting privacy. Data can be updated year-round by district and school staff, allowing for real-time data and ensuring accurate data. The Data Pipeline has enabled the SEA to move from a program-centric collection system to a student-centric collection system, thereby increasing reliability and usefulness of the data to inform decisions about student needs. Tracking of student-level data is enhanced by integrating data from the state’s Record Integration Tracking System (RITS) and Educator Identifier System (EDIS). This integration allows the system to track students who move to a different district, providing needed information to the new district while notifying the previous district.
To support users of the Data Pipeline, the SEA maintains a calendar of Data Pipeline town hall webinars and archives the meetings online for ongoing access and reference. To learn more about the Colorado Department of Education Data Pipeline, including a quick reference fact sheet, please visit: https://www.cde.state.co.us/datapipeline.
Data Collection Resources
Common Education Data Standards (CEDS) Program:
Common Education Data Standards. Using CEDS Tools to Compare State Data Collections and Federal Requirements: An Example from California. Washington, DC: Common Education Data Standards, 2017.https://ceds.ed.gov/pdf/Using_CEDS_Tools_to_Compare_State_Data_Collections_and_Federal_Requirements_Spring_2017.pdf.
SLDS Military Connected Workgroup. Military Student Indicator – ESSA. Washington, DC:Common Education Data Standards, 2017. https://ceds.ed.gov/connectReport.aspx?uid=1833.
State Longitudinal Data Systems (SLDS) Grant Program:
Chatis, Corey, and Kathy Gosa. SLDS Issue Brief: Considerations for Collecting New Data Elements. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, 2016. https://slds.grads360.org/services/PDCService.svc/GetPDCDocumentFile?fileId=18636.
Henderson, William, and Missy Coffey. SLDS Webinar: Including Homeless Data in an ECIDS/SLDS. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, 2016. https://slds.grads360.org/#communities/pdc/documents/13248.
United Nations Educational, Scientific, and Cultural Organization (UNESCO)
UNESCO Institute for Statistics (2017). The Quality Factor: Strengthening National Data to Monitor Sustainable Development Goal 4. Montreal, QC: UNESCO, 2017. http://uis.unesco.org/sites/default/files/documents/quality-factor-strengthening-national-data-2017-en.pdf.