Our K-8 math review teams are comprised of expert educators from across the country. The teams, consisting of reviewers, review team leaders, and calibrators, meet weekly in virtual professional learning communities to complete their reviews. Review teams consider materials by grade span (typically K-2; 3-5; 6-8) to produce a set of grade-level reports for each series under review. Each grade-level report includes detailed evidence for each indicator and grade-specific alignment and usability determinations.
All reviewers consult the Quality Instructional Materials Tool for K-8 Mathematics and related evidence guides for each indicator that provides looks-fors, references to key documents, and scoring details. Team members spend on average more than five hours per week over the course of several months independently conducting reviews and also meet weekly with their teams in a virtual PLC.
Focus and Coherence (Gateway 1)
Teams begin their reviews by looking at Focus and Coherence indicators that are found in Gateway 1. Focus and Coherence consists of three criteria and six indicators. Review teams begin on a single grade level to independently find evidence, calibrate on that evidence during weekly PLCs, and then determine a score for each indicator.
Most indicators in the Quality Instructional Materials Tool for K-8 Mathematics have a 2-point scale.
- 2 - A rating of 2 means that the materials meet the full intention of the indicator.
- 1 - A rating of 1 means that the materials partially meet the intention of the indicator.
- 0 - A rating of 0 means that the materials do not meet the intention of the indicator.
Materials must meet or partially meet expectations for Focus and Coherence in order to be reviewed for Rigor and Mathematical Practices (Gateway 2). Educators designed this principle of our review tool to ensure that only materials that cover the correct grade level standards with sufficient depth and that are coherent and consistent with the standards will be reviewed for rigor, mathematical practices and usability.
EdReports.org's emphasis on the importance of Focus is evident in the scoring of the first two indicators of Gateway 1 (1a and 1b). Materials cannot receive a “partial” rating for these two important indicators. Indicator 1a can only be scored as a 2 or a 0; Indicator 1b can only be scored as a 4 or a 0. Focus indicators include identifying that instructional materials assess the appropriate grade-level content and that the materials spend the majority of class time on the major clusters of each grade.
The coherence indicators in Gateway 1 score the degree to which supporting content enhances focus and coherence, the amount of content designated for one grade level is viable for one school year, that the materials are consistent with the progressions in the standards, and that materials foster coherence through connections at a single grade.
Rigor and Mathematical Practices (Gateway 2)
Materials that meet or partially meet expectations for focus and coherence are then reviewed for rigor and the mathematical practices. This gateway is comprised of two criteria and seven indicators.
For the rigor criterion, reviewers look to see if each grade’s instructional materials reflect the balances in the standards and help students meet the standards’ rigorous expectations by helping students develop conceptual understanding, procedural skill and fluency, and application.
For the mathematical practices criterion and sub-indicators, review teams gather evidence and scores for how well materials meaningfully connect the Standards for Mathematical Content and the Standards for Mathematical Practice.
Instructional Supports and Usability (Gateway 3)
Only materials that fully meet the expectations for the first two gateways will be reviewed for Instructional Supports and Usability (Gateway 3). The last set of indicators that our reviewers examine are around how well materials support student learning and engagement and support teacher learning and understanding of the Standards. They also look to see if materials also offer supports to differentiate instruction for diverse learners and enrich instruction through technology. There are four scored criteria and one non-scored criterion. For ‘Effective Technology Use,’ indicators are not rated but evidence is still collected to be included in the review. EdReports.org considers technology use to be an important element of usability, but since printed and online materials vary widely in their use of technology we are not scoring these indicators at this time.