PARCC Math Test Readability

Two weeks ago I ventured into the world of PARCC testing with several posts on the readability of the reading comprehension passages of the new Common Core aligned PARCC tests, which are being administered right now in many states. You can find those posts here, here and here. Some readers expressed concern about the readability of the PARCC math exams and asked me to take a look at it.

Background: Readability on a math exam matters. While we might assume that a math exam assesses a students ability to perform various mathematical computations, all of the math questions on the PARCC required some literacy skills as well. A study by Abedi and Lord published in Applied Measurement in Education found that linguistic complexity of math word problems can have a significant impact on the test scores of inexperienced problem solvers, English Language Learners and students with disabilities. The question that must be asked is simply, "Does the PARCC measure computation skills or a combination of literacy skills and computation skills?" And we might further ask, "Will students with on grade level reading skills be disadvantaged by the reading required on the math exam?"

Method: I will not rehash all my reservations about readability measures here. You can look at the posts on the reading comprehension part of the PARCC if you would like a fuller explanation. Suffice it to say here that readability measures can only give us an approximation of the difficulty of any one text on any one reader, so all results need to be taken with a grain of salt.

For the purposes of this post, I looked at the PARCC Mathematics Practice Tests. In order to get a sample of 300 hundred words to do a readability measure, I sampled word problems from the beginning, middle, and end of the test. I hoped this would give me a sense of the readability of the word problems. I ran the passages for each grade level 3-8 through several readability formulas: Lexile, Flesch-Kincaid (FK), Fry and Flesch Reading Ease (FRE) scale. Lexile is the preferred readability formula of the Common Core architects and are expressed as grade ranges. These Lexile ranges were adjusted upward as a part of the Common Core's push for "college and career readiness."The other scales are commonly used readability measures. The Flesch Reading Ease Scale provides a number rating based on 90-100 being easy reading for 11-year-olds and 60-70 being easy for most 12-13 year-olds.

Findings: 

• 3rd Grade
• Lexile        830 (3rd grade range is 520 - 820)
• FK              4.6 grade level
• Fry             4 grade level
• FRE            81.6
• 4th Grade
• Lexile        890  (4th Grade range is 740 - 940)
• FK              5.1
• Fry             5.3
• FRE           80.3
• 5th Grade   
• Lexile         820 (5th Grade range is 830 - 1010)
• FK              5.4
• Fry             5.8
• FRE           77.8
• 6th Grade
• Lexile        1000 (6th grade range is 925 - 1070)
• FK              4.6
• Fry              5
• FRE            77.8
• 7th Grade
• Lexile         810 (7th grade range is 970-1120)
• FK              4.8
• Fry             5.2
• FRE           80.5
• 8th Grade
• Lexile        1000 (8th grade range is1010-1185)
• FK              8.1
• Fry             8.2
• FRE           60.9

Discussion: 

1. On grade level readers in grades 3 and 4 are going to find the reading required on the PARCC math tests to be very challenging. This will surely impact their scores on the test.
2. On grade level readers in grades 5-8 should be able to handle the reading demands of the test.
3. Below grade level readers, English Language Learners, and students with disabilities related to language processing will find the reading required for these tests very challenging. This will impact their scores on the PARCC math tests.

Conclusions: Because readability formulas are volatile and inexact, we must draw conclusions carefully. I have not examined  the qualitative aspects of these texts (how readable do they appear in light of the age of the children reading them); however, some tentative conclusions can be drawn from this initial look.

1. Teachers, administrators and parents must treat the results of the PARCC math tests with extreme caution.  The math test scores will surely be influenced by the ability of the students to read the material. Separating out what is a computational weakness and what is a reading weakness will be left to the observation and intervention of the classroom teacher.
2. Questions must be asked about the validity of the test scores in grades 3 and 4 based on the challenging level of the reading. In third grade, even by the revised and Common Core championed Lexile measures, the reading is very challenging and perhaps inappropriate for the age and grade level of the children.
3. These tests are clearly too imprecise to be used for any kind of high stakes decisions, including student placement, teacher evaluation or school effectiveness. Any attempts to use these tests for such purposes will be fraught with error and would have potentially damaging results for children, teachers, parents and schools.