At the start, I believe that it is fair to note that
Appendix A of the
Common Core State Standards, which deals with text complexity,
recognizes the student as a critical consideration in determining the
readability of a text. Here is part of what that document says about matching
students to texts:
“[H]arder texts may be
appropriate for highly knowledgeable or skilled readers, who are often willing
to put in the extra effort required to read harder texts that tell a story or
contain complex information. Students who have a great deal of interest or
motivation in the content are also likely to handle more complex texts.”
And Appendix A adds significantly, “Teachers employing their professional judgment, experience, and
knowledge of their students and their subject are best situated to make such
appraisals.”
So the importance of the reader in any measure of
readability is universally acknowledged. Unfortunately, standardized tests
cannot match texts to readers like the child’s teacher can; therefore, the ease
or difficulty of any reading passage can never be fully ascertained. This is
why good standardized tests are pilot tested to determine if some passages
cause some readers comprehension difficulties not based on reading skills
alone.
Constructivist reading research has identified five things
about the reader that matter in the reader’s comprehension of text: reading
skills, reader prior knowledge, reader cognitive development, reader culture,
and reader purpose. Presumably, we would want a reading comprehension test to
measure student reading skills and cognitive development. We also know the
purpose of reading in the case of any standardized test is to answer questions
after reading. But what of prior knowledge and culture?
We would hope that the test would not advantage or
disadvantage a student because of differences in prior knowledge or
culture. We would hope, but years and
years of standardized testing have shown that this is never the case. Over
their many years of use, standardized tests have not been able to keep prior
knowledge and culture neutral in assessing reading comprehension.
Prior knowledge is critical to reading comprehension.
Essentially, the more you know about a topic before you read about that topic,
the better you will be able to comprehend that material and the better you will
be able to accommodate any new material you encounter during that reading. In
an article titled,
Individual Differences that Influence
Reading Comprehension, Darcia Narvaez, a professor at the University of
Minnesota, puts it this way, “If a person [who] has a great familiarity with a
grocery store reads a text about a grocery store, the person will activate a
grocery store script.” This script aids the reader in making sense of text. If
the reader has little or no script to bring to the text, say if the script were
about a trip to an outdoor market in Morocco, the comprehension could be more
limited.
“This item involved a
passage about a visit of Captain Cook to a group of islands in the South
Pacific. The critical section was,
‘he called them the Friendly
Islands because of the character of their people. Today, the Tongans still
provide visitors with a warm welcome.’
The test item asked
for the meaning of the word character as
it was used in the story. Most whites chose nature, the answer scored as correct. Blacks frequently chose style. This is a term used more in
black than white communities, and it can be argued that in its colloquial sense
style is more apt than nature as a synonym for character. It is apparent, at
least, that style is not a wrong
answer.”
With all the study that has been put into issues of cultural
bias, we can expect that passages and questions from any reputable test
developer will be vetted for this bias, but again, these issues can be subtle
and might best be accounted for through extensive pilot testing.
Now let’s take a look at the sample passages in the PARCC
that I have discussed in the previous two posts and see how they measure up on
the background knowledge and cultural bias scale from one literacy specialist’s
perspective.
Grade 3 – A Once in a Lifetime Experience
This is a pleasant and innocuous story typical of the
Highlights for Children magazine from which it was taken. As I said in a
previous post, the quantitative readability seems appropriate. The story
involves a camping trip with two friends and a dad. Some background knowledge
on camping, fishing and boating might advantage students over those who have no
such experience, but I do not think the impact would be great. More
problematic, for me, is that answering one of the questions requires knowledge
of the word “jostle.” The word cannot be clearly defined through context and I
would not deem the word as one that most third graders would know. While the
text provides students with the definition of “bail” and “adrift”, students are
left to their own devices on “jostle.”
Grade 4 – Just Like Home
As I stated in my first post in this series, Just Like Home,
by every quantitative measure, is too difficult for use in a reading
comprehension test for fourth grade. As far as cultural bias or giving
advantage to some who might have greater background knowledge, the concern is
slight. The story is an obvious attempt to include a multi-cultural perspective
to the test, and that is a good thing. It takes place on a school playground
and the concerns seem fairly universal. My favorite part of the story is when
the protagonist, Priya, says that the only thing she likes about her new school
is art, because she did not have art in her old school. I immediately thought
that art had been cut in her previous school so that the kids could do more
test prep, but now my bias is showing.
Grade 5 – Moon Over Manifest
This passage is taken from the 2011 Newbery award winning
book by Clare Vanderpool. The passage is well written, but it has some
characteristics that may make it challenging for some readers. The story is set
in the 1930s and references to such things as pocket watches, satchels,
storefronts and bustling townsfolk may prove problematic to some. Typical of
Newberry award books, the author uses lots of rich figurative language that
puts a further burden on the reader, fine for some fifth grade readers, but
challenging for others with a less rich background knowledge. Finally, the
passage assumes knowledge of what has gone before in the story, the main
character is a veteran of hitching rides on trains with her father, which may
throw off some fifth grade readers.
Grade 6 – Emancipation:
A Life Fable
This fable, written in the 1860s by the well-regarded
proto-feminist author, Kate Chopin, seems an odd choice for a test passage.
Because it was written 150 years ago, it is replete with word choices and
sentence constructions that may be unfamiliar to 11- and 12-year-old readers.
Here are two examples:
Here he grew, and
throve in strength and beauty under the care of an invisible protecting hand.
Hungering, food was ever at hand.
Back to his corner but
not to rest, for the spell of the Unknown was over him, and again and again he
goes to the open door, seeing each time more Light.
The fable is actually a good example of why quantitative
readability measures like Lexiles are problematic. They cannot measure the
impact of arcane language or parse the allegorical nature of a fable. This is
an extremely challenging passage for a sixth grader, one that would be best
used in the classroom with plenty of support from the teacher, certainly not in
a testing situation.
Grade 7 – from The Count of Monte Cristo
The Count of Monte Cristo is, of course, the classic
adventure novel written in the 1840s by Alexandre Dumas. Written in French, the
text has been translated and updated many times over the years. The translation
used by PARCC is identified as in the Public Domain, so I would assume the test
passage is from an older translation. We know that older texts, which employ
language patterns and structures that are unfamiliar to many children, provide
a greater reading comprehension challenge than more contemporary texts, so once
again the difficulty of this text cannot be accurately measured by a Lexile
score.
The text is replete with vocabulary that seventh graders
will find challenging and that will impact their comprehension. Words in the
passage include countenance, lucidity,
well-nigh, loathsome, delirious, ascertain, recurrent. While some of these
words might be determined through context by a skilled reader, most cannot. The
setting of the story, a dungeon in early twentieth century France during the
Bourbon Restoration, also would provide readers with a unique challenge.
The Count of Monte Cristo is a wonderful adventure story and
would make entertaining reading for a certain subset of skilled middle school
readers. The advisability of using it as general reading in a reading
comprehension test for seventh graders is highly questionable.
Grade 8 – Elephants Can Lend a Helping Trunk
This is a non-fiction passage describing an experiment conducted
by a team of scientists to test the social cognition of elephants. The passage
is clearly written, is cohesive, and for the most part does not present an
extraordinary vocabulary challenge. An included photo should help students
visualize the experiment.
A minor quibble with this passage is that it was clearly
written by a person who uses United Kingdom English. So crows are called “rooks”
and some sentence patterns are slightly different than American English.
The biggest concern with this passage is that the
understanding of the entire passage depends on the understanding of the words “cognitive” and “cognition.” Indeed the first two questions deal with these two
terms. The main body of the passage discusses the elephants’ skills at social
cooperation, but the text never draws a clear line from the understanding of
cooperation and an understanding of “cognition” as mental processes. I think
this passage is appropriate for a high school class in psychology, but question
it as a test passage for 8th graders. I recently had a discussion
with my college freshman class about the term “cognition” and some of its related
terms cognate, recognize, metacognition,
cogitate. I can report that the word cognitive was not in most of my
college freshmen’s vocabulary.
Standardized tests by their very nature cannot be well
matched to individual readers. Text matching takes a skilled and informed
teacher with deep knowledge of her students, the reading to be done and
the task to be completed. It is not fair to ask standardized tests to meet
these criteria. I believe it is also not sound test science to choose as test
passages those passages that contains vocabulary beyond most students in the
grade, passages whose targeted audience was adults not children and passages
that use archaic language and sentence structures.
The high readability of most of these passages, the unique
challenges of the questions asked on the test and the failure of some of the
passages to be considerate of the background knowledge and culture of many of the children who will be encountering
these tests, guarantees that many children will struggle. Ultimately, the results of the PARCC tests will tell us more about the tests themselves than it tells us about the students taking the tests. One thing it will tell us, I believe, is that these tests are not useful for making any high stakes decisions about individual children, teachers, or schools.
The one thing I can guarantee as an outcome of these tests is
that, overall, children living in areas of affluence will do considerably better than
children living in areas of poverty. I can guarantee this because it is true of every
standardized test ever given, and so it always shall be. If education reformers
want to learn from standardized tests, this is the lesson to be learned.
The real issue in education is inequity, not the ability of a seventh grader to
parse The Count of Monte Cristo.
