Nonpartisan Education Review / Testimonials
Access
this testimonial in .pdf
format
DC Education Reform Ten Years After,
Part 2: Test Cheats
Richard P Phelps
Ten years ago, I worked as the Director of Assessments
for the District of Columbia Public Schools (DCPS). For temporal context, I
arrived after the
first of the infamous test cheating scandals and left just before the
incident that spawned a
second. Indeed, I filled a new position created to both manage test
security and design an expanded testing program. I departed shortly after
Vincent Gray, who opposed an expanded testing program, defeated Adrian Fenty in
the
September 2010 DC mayoral primary. My tenure coincided with Michelle Rhee’s
last nine months as Chancellor.
The recurring test cheating scandals of the
Rhee-Henderson years may seem extraordinary but, in fairness, DCPS was more
likely than the average US school district to be caught because it received a
much higher degree of scrutiny. Given how tests are typically administered in
this country, the incidence of cheating is likely far greater than news
accounts suggest, for several reasons:
· in most cases, those who administer tests—schoolteachers
and administrators—have an interest in their results;
· test security protocols are numerous and complicated yet,
nonetheless, the responsibility of non-expert ordinary school personnel, guaranteeing
their inconsistent application across schools and over time;
· after-the-fact statistical analyses are not legal
proof—the odds of a certain amount of wrong-to-right erasures in a single
classroom on a paper-and-pencil test being coincidental may be a thousand to
one, but one-in-a-thousand is still legally plausible; and
· after-the-fact investigations based on interviews are
time-consuming, scattershot, and uneven.
Still, there were measures that the Rhee-Henderson
administrations could have adopted to substantially reduce the incidence of
cheating, but they chose none that might have been effective. Rather, they dug
in their heels, insisted that only a few schools had issues, which they
thoroughly resolved, and repeatedly denied any systematic problem.
Cheating scandals
From 2007 to 2009 rumors percolated of an
extraordinary level of wrong-to-right erasures on the test answer sheets at
many DCPS schools. “Erasure analysis” is one among several “red flag”
indicators that testing contractors calculate to monitor cheating. The testing
companies take no responsibility for investigating suspected test cheating,
however; that is the customer’s, the local or state education agency.
In her autobiographical
account of her time as DCPS Chancellor, Michelle Johnson (nee Rhee), wrote
(p. 197)
“For
the first time in the history of DCPS, we brought in an outside expert to
examine and audit our system. Caveon Test Security – the leading expert in the
field at the time – assessed our tests, results, and security measures. Their
investigators interviewed teachers, principals, and administrators.
“Caveon
found no evidence of systematic cheating. None.”
Caveon, however, had not looked for “systematic”
cheating. All they did was interview a few people at several schools where the
statistical anomalies were more extraordinary than at others. As none of those
individuals would admit to knowingly cheating, Caveon branded all their excuses
as “plausible” explanations. That’s it; that is all that Caveon did. But,
Caveon’s statement that they found no evidence of “widespread” cheating—despite
not having looked for it—would be frequently invoked by DCPS leaders over the
next several years.[1]
Incidentally, prior to the revelation of its infamous
decades-long, systematic test cheating, the Atlanta Public Schools had
similarly retained Caveon Test Security and was, likewise, granted a clean bill of health.
Only later did the Georgia state attorney general swoop in and reveal the
truth.
In its defense, Caveon would note that several
cheating prevention measures it had recommended to DCPS were never adopted.[2]
None of the cheating prevention measures that I recommended were adopted,
either.
The single most effective means for reducing
in-classroom cheating would have been to rotate teachers on test days so that
no teacher administered a test to his or her own students. It would not have
been that difficult to randomly assign teachers to different classrooms on test
days.
The single most effective means for reducing school
administrator cheating would have been to rotate test administrators on
test days so that none managed the test materials for their own schools. The
visiting test administrators would have been responsible for keeping test
materials away from the school until test day, distributing sealed test
booklets to the rotated teachers on test day, and for collecting re-sealed test
booklets at the end of testing and immediately removing them from the school.
Instead of implementing these, or a number of other
feasible and effective test security measures, DCPS leaders increased the
number of test proctors, assigning each of a few dozen or so central office
staff a school to monitor. Those proctors could not reasonably manage the volume
of oversight required. A single DC test administration could encompass a
hundred schools and a thousand classrooms.
Investigations
So, what effort, if any, did DCPS make to counter test
cheating? They hired me, but then rejected all my suggestions for increasing
security. Also, they established a telephone tip line. Anyone who suspected
cheating could report it, even anonymously, and, allegedly, their tip would be
investigated.
Some forms of cheating are best investigated through
interviews. Probably the most frequent forms of cheating at DCPS—teachers
helping students during test administrations and school administrators looking
at test forms prior to administration—leave no statistical residue. Eyewitness
testimony is the only type of legal evidence available in such cases, but it is
not just inconsistent, it may be socially destructive.
I remember two investigations best: one occurred in a
relatively well-to-do neighborhood with well-educated parents active in school
affairs; the other in one of the city’s poorest neighborhoods. Superficially,
the cases were similar—an individual teacher was accused of helping his or her
own students with answers during test administrations. Making a case against
either elementary school teacher required sworn testimony from eyewitnesses,
that is, students—eight-to-ten-year olds.
My investigations, then, consisted of calling children
into the principal’s office one-by-one to be questioned about their teacher’s
behavior. We couldn’t hide the reason we were asking the questions. And, even
though each student agreed not to tell others what had occurred in their visit
to the principal’s office, we knew we had only one shot at an uncorrupted jury
pool.
Though the accusations against the two teachers were
similar and the cases against them equally strong, the outcomes could not have
been more different. In the high-poverty neighborhood, the students seemed
suspicious and said little; none would implicate the teacher, whom they all
seemed to like.
In the more prosperous neighborhood, students were
more outgoing, freely divulging what they had witnessed. The students had
discussed the alleged coaching with their parents who, in turn, urged them to
tell investigators what they knew. During his turn in the principal’s office,
the accused teacher denied any wrongdoing. I wrote up each interview, then
requested that each student read and sign.
Thankfully, that accused teacher made a deal and left
the school system a few weeks later. Had he not, we would have required the
presence in court of the eight-to-ten-year olds to testify under oath against
their former teacher, who taught multi-grade classes. Had that prosecution not
succeeded, the eyewitness students could have been routinely assigned to his
classroom the following school year.
My conclusion? Only in certain schools is the
successful prosecution of a cheating teacher through eyewitness testimony even
possible. But, even where possible, it consumes inordinate amounts of time and,
otherwise, comes at a high price, turning young innocents against authority
figures they naturally trusted.
Cheating blueprints
Arguably the most widespread and persistent testing malfeasance
in DCPS received little attention from the press. Moreover, it was directly propagated
by District leaders, who published test blueprints on the web. Put simply, test
“blueprints” are lists of the curricular standards (e.g., “student shall
correctly add two-digit numbers”) and the number of test items included in an
upcoming test related to each standard. DC had been advance publishing its
blueprints for years.
I argued that the way DC did it was unethical. The
head of the Division of Data & Accountability, Erin McGoldrick, however,
defended the practice, claimed it was common, and cited its existence in the
state of California as precedent. The next time she and I met for a conference
call with one of DCPS’s test providers, Discover Education, I asked their sales
agent how many of their hundreds of other customers advance-published blueprints.
His answer: none.
In the state of California, the location of
McGoldrick’s only prior professional experience, blueprints were, indeed,
published in advance of test administrations. But their tests were longer than
DC’s and all standards were tested. Publication of California’s blueprints
served more to remind the populace what the standards were in advance of each
test administration. Occasionally, a standard considered to be of unusual
importance might be assigned a greater number of test items than the average,
and the California blueprints signaled that emphasis.
In Washington, DC, the tests used in judging teacher
performance were shorter, covering only some of each year’s standards. So, DC’s
blueprints showed everyone well in advance of the test dates exactly which
standards would be tested and which would not. For each teacher, this posed an
ethical dilemma: should they “narrow the curriculum” by teaching only that
content they knew would be tested? Or, should they do the right thing and teach
all the standards, as they were legally and ethically bound to, even though it
meant spending less time on the to-be-tested content? It’s quite a conundrum
when one risks punishment for behaving ethically.
Monthly meetings convened to discuss issues with the
districtwide testing program, the DC Comprehensive Assessment System (DC-CAS)—administered
to comply with the federal No Child Left Behind (NCLB) Act. All public schools,
both DCPS and charters, administered those tests. At one of these regular
meetings, two representatives from the Office of the State Superintendent of
Education (OSSE) announced plans to repair the broken blueprint process.[3]
The State Office employees argued thoughtfully and
reasonably that it was professionally unethical to advance publish DC test
blueprints. Moreover, they had surveyed other US jurisdictions in an effort to
find others that followed DC's practice and found none. I was the
highest-ranking DCPS employee at the meeting and I expressed my support, congratulating
them for doing the right thing. I assumed that their decision was final.
I mentioned the decision to McGoldrick, who expressed
surprise and speculation that it might have not been made at the highest level
in the organizational hierarchy. Wasting no time, she met with other DCPS
senior managers and the proposed change was forthwith shelved. In that, and
other ways, the DCPS tail wagged the OSSE dog.
* * *
It may be too easy to finger ethical deficits for the
recalcitrant attitude toward test security of the Rhee-Henderson era ed
reformers. The columnist Peter Greene insists
that knowledge deficits among self-appointed education reformers also matter:
“… the reformistan bubble … has been built from Day
One without any actual educators inside it. Instead, the bubble is populated by
rich people, people who want rich people's money, people who think they have
great ideas about education, and even people who sincerely want to make
education better. The bubble does not include people who can turn to an Arne
Duncan or a Betsy DeVos or a Bill Gates and say, ‘Based on my years of
experience in a classroom, I'd have to say that idea is ridiculous bullshit.’”
“There are a tiny handful of people within the bubble
who will occasionally act as bullshit detectors, but they are not enough. The
ed reform movement has gathered power and money and set up a parallel education
system even as it has managed to capture leadership roles within public
education, but the ed reform movement still lacks what it has always
lacked--actual teachers and experienced educators who know what the hell
they're talking about.”
In my twenties, I worked for several years in the
research department of a state education agency. My primary political lesson
from that experience, consistently reinforced subsequently, is that most
education bureaucrats tell the public that the system they manage works just
fine, no matter what the reality. They can get away with this because they
control most of the evidence and can suppress it or spin it to their advantage.
In this proclivity, the DCPS central office leaders of
the Rhee-Henderson era proved themselves to be no different than the
traditional public-school educators they so casually demonized.
US school systems are structured to be opaque and, it
seems, both educators and testing contractors like it that way. For their part,
and contrary to their rhetoric, Rhee, Henderson, and McGoldrick passed on many
opportunities to make their system more transparent and accountable.
Education policy will not improve until control
of the evidence is ceded to genuinely
independent third parties, hired neither by the public education
establishment nor by the education reform club.
The
author gratefully acknowledges the fact-checking assistance of Erich Martel and
Mary Levy.
Access
this testimonial in .pdf
format
Citation: Phelps, R. P. (2020, September). Looking Back
on DC Education Reform 10 Years After, Part 2: Test Cheats. Nonpartisan
Education Review / Testimonials. https://nonpartisaneducation.org/Review/Testimonials/v16n3.htm
[1] A perusal of Caveon’s website clarifies that their mission is to help their clients–state and local education departments–not get caught. Sometimes this means not cheating in the first place; other times it might mean something else. One might argue that, ironically, Caveon could be helping its clients to cheat in more sophisticated ways and cover their tracks better.
[2] Among them: test booklets should be sealed until the students open them and resealed by the students immediately after; and students should be assigned seats on test day and a seating chart submitted to test coordinators (necessary for verifying cluster patterns in student responses that would suggest answer copying).
[3] Yes, for those new to the area, the District of Columbia has an Office of the "State" Superintendent of Education (OSSE). Its domain of relationships includes not just the regular public schools (i.e., DCPS), but also other public schools (i.e., charters) and private schools. Practically, it primarily serves as a conduit for funneling money from a menagerie of federal education-related grant and aid programs.