Lake Woebegone, Twenty Years Later

Lake Woebegone, Twenty Years Later

Nonpartisan Education Review / Essays, Volume 2 Number 1

Access this article in pdf format.





“Lake Woebegone,” Twenty Years Later Footnote


John Jacob Cannell, MD

The Vitamin D Council, Inc.


February 2006


Abstract


Twenty years ago, John Cannell developed data on test scores that became known as the “Lake Woebegone effect.” This commentary describes that experience.



Almost twenty years ago, I wrote - and then privately published - the two “Lake Woebegone” reports, named after Garrison Keillor’s mythical Minnesota town where “all the women are strong, all the men are good-looking, and all the children are above average.” The first “Lake Woebegone” report documented that all fifty states were testing above the national average in elementary achievement and concluded the testing infrastructure in America’s public schools was corrupt.1 The second report delineated the systematic and pervasive ways that American educators cheat on standardized achievement tests.2 Both reports received widespread national publicity, were extensively discussed in academic journals, and helped spur the testing reform movement.


In 1991, five years after I began, I abruptly left the testing reform movement. This paper discusses how I learned about “Lake Woebegone” testing, the reason why I left the testing reform movement, and my observations on where testing is today. Is No Child Left Behind (NCLB) testing much different from what was occurring during the “Lake Woebegone” years?


I continue to assume that tests are simply sampling techniques, in which a relatively small number of test questions sample a larger body of knowledge. Any corruption of the sampling technique, in which the test questions become more familiar to students than the larger body of knowledge being sampled, invalidates the test. As always, my observations are those of a consumer, a parent, a physician, and an activist - not those of a psychometrician.


My education about the corruption of American public school achievement testing was a gradual process. It started in my medical office in a tiny town in the coal fields of Southern West Virginia, led to school rooms in the county and then the state, to the offices of testing directors and school administrators around the country, to the boardrooms of commercial test publishers, to the office of the U.S. Secretary of Education, to schools of education at major American universities, to various governors’ offices, and finally, to two American presidents.


One day in 1985, West Virginia newspapers announced all fifty-five West Virginia counties had tested above the national average. Amid the mutual congratulations, I asked myself two things. How could all the counties in West Virginia, perhaps the poorest and most illiterate state in the union, be above the national average? Moreover, if West Virginia was above average, what state was below?


In my Flat Top, West Virginia, clinic, illiterate adolescent patients with unbelievably high standardized achievement test scores told me their teachers drilled them on test questions in advance of the test. How did the teachers know what questions would be on a standardized test?


Then I learned that West Virginia schools, like most other states, used what seemed to me as a physician to be very unusual standardized tests. Unlike the standardized tests that I knew - such as college entrance, medical school admission, or medical licensure examinations - public school achievement exams used the same exact questions year after year and then compared those scores to an old, and dubious, norm group - not to a national average. Furthermore, educators - the group really being tested - had physical control of the tests and the teachers administered them without any meaningful test security.


In fact, CTB/McGraw-Hill explicitly instructed the teachers to look at all the questions before giving the tests to the students, saying; “arrangements should be made so the proctors and examiners actually can take the test.”3 Numerous teachers - usually my patients - told me they simply memorized or copied the test questions and taught their students the answers the following year to ensure high scores. If they failed to do so, parents would be angry, colleagues critical, newspapers disparaging, and administrators livid. Teacher’s annual evaluations would inevitably suffer.


I recently contacted the West Virginia Department of Education to see if anything had changed in West Virginia during the last 15 years. I found that from 1996 to 2002, West Virginia used the exact same test questions for eight years in a row [Stanford-9 Achievement Test (SAT-9, form S)]. Scores soared. I dare the reader to find even one of the fifty-five West Virginia counties in which the majority of elementary students tested below the national average for the four years between 1999 and 2002.4 In 2003, West Virginia began excluding fewer lower functioning students from test reporting and several dirt-poor counties in the southern coalfields fell slightly below the national average. However, in 2003,West Virginia statewide SAT-9 scores were still above the national average at all grade levels tested.


In 2004,West Virginia adopted a criterion-referenced test, the WestTest, in consultation with CTB/McGraw-Hill. I called the Office of Student Assessment Services of the West Virginia Department of Education and learned that eighty percent of the WestTest questions are the same year-to-year.5 I could find nothing in their “Testing Code of Ethics” which even suggests that the teachers should not look at WestTest question while students are taking the exam.6 Using these testing procedures, 78 percent of West Virginia third graders tested“at or above mastery” in reading/language arts in 2005.7


However, in 2005, the National Assessment of Educational Progress (NAEP)found something quite different. (NAEP is a national test, which samples participating school districts in a secure manner.) NAEP reported only 26percent - not 78 percent - of West Virginia’s fourth graders were proficient in reading.8 It seems that West Virginia students are proficient in West Virginia reading but not in American reading!


Like 20 years ago, the most parsimonious explanation for this discrepancy is that teachers in West Virginia are doing today what they were doing twenty years ago, memorizing the West Virginia test questions and teaching the answers the following year - something they cannot do with the NAEP tesst. Although I didn’t know it at the time, and was denied anyway of proving it now, it seems likely that West Virginia educators rely on curriculum preparation materials that are virtually identical to the WestTest, also something they cannot do with NAEP.


Obviously, this is not occurring in a vacuum. School administrators design testing procedures that assure that teachers read the test, but cover their own culpability with loosely worded and largely meaningless “Testing Codes of Ethics.”Although I could not obtain copies of the WestTest to compare those items with test or curriculum material, it is my experience - as the reader will see- that many schools use test preparation materials that are repeatedly laced with the test questions.


School administrators are the principal beneficiaries of rapidly rising scores and they want no part of accurate tests. Administrators need rapidly rising scores to show they are doing a good job. In the past, when states like West Virginia finally change to an equivalent, but different, test –and teachers see an unfamiliar set of questions - scores plummeted, parents complain, newspapers criticize, and administrators play musical chairs and exchange jobs. However, next year’s scores start their inexorable and flattering ascent.


In the 1980’s, school principals in West Virginia often insisted that their teachers take the tests beforehand (as the official test manual sometimes suggested) so the teachers had ample time to memorize the questions. When this didn’t work, some administrators in California- who had physical control of the tests - simply erased the student’s wrong answers and marked the correct ones.9 At that point, I had no idea of the role that crib-sheet-like test preparation materials played in rapidly rising scores.


By 1985, I wanted to know who was supplying the schools with tests that could so easily be misused to deceive parents, children, and the community. Posing asa school administrator, I called a major test publisher. The woman I spoke with was more than happy to supply tests with any “national norms” that I requested- all certified by respected testing consultants from major universities. The publisher would sell inner city norms, low-socioeconomic norms, adjusted norms, etc., with their tests. She explained that choosing the right “national norm” was very important.


If I chose a low performing norm group, I could look forward to high initial scores but year-over-year gains would quickly become unbelievably high. If the initial norms were “tougher,” I would look bad the first year but could look forward to very flattering year-over-year gains. The woman finally caught on tome when I asked how she knew the scores would go up every year. Of course, she knew that unchanging test questions guarantee rising scores.


By 1986, I had found out several important things. The test publishers would supply any “national norms” school districts wanted. They also knew that using the same test questions year after year assured that enough teachers would read the test and cause flattering year-over-year gains. Finally, the publishers were making good money selling these “standardized tests,”and wanted no part in reforming them. What I didn’t know is who was consulting for these publishers, giving them academic cover?


In 1987, I formed an education reform group, Friends for Education. We conducted a series of campaigns to improve schools in West Virginia. For example, we held “The Cleanest School in West Virginia Contest.” When that was ignored, we held a “Dirtiest School in West Virginia Contest,” promising a bucket, mop and broom to the winner.10 We held public rallies focusing on improving the worst schools in the state.11 We also filed complaints with the Equal Opportunity Employment Commission, claiming school officials were illegally denying women administrative positions in West Virginia schools.12


However, I kept wondering about the tests. The American educational system is built around testing. Could the entire American testing infrastructure be corrupt? If West Virginia was testing above the national average, then perhaps all the states were reporting the same thing and no one knew it. I called the office of then U.S. Secretary of Education William Bennett only to be told they did not collect, nor did they know, scores from individual states.


Between patients, I had the nurse, x-ray technician, and lab technician from my medical clinic call the education departments of all fifty states and collect their test scores. For reasons I was beginning to understand, the states and districts proved to be quite defensive in releasing scores. Often my clinic staff had to make up a compelling reason to get the scores, such as they were thinking of moving to the state in question.


Finally, there it was, I had the data: all fifty states were reporting they were above the national average in elementary achievement. I realized for the first time that newspapers throughout the country were repeatedly running flattering stories on state or local school achievement on one page - and dire warnings that the United States was “A Nation at Risk” on another.


CTB-McGraw Hill wrote me and threatened to sue me should I publish my findings.13 I promptly dipped into my personal savings and published my first report: Nationally Normed Elementary Achievement Testing in America's Public Schools: How All 50 States Are above the National Average. Although we had no money for legal action, Friends for Education immediately sent consumer fraud complaints to the Attorneys General in all fifty states, claiming commercial test publishers were engaged in deceptive testing practices.14 My first report showed up in headlines around the country, including the front page of the Washington Post and an article in the New York Times with the headline: “Standardized Test Scores: Voodoo Statistics.”15,16


The 1988 summer issue Educational Measurement: Issues and Practice, was devoted to my first report, with invited responses from the four major commercial test publishers, several academics, and the U.S. Department of Education.17 I responded in the winter issue.18 Two years later, the same journal devoted another issue to a U.S. government funded study on my “Lake Woebegone”report. It concluded, “The results of the present study provide support for Cannell’s general finding that for the elementary grades almost all states and the majority of districts are reporting norm-referenced achievement test results that are above the national median.”19


Later in 1988, my education about standardized testing in America continued at a special meeting of test publishers and academicians at the U.S. Department of Education. Secretary William Bennett called the meeting, asking me to explain my findings that all fifty states were testing above the national average. It was during that meeting Assistant U. S. Secretary of Education Chester Finn first labeled my findings, the “Lake Woebegone Report.”


At the meeting, I learned that a handful of academicians at major American universities consulted with test publishers to develop both the tests and the various norms. These academics denied that “Lake Woebegone” tests were the problem, insisting that educators were misusing their tests due to a “high stakes”testing environment. The representatives from the college entrance exam publishers, the SAT and ACT, expressed amazement at this explanation. After all, their tests suffered from no “Lake Woebegone” psychometrics.


I did not understand why some of the academics insisted on explaining it was due to “high stakes” testing when the obvious problem was a corrupt testing infrastructure. If I cheated on my federal tax return, explaining it saved my family “high stakes” money, few federal courts would exonerate me under such a defense - indeed few defense attorneys would proffer such a meaningless defense.


I left the meeting having learned three things. One, both William Bennett and Chester Finn claimed they knew nothing about the cheating. Two, officials from the college entrance exams knew about the cheating but were powerless to stop it. Three, some of academicians knew all about the cheating and, for reasons I didn’t understand, wanted “Lake Woebegone” testing to continue.


By that time, I rightly or wrongly assumed most politicians knew what was going on. While speaking at an April 1990 meeting of the Education Writers Association in Chicago, I angrily confronted another speaker, Governor Bill Clinton. I told Clinton, early in his initial run for the White House, of the unbelievably high, rapidly improving, and very politically flattering, test scores in illiterate Arkansas. Clinton vehemently denied any knowledge of cheating.


The following week, Clinton encountered a front-page story in the states’ largest newspaper about my charges of widespread cheating in Arkansas.20 Clinton then called me and spent thirty minutes asking me questions about things he could do to stop the cheating. I told him the keys to preserving the validity of the test is changing questions every year, having a large bank of questions, maintaining a broad curriculum, testing infrequently, and not focusing on test preparation. Another Arkansas newspaper quoted Clinton’s response:“When he(Cannell) told me that, I said ‘Gosh’ we’ll look into that. It may cost a few more thousand dollars but it’s worth it if it preserves the integrity of the test.”21 A few weeks later, Arkansas announced plans for improvements in test security.22 In1996, then President Clinton went on to recommend a national achievement test with strict security - a proposal refused by the Republican Congress.


Unlike Clinton, Bush was not spared national media attention about cheating in Texas. One month before the election of 2000, the Rand Corporation claimed that the “Texas Miracle,” the dramatic gains in Texas school achievement that propelled George Bush to the White House, were suspect. Rand compared gains on the Texas test with Texas gains on National Assessment of Educational Progress (NAEP) and concluded that most of the gains on the Texas test were bogus.23 In a separate review of Texas testing, Haney concurred, saying the gains “are more illusory than real,” and “the Texas Miracle is more hat than cattle.”24


More recently, the Dallas Morning News has uncovered evidence of cheating in more than 200 Texas schools and this may be the tip of the iceberg.25 Although the basic statistical technique used to detect cheating is easy to perform, the Texas Education Agency chooses not to use it. Furthermore, the Texas Education Agency performs erasure analysis (to detect if school officials are erasing the kid’s wrong answers and filling in the correct ones) but does nothing with the information unless they get a complaint! It is interesting to note that Bush, unlike Clinton, has failed to propose reforms that would address America’s corrupt public school testing infrastructure.


Successful governors, like Clinton and Bush, may have learned long ago of the political value of “Lake Woebegone” testing, or, like most Americans, they just assume a standardized test is a standardized test. However, public educators, test publishers, and academic testing experts cannot claim ignorance –they are too intimately involved with the corruption.


I was hopeful the first “Lake Woebegone” report would reform testing. After all, it made headlines around the country. In 1989, when nothing changed, I again surveyed all 50 states and published a second report with the help of a grant from the Kettering Family Foundation. The “Lake Wobegon” Report, How Public Educators Cheat on Standardized Achievement Tests detailed the extent of cheating and how to detect it.


My second report received even more publicity then the first, with front-page coverage by the Wall Street Journal.26 The Rocky Mountain News said I had uncovered “the Great Testing Lie.”27 Surely, policy makers would be outraged a tour corrupt testing infrastructure and realize that American schools would never improve until we had honest tests. I then wrote and distributed Testing Ethics Model Legislation; forlornly hoping state legislators would enact simple laws reform testing.28


No response. However, in 1990 my hopes soared when Sixty Minutes called. They were doing a story about teachers and school administrators cheating on tests, highlighting my second report. I was sure that would do it. Scandals exposed by Sixty Minutes often led to reform. Morley Safer came to my house and brought copies of some “test preparation” materials that CBS News had obtained from various states. He asked me to look them over before filming, Teacher is a Cheater.


It hit me like a brick. The “test preparation” materials contained all the answers to the test questions; any one who mastered this material would know exactly what was on the test. Some of the same academicians who claimed“misunderstandings” and “high stakes” testing were the problem, and who made money collaborating with test publishers developing “Lake Woebegone”tests, had side businesses; they provided school districts with crib sheets.


I waited for the fallout from the Sixty Minutes report. On March 25, 1990, Sixty Minutes ran Teacher is a Cheater and reported the tests were fraudulent and that cheating by educators was rampant in American schools.29 The cheating took many forms, from outright falsification of children’s answer sheets and teachers memorizing test questions, to the most insidious - closely aligning a narrow curriculum with a dumbed-down test while using crib-sheet-like test preparation materials. Sixty Minutes, like the Wall Street Journal, reported the most common corruption was assuring that scores steadily climbed due to crib-sheet-like test preparation-materials.30


However, nothing happened. The Sixty Minutes report provoked no outrage, no commissions, no hearings, nothing. I could do no more. I resigned from Friends for Education and the organization fell apart. I quickly became involved with two entirely different causes: first, false recovered memories of sexual abuse, and subsequently, widespread vitamin D deficiency.31


My education reform days are painful to remember, mainly because I fear that little has changed and my work was for naught. Every year, from my California home, I read hundreds of stories online, published in local California newspapers, all about how test scores are improving in their local schools. The same articles praise California politicians and the California State Department of Education for statewide improvements.


As part of their state-run STAR testing program, the California Department of Education administered the Stanford-9 from 1998 to 2002, using the same booklets - and the same questions - for five years in a row.32 California fourth grade national percentile rank reading scores on the Stanford-9 section of the California STAR testing program are instructive:

 

Year    Reading          Language

1998    40                    4433

1999    42                    4634

2000    45                    5035

2001    47                    5336

2002    50                    5537


Pretty impressive. In five years, California went from among the lowest reading and language scores in the country to at or above the national average. However, in 2003, California changed to the California Achievement Test-6(CAT-6), and scores plummeted when the teachers encountered unfamiliar test questions. Fourth grade California reading scores fell to 39 and language scores fell to 42, lower than they were in 1998.38


In 2002, California began to emphasize the other component of their STAR testing program; the state developed, California Standards Test (CST). I understand, but the California Department of Education will neither confirm nor deny, that about 50% of the questions on the CST are the same year-to-year. Fourth grade mean scaled English/Language Arts scores on the CST give a similar, but less dramatic, impression of improving learning in California.39

 

2002    333

2003    339

2004    339

2005    346


Compare the Stanford-9 and CST to the fourth grade NAEP reading scores for California. Examine the percentage of California children who performed at or above the rudimentary “basic”reading level on NAEP:40

 

1998    48

2002    50

2003    50

2005    50


NAEP reports no significant difference in fourth grade reading average scale scores during the same time although increasing participation rates may have masked any gain. Footnote Even more telling is California’s NAEP ranking: NAEP reports that of the 52 other states and jurisdictions that participated in the 2005assessment, California performed better than only one other jurisdiction.


The Thomas B. Fordham Foundation recently came to the same conclusion. They listed California as one of several states with rather dramatic gains on state controlled tests, but no progress on NAEP testing.41 The same study found Tennessee was the most brazen. In 2005, eighty-eight percent of Tennessee eighth-graders tested proficient on their “Lake Woebegone” state reading test, while only 26% were proficient on NAEP. That is, virtually all Tennessee students are proficient in Tennessee reading, but very few in American reading.


Fifteen years after “Lake Woebegone” testing was uncovered, California, the largest state in the union, continues to conduct corrupt testing. Although I was not able to learn if academics are still supplying “test preparation materials” to California schools, it matters little. California teachers tell me they really don’t have to memorize the test questions anymore, the curriculum materials supplied by the state are laced, repeatedly, with the test questions.


Recent federal legislation has undoubtedly made the cheating worse because No Child Left Behind (NCLB) increased the consequences of the tests but Bush and the Congress failed to reform America’s testing infrastructure. Furthermore, my experience with recent graduates from American schools of education is that they are misinformed about the essence of testing (accurate sampling) and pathologically opposed to virtually all forms of testing. Such mind sets are fertile fields for widespread corruption.


However, it is unfair to blame NCLB for corrupt testing; it existed long before NCLB. Two recent press articles detailed widespread cheating on NCLB testing, blaming it on the “high stakes”testing environment created by NCLB.42,43 However, the reporters failed to note that the media reported on widespread cheating in state run testing programs well before NCLB.44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60 What NCLB did was greatly increase the stakes of testing, while relying on a corrupt testing infrastructure to measure results.


When NAEP scores are lower than NCLB state scores, state superintendents of schools never mention their crib-sheet-like test preparation material, their lack of outside proctors, or the fact that teachers know what is on the test because many of their questions are the same every year. Instead, they simply explain that they have not aligned their curriculum with the NAEP test.


California Superintendent of Schools, Jack O’Connell, recently explained to the Los Angeles Times, “Results on our statewide tests, which are aligned to our rigorous standards, indicate that a focus on high expectations is leading to steady gains in student achievement.”61 He failed to mention that 50% of the questions on the statewide test are the same year-to year, that test preparation materials are laced with test questions, and California fails to use independent test proctors. Although less dramatic then West Virginia and Tennessee, fourth grade students in California show steady improvement in California reading but not in American reading.


As Samuel Johnson said in Boswell’s Life, the solution “is as well-known, and has long been as well known, as ever it will be.” The vast majority of test questions must be different every year, questions must come from a large bank of questions, test preparation materials should be minimal, and outside proctors, not school officials, should administer the tests. The Director of the Oklahoma State Bureau of Investigation, Robert Hicks, said it well.


In responding to Friends for Educations’ charges of widespread cheating in Oklahoma, Hicks wrote to then Governor Henry Bellman, who wanted to know if the state should prosecute the cheating teachers. Director Hicks advised against it. He explained, “The tests lend themselves to being compromised by using the same questions repeatedly for several years and also allowing the tests to be administered by school officials.”62


I agree. It is unfair to blame only the educators for a corrupt testing infrastructure, an infrastructure now required by the federal government. If I were a state superintendent of schools in one of the 25 lower-performing states, I would demand a test that showed we were meeting NCLB standards and the easiest way to do that is to use the same questions year after year or to produce test preparation materials or curriculum materials that are laced with test questions. Likewise, if I was a county or district superintendent of schools, I ‘d make sure my principals had easy access to the tests and concentrated on test question-laced-curriculum materials. If I were a principal, I’d do the same. If I were a teacher, I’d either carefully study the exam questions and, the next year, teach the children enough of the answers to be sure I could meet NCLB standards or narrow my curriculum to the corrupt test preparation/curriculum materials. I cannot blame educators for doing what I would do, where I caught in a corrupt system.


In their lives, my two young daughters will compete with children from other countries who have been ruthlessly educated. My daughters need a broad-based education driven by an incorruptible test. As someone once said, “If the educators want to know what will be on the test, tell them the English section will have lots of letters and the math section lots of numbers.”


Instead of preparing children with a broad and challenging curriculum, American schools teach a narrow curriculum driven by corrupted tests and we are now doing so in the name of leaving no child behind. Like the “Lake Woebegone” testing that preceded it, NCLB testing mollifies parents, compliments educators, promotes political careers, enriches publishers, and ensures profitable consultation fees to academics at many of America’s major schools of education. Until we repair America’s corrupt testing infrastructure, American schools will continue to flounder in a sea of mediocrity.



Access this article in pdf format.


Citation: Cannell, J.J. (2006). “Lake Woebegone,” Twenty Years Later.” Nonpartisan Education Review / Essays, 2(1). Retrieved [date] from http://nonpartisaneducation.org/Review/Essays/v2n1.pdf



References

1. Cannell, JJ. Nationally Normed Elementary Achievement Testing in America's Public Schools: How All 50 States Are above the National Average. Daniels, WV, 1987 (ERIC # EJ374428, Educational Measurement: Issues and Practice. v7 n2 p5-9 Sum 1988)

2. Cannell, JJ. How Public Educators Cheat on Standardized Achievement Tests: The "Lake Wobegon" Report. Friends for Education, Albuquerque, NM, 1988.(ERIC # ED314454.)

3. CTB/McGraw-Hill, California Achievement Test, Test Coordinators Handbook, Monterey, CA, 1986 

4. West Virginia Report Cards. http://wvde.state.wv.us/data/report_cards/.Accessed 11/30/05

5. WESTEST Maps to CSOs. http://westest.k12.wv.us/blueprintmap.htm.Accessed 2/10/05

6. Examiner’s Manual. http://westest.k12.wv.us/pdf/45678ExMan_3-8_05WV.pdf. Accessed 2/9/06

7. WestTest. http://westest.k12.wv.us/. Accessed 2/08/06

8. Source: National Center for Education Statistics. NAEP results for West Virginia. http://nces.ed.gov/nationsreportcard/nde/viewresults.asp. Accessed12/04/05

9. Pressure for High Scores Blamed in CAP Test Cheating. McGraw, C and Wood T. Los Angeles Times, September 18, 1988.

10. Editorial. Friends’ cleanup proposal sound. The Register/Herald. Beckley, WV, April 13, 1986

11. Inferior Schools, Lack of Standard Criticized. Q and A, Sunday Gazette-Mail, Charleston, WV, March 23, 1986

12. Sexual Bias Information Sought from State Education Department. Austin, C.,The Charleston Gazette, Charleston, WV. August 12, 1987

13. Deffley David, General Manager, CTB/McGraw-Hill. Letter to Dr. John Jacob Cannell, President, Friends for Education, July 24, 1987. Copy available on request.

14. Educational Group Asks States to Stem 'Deceptive Practices' by 4 Testing Firms. Putka G.,Wall Street Journal, March 16, 1988

15. Above Average Has Become Testing Norm; States' Scores Called Misleadingly High. Lawrence Feinberg, The Washington Post, Feb 8, 1988

16. Standardized Test Scores: Voodoo Statistics. Fiske E., The New York Times, February 17, 1988.

17. Educational Measurement: Issues and Practice, Vol. 7(2), Summer, 1988

18. The Lake Woebegone Effect Revisited, Cannell, J., Educational Measurement: Issues and Practice, Vol. 7(4), pgs 12-15, Winter, 1988

19. Educational Measurement: Issues and Practice, Vol. 9(3), Fall, 1990

20. State Cheating for Test Scores, Author Charges. Howell, C., Arkansas Democrat, April 10, 1990

21. Teachers Could Be Cheating: Change Questions to Keep Integrity of Test Analyst Says. Morris, S., Arkansas Democrat Gazette, April 10, 1990

22. Dishonesty in Teachers Exists, but is Rare, State Educators Say. David, A.,Arkansas Democrat Gazette, April 21, 1990

23. What Do Test Scores in Texas Tell Us? Stephen P. Klein, Laura S. Hamilton, Daniel F. McCaffrey, Brian M. Stecher. Rand Corporation, October 24, 2000http://epaa.asu.edu/epaa/v8n49/ accessed 12/24/05)

24. The myth of the Texas miracle in education. Haney, W. Education Policy Analysis Archives, 8(41). http://epaa.asu.edu/epaa/v8n41/ accessed 12/24/05

25. Tools may stem cheating on tests. Benton J., Dallas Morning News, March 31,2005.http://www.dallasnews.com/s/dws/news/longterm/stories/122004dnmetcheating.4c687.html Accessed 2/11/06

26. Classroom Scandal: Cheaters in Schools May not be Students but their Teachers. Putka G., Wall Street Journal, November 2, 1988

27. The Great Testing Lie. Editorial, The Rocky Mountain News, March 16, 1988

28. Testing Ethics Model Legislation. John Jacob Cannell. Eric # ED322155 (1990)

29. Teacher is a Cheater. Sixty Minutes, CBS Television, March 25, 1990

30. School Tests Closely Match Material In Commercial Practice Kits, Booklets. Gary Putka. The Wall Street Journal; Nov 2, 1989.

31. The Vitamin D Council, Inc. http://www.vitamindcouncil.com/biography.pdf. Accessed 12/1/05

32. Standardized Testing and Reporting (STAR) Results. http://star.cde.ca.gov/.Accessed 10/21/05

33. STAR State Summary Report for All Students, Spring, 1998.http://star.cde.ca.gov/star98/reports/00-00000-0000000.html. Accessed 2/9/06

34. STAR Score Summaries Report for All Students, Spring 1999.http://star.cde.ca.gov/star99/reports/English.html. Accessed 2/9/06

35. STAR Score Summaries Report for All Students, Spring 2000.http://star.cde.ca.gov/star2000F/reports/01.html. Accessed 2/9/06

36. STAR Score Summaries Report for All Students, Spring 2001.http://star.cde.ca.gov/star2001/report.asp?CDSCode=00000000000000000000&CL=1. Accessed 2/9/06

37. STAR Score Summaries Report for All Students, Spring 2002.http://star.cde.ca.gov/star2002/report.asp?CDSCode=00000000000000000000&CL=1. Accessed 2/9/06

38. California Standardized Testing and Reporting (STAR).http://star.cde.ca.gov/star2003/viewreport.asp. Accessed 2/9/06

39. About STAR 2002. http://star.cde.ca.gov/star2002/help/AboutSTAR.html.Accessed 2/9/06

40. National Center for Education Statistics. NAEP results for California. http://nces.ed.gov/nationsreportcard/nde/viewresults.asp. Accessed10/21/05

41. Gains on State Reading Tests Evaporate on 2005 NAEP. Thomas B. Fordham Foundation. October 19, 2005. http://www.edexcellence.net/foundation/about/press_release.cfm?id=19accessed 11/30/05

42. A Spate Of Cheating -- By Teachers. BusinessWeek Online, July 5, 2004http://www.businessweek.com/magazine/content/04_27/b3890129_mz021.htm accessed 11/30/05

43. When Tests' Cheaters are the Teachers. The Christian Science Monitor, January 11, 2005. http://www.csmonitor.com/2005/0111/p01s03-ussc.html.Accessed 11/30/05

44. Five Fairfax Educators Disciplined Over Coaching on Tests. Cohnm D., The Washington Post, August 7, 1987

45. Cheating by any other name (editorial) and County’s Trick for Test Called“Irregular.” Murphey H., St. Petersburg Times, St. Petersburg , FL, September 17,1989

46. “Teaching to the Test”: Dim Ethical Area for Educators. Woo E., Los Angeles Times, September 19, 1988

47. School Officials Say Hard Work Raised Scores in Clinton Co.. Stroud, J., The Lexington Herald-Leader, Lexington, KY, September 24, 1989

48. Irvington: Pupils Got Tests back. Schoenberg, N., Gannett Westchester Newspaper, White Plains, NY, September 12, 1989

49. What’s the Score: Test Tampering or True Teaching. McCowan, K., The Arizona Republic, January 21, 1990

50. Teacher in Court of Administration-bashing. Glazer, r., Middletown Times Herald Record, Middletown, N.Y., March 10, 1990

51. Schools Cheating to Raise Score, Two Teachers Say. Sponhour M., The State, Columbia, S.C., April 3, 1990

52. Encouraging Signs on Cheating Issue, Editorial, The News and Courier, Charleston, SC, April 13, 1990

53. Parents of First-grade Girl Claim School Fudged on Test, White, B., The Atlanta Journal/The Atlanta Constitution, April 22, 1990

54. Test Code Seeks to Curb Cheating. Lovejoy, E., The Trenton Times, Trenton, NJ, April 25, 1990

55. Parents Object to Pre-Test Preparation. Robison, J., The Newman Times-Herald, Newman, GA, May 10, 1990

56. Teachers who Cheat. Clarke, R., The News and Observer, Raleigh, NC, June 3,1990

57. 7 Schools Cited in Test Probe. Ruth, D., The Times-Picayune, New Orleans, LA, October, 20, 1990

58. Grade Inflation; New Math: No One is Below Average. Hartocollis, A., New York Times, Week in Review Desk, June 20, 1999

59. Teachers are Said to Aid Cheating. Abby Goodnough. The New York Times, December 8, 1999

60. Cheating Teachers, Mejia, F., CNN, May 5, 2000.http://archives.cnn.com/2000/fyi/sb/05/04/cheating.teachers/ Accessed12/3/05

61. California Students Are Still Struggling. Vaughn, E. Los Angles Times, October 20, 2005.http://www.latimes.com/news/nationworld/nation/la-naeduc20oct20,0,2092857.story?coll=la-home-nation. Accessed 10/21/05

62. Letter to The Honorable Henry Bellman, Governor of Oklahoma. Robert J. Hicks, Director, Oklahoma Bureau of Investigation, pg. 13, November 21, 1989.