| Author | Co-author(s) | Dismissive Quote | type | Title | Source | Funders | Link1 | |
| 1 | Mike Petrilli | "Students who learn dramatically more at school, as measured by valid and reliable assessments, will go on to graduate from high school, enroll in and complete postsecondary education, and earn more as adults than similar peers who learn less. You would think that there would be lots of studies looking at studentsÕ learning gains in elementary or middle school and how that impacts their high school graduation or college enrollment rates. Yet to my knowledge none exist. (Academics: LetÕs change that please!)" | Dismissive | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 2 | Mike Petrilli | "What we do have is the famous Raj Chetty et al. study examining teacher value-added, which found that students who learn more in elementary school earn more as adults. ItÕs just one study, but itÕs a remarkable finding, one that might be hard to replicate unless more scholars can gain access to the tax data Chetty and his colleagues have." | 1stness | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 3 | Mike Petrilli | "Elementary and middle schools that dramatically boost the achievement of their students should also boost their long-term outcomes, including high school graduation, postsecondary enrollment, performance, and completion, as well as later earnings. Here we have a bit more to go on, at least if we look at studies that examine both individual schools and programs that are focused at least in part on elementary or middle schools. Remember that weÕre interested in schools or programs that make a significant impact on achievement, for good or ill. According to Hitt, McShane, and WolfÕs review, there are four of those." | Dismissive | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 4 | Mike Petrilli | "High schools that dramatically boost the achievement of their students should also boost their long-term outcomes, including postsecondary enrollment, performance, and completion, and earnings. Here the research base is a tad larger. We can start with a 2016 study of TexasÕs accountability system by all-stars David J. Deming, Sarah Cohodes, Jennifer Jennings, and Christopher Jencks, ... | Dismissive | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 5 | Mike Petrilli | "The research base is very thin—too thin for a serious meta-analysis. With only nine relevant studies, this is clearly a field still in its infancy." | Dismissive | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 6 | Mike Petrilli | "No doubt this debate will continue; we plainly need a lot more empirical evidence to inform it." | Dismissive | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 7 | David Steiner | "Research comparing one curriculum to another is very rare and, therefore, not usually actionable." | Dismissive | Curriculum Research: What We Know and Where We Need to Go | March 2017, p.1 | StandardsWork | https://standardswork.org/wp-content/uploads/2017/03/sw-curriculum-research-report-fnl.pdf | |
| 8 | David Steiner | "The paucity of evidence upon which sound instructional, purchasing, and policy decisions can be made is a matter of deep concern and urgent need." | Dismissive | Curriculum Research: What We Know and Where We Need to Go | March 2017, p.1 | StandardsWork | https://standardswork.org/wp-content/uploads/2017/03/sw-curriculum-research-report-fnl.pdf | |
| 9 | David Steiner | "To date, research on the curriculum effect has told us little about what makes a particular curriculum or genre of curriculum especially effective or not. We encounter only occasional, anecdotal observations on this in the research. | Denigrating | Curriculum Research: What We Know and Where We Need to Go | March 2017, p.7 | StandardsWork | https://standardswork.org/wp-content/uploads/2017/03/sw-curriculum-research-report-fnl.pdf | |
| 10 | David Steiner | "The rapid growth of online, personalized learning platforms will likely change classroom instruction further. As of yet, there exists no high-quality research on the impact of such platformsÉ" | Denigrating | Curriculum Research: What We Know and Where We Need to Go | March 2017, p.6 | StandardsWork | https://standardswork.org/wp-content/uploads/2017/03/sw-curriculum-research-report-fnl.pdf | |
| 11 | Blake Heller | Matthew Davis | "And although there is a robust positive correlation between test performance and college enrollment, there is little existing evidence as to whether schools that increase test scores the most also help their students succeed at the next level." | Dismissive | Raising more than test scores | Education Next, WINTER 2017 / VOL. 17, NO. 1 | http://educationnext.org/raising-more-than-test-scores-noble-charter-no-excuses/ | |
| 12 | Adam Edgerton | Morgan Polikoff, Laura Desimone | Five-plus years into the experiment with new Òcollege- and career-ready standardsÓ, we know little about teachersÕ implementation and the ways policy can support that implementation. This paperÉ | Dismissive | How is policy affecting classroom instruction?, p.1 | Brookings Institution, Evidence Speaks Reports, Vol 2, #14, May 11, 2017 | Laura and John Arnold Foundation, USA Funds | https://www.brookings.edu/research/how-is-policy-affecting-classroom-instruction/ |
| 13 | Adam Edgerton | Morgan Polikoff, Laura Desimone | We see little evidence that teachersÕ beliefs about state policy are associated withtheir instructional choices. Certainly, there is some evidence that the accountability pressures that typically come with standards-based reforms can induce student learning gains. | Dismissive | How is policy affecting classroom instruction?, p.2 | Brookings Institution, Evidence Speaks Reports, Vol 2, #14, May 11, 2017 | Laura and John Arnold Foundation, USA Funds | https://www.brookings.edu/research/how-is-policy-affecting-classroom-instruction/ |
| 14 | Morgan Polikoff | Tenice Hardaway | Whereas most of the energy in the school choice debates has focused on vouchers and charter schools, relatively little attention has been paid to another important choice model that serves as many students as charters and has been in existence for longer—magnet schools. | Dismissive | Don't forget magnet schools when thinking about school choice, p.1 | Brookings Institution, Evidence Speaks Reports, Vol 2, #8, March 16, 2017 | Laura and John Arnold Foundation, USA Funds | https://www.brookings.edu/research/dont-forget-magnet-schools-when-thinking-about-school-choice/ |
| 15 | Susan M. Dynarski | Steven W. Hemelt, Joshua M. Hyman | NSC [National Student Clearinghouse] data are relatively new to academic researchers and policymakers. A growing number of papers make use of NSC data for research purposes (e.g., Bettinger, Long, Oreopoulos, & Sanbonmatsu, 2012; Chingos & Peterson, 2012; Deming, Hastings, Kane, & Staiger, 2014; Dynarski, Hyman, & Schanzenbach, 2013; Goldrick-Rab, Harris, Kelchen, & Benson, 2012; Hemelt, Roth, & Eaton, 2013; Hyman, 2013; Kane, 2003; Richburg-Hayes et al., 2009). | Dismissive | The Missing Manual: Using National Student Clearinghouse Data to Track Postsecondary Outcomes, p.54 | Educational Evaluation and Policy Analysis, May 2015, Vol. 37, No. 1S, pp. 53S–79S | Institute of Education Sciences, U.S. Department of Education, through Grants R305B110001 and R305E100008 to the University of Michigan, as well as through Grant R305C110011-11A to the Teachers College, Columbia University | http://journals.sagepub.com/doi/pdf/10.3102/0162373715576078 |
| 16 | Susan M. Dynarski | Steven W. Hemelt, Joshua M. Hyman | NSC [National Student Clearinghouse] data are relatively new to academic researchers and policymakers. A growing number of papers make use of NSC data for research purposes (e.g., Bettinger, Long, Oreopoulos, & Sanbonmatsu, 2012; Chingos & Peterson, 2012; Deming, Hastings, Kane, & Staiger, 2014; Dynarski, Hyman, & Schanzenbach, 2013; Goldrick-Rab, Harris, Kelchen, & Benson, 2012; Hemelt, Roth, & Eaton, 2013; Hyman, 2013; Kane, 2003; Richburg-Hayes et al., 2009). | Dismissive | The Missing Manual: Using National Student Clearinghouse Data to Track Postsecondary Outcomes, p.54 | Educational Evaluation and Policy Analysis, May 2015, Vol. 37, No. 1S, pp. 53S–79S | Institute of Education Sciences, U.S. Department of Education, through Grants R305B110001 and R305E100008 to the University of Michigan, as well as through Grant R305C110011-11A to the Teachers College, Columbia University | http://journals.sagepub.com/doi/pdf/10.3102/0162373715576078 |
| 17 | Ann Huff Stevens | Michal Kurlaender, Michel Grosz | "While career technical education (CTE) programs have often been mentioned as an attractive alternative to four-year colleges for some students, very little systematic evidence exists on the returns to specific vocational certiticates and degrees." | Dismissive | Career technical education and labor market outcomes: Evidence from California Community Colleges | National Bureau of Economic Research, NBER Working Paper #21137, Issued in April 2015, ' | ||
| 18 | Eric S. Taylor | John H. Tyler | "...very little is known about how the availability of new information, or the experience of being evaluated, might change teacher effort and effectiveness." | Dismissive | Can teacher evaluation improve teaching? Evidence of systematic growth in the effectiveness of midcareer teachers | Education Next, FALL 2012 / VOL. 12, NO. 4 | http://educationnext.org/can-teacher-evaluation-improve-teaching/ | |
| 19 | Eric S. Taylor | John H. Tyler | "In short, there are good reasons to expect that well-designed teacher-evaluation programs could have a direct and lasting effect on individual teacher performance. To our knowledge, however, ours is the first study to test this hypothesis directly. | 1stness | Can teacher evaluation improve teaching? Evidence of systematic growth in the effectiveness of midcareer teachers | Education Next, FALL 2012 / VOL. 12, NO. 4 | http://educationnext.org/can-teacher-evaluation-improve-teaching/ | |
| 20 | Finley Edwards | "Despite this attention, there is little rigorous evidence directly linking school start times and academic performance." | Denigrating | Do schools begin too early? The effect of start times on student achievement | Education Next, Summer 2012 / VOL. 12, NO. 3 | http://educationnext.org/do-schools-begin-too-early/ | ||
| 21 | Sa Bui | Steven Craig, Scott Imberman | "Three million students in the United States are classified as gifted, yet little is known about the effectiveness of traditional gifted and talented (G&T) programs." | Dismissive | Poor results for high achievers: New evidence on the impact of gifted and talented programs | Education Next, Winter 2012 / Vol. 12, No. 1 | http://educationnext.org/poor-results-for-high-achievers/ | |
| 22 | Sa Bui | Steven Craig, Scott Imberman | "To our knowledge, no existing studies offer convincing evidence on the causal effect of G&T programs on student achievement. Our research begins to fill this gap withÉ" | Dismissive | Poor results for high achievers: New evidence on the impact of gifted and talented programs | Education Next, Winter 2012 / Vol. 12, No. 1 | http://educationnext.org/poor-results-for-high-achievers/ | |
| 23 | David E. Marcotte | Benjamin Hansen | "This is not to say that there is no interest in extending the school year. While there has been little solid evidence that doing so will improve learning outcomes, the idea is often endorsed." | Denigrating | Time for school? When the snow falls, test scores also drop | Education Next, Winter 2010, Vol. 10, No. 1 | http://educationnext.org/time-for-school/ | |
| 24 | David E. Marcotte | Benjamin Hansen | "...in 1994 included not one study on the impact of additional instruction on learning. Researchers at that time simply had little direct evidence to offer." | Dismissive | Time for school? When the snow falls, test scores also drop | Education Next, Winter 2010, Vol. 10, No. 1 | http://educationnext.org/time-for-school/ | |
| 25 | David E. Marcotte | Benjamin Hansen | "Among the first researchers to try to identify the impact of variation in instructional time were economists studying the effect of schooling on labor market outcomes such as earnings." | 1stness | Time for school? When the snow falls, test scores also drop | Education Next, Winter 2010, Vol. 10, No. 1 | http://educationnext.org/time-for-school/ | |
| 26 | Helen F. Ladd | "Given what a decade of research tells us about test-based accountability it seems reasonable to think about policy changesÉ Here is my vision...." | Dismissive | Education Week, January 23, 2008, p.27. | ||||
| 27 | Daniel D. Goldhaber | "Few studies link principal attributes directly to student achievement,... This report includes new empirical research...." | 1stness | Principal compensation - More research needed on a promising reform | Center for American Progress, December 4, 2007 | |||
| 28 | Cecelia Elena Rouse | Jane Hannaway, Dan Goldhaber, & David Figlio | "Éthere has been little attention paid to substantive changes in instructional policies and practices resulting from school accountability. The lack of research is primarily due to the unavailability of appropriate data to carry out such analysis. This paper brings to bear new evidence from a remarkable five-year survey..." | 1stness | Feeling the Florida heat? How low-performing schools respond to voucher and accountability pressures, Abstract | National Center for Analysis of Longitudinal Data in Educational Research, Working Paper 13, November 2007 | ||
| 29 | Cecelia Elena Rouse | Jane Hannaway, Dan Goldhaber, & David Figlio | "Surprisingly, there has been little systematic effort to determine the substantive ways in which schools alter their methods of delivering education in response to school accountability and school choice pressures (see Hannaway and Hamilton, 2007, for a review). | Dismissive | Feeling the Florida heat? How low-performing schools respond to voucher and accountability pressures, Abstract | National Center for Analysis of Longitudinal Data in Educational Research, Working Paper 13, November 2007 | ||
| 30 | T.R. Stinebrickner | R. Stinebrickner | "Despite the large amount of attention that has been paid recently to understanding the determinants of educational outcomes, knowledge of the causal effect of the most fundamental input in the education production function - student study time and effort - has remained virtually non-existent. In this paper..." | Dismissive | The causal effect of studying on academic performance. | NBER Working Paper No. 13341, 2007 | ||
| 31 | Jane Hannaway | Andrew J. Rotherham | "We then compared notes about what we knew from research about collective bargaining and who was pursuing the topic in an analytic way. Despite the work of a few well-known figures, the landscape was sparsely populated.", p.1 | Dismissive | Collective Bargaining in Education: Negotiating Change in TodayÕs Schools | Harvard Education Press, Cambridge, Massachusetts | ||
| 32 | Tom Loveless | "It is important to keep in mind the limited body of data on the subject." | Dismissive | quoted in ÒNew Report Confirms,Ó February 11, 2003. | U.S. Congress: Committee on Education and the Workforce, news release | |||
| 33 | Tom Loveless | "We are just getting started in terms of solid research on standards, testing and accountability.Ó | Denigrating | quoted in ÒNew Report Confirms,Ó February 11, 2003. | U.S. Congress: Committee on Education and the Workforce, news release | |||
| 34 | Daniel D. Goldhaber | ÒHoover Institution senior fellow Eric Hanushek showed that only a small proportion of studies find these teacher characteristics to be statistically significant in the expected direction.Ó p. 53 | Dismissive | The Mystery of Good Teaching | Education Next, Spring 2002 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 35 | Daniel D. Goldhaber | ÒThere are also statistical shortcomings in many of the studies cited by both Hanushek and Greenwald et al.Ó p. 53 | Denigrating | The Mystery of Good Teaching | Education Next, Spring 2002 | http://educationnext.org/files/ednext20021_50.pdf | ||
| 36 | Daniel D. Goldhaber | ÒBut only four of these studies were based on studentsÕ outcomes and most of them were more than 25 years old, which means they predated the Ôvalue addedÕ methodology of assessing educational effects that is now standard practice.Ó p. 54 | Denigrating | The Mystery of Good Teaching | Education Next, Spring 2002 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 37 | Daniel D. Goldhaber | Ò[T]here is little research directly assessing the influence of pedagogical training on student outcomes, . . . .Ó p. 54 | Dismissive | The Mystery of Good Teaching | Education Next, Spring 2002 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 38 | Daniel D. Goldhaber | ÒAs Carolyn Evertson and her colleagues write, ÔInvestigations of teacher education do not represent a strong body of research.ÕÓ p. 54 | Denigrating | The Mystery of Good Teaching | Education Next, Spring 2002 | http://educationnext.org/files/ednext20021_50.pdf | ||
| 39 | Henry Levin | in G. Orfield & M. Kornhaber, (Eds. | ÒÉdisinterested appraisals of the research on the predictive validity of test scores conclude that there is only a very modest connection between test scores and productivity ratings by supervisors. Indeed, an overall summary of the potential economic gains from using test scores for employment selection suggests that the economic claims of industrial psychologists are flawed and highly exaggerated.Ó | Denigrating | High-stakes testing and economic productivity | Raising standards or raising barriers? Inequality and high-stakes testing in public education, New York: Century Foundation | ||
| 40 | Julian Betts | Jeff George | "Despite recent theoretical work and proposals from educational reformers, there is little empirical work on the effects of higher grading standards. In this paperÉ" | Dismissive | The impact of grading standards on student achievement, educational attainment, and entry-level earnings, abstract | National Bureau of Economic Research, Working Paper 7875 | This research was supported by a grant from the American Educational Research Association which receives funds for its "AERA Grants Program" from the National Science Foundation and the National Center for Education Sttistics under NSF Grant #RED-9452861. | http://www.nber.org/papers/w7875 |
| 41 | Julian Betts | Jeff George | "...it is surprising how little empirical work has been devoted to understanding how other aspects of the educational environment affect student behavior. In particular, given economistsÕ general interests in incentive schemes, it is surprising how little empirical work has focused on educational incentives." | Dismissive | The impact of grading standards on student achievement, educational attainment, and entry-level earnings, p.1 | National Bureau of Economic Research, Working Paper 7875 | This research was supported by a grant from the American Educational Research Association which receives funds for its "AERA Grants Program" from the National Science Foundation and the National Center for Education Sttistics under NSF Grant #RED-9452861. | http://www.nber.org/papers/w7875 |
| 42 | Julian Betts | Jeff George | "The only other empirical study that we know of that addresses similar questions is Lillard and DeCicca (forthcoming)." | Dismissive | The impact of grading standards on student achievement, educational attainment, and entry-level earnings, p.1 | National Bureau of Economic Research, Working Paper 7875 | This research was supported by a grant from the American Educational Research Association which receives funds for its "AERA Grants Program" from the National Science Foundation and the National Center for Education Sttistics under NSF Grant #RED-9452861. | http://www.nber.org/papers/w7875 |
| 43 | Helen F. Ladd | "Given the widespread interest in school-based recognition and reward programs, it is surprising how little evaluation has been done of their impacts." | Dismissive | The Dallas School Accountability and Incentive Program: An Evaluation of its Impacts on Student Outcomes | Economics of Education Review, 1999 | |||
| 44 | Helen F. Ladd | "...several states and a few local districts have introduced school-based incentive programs. This paper provides one of the few evaluations of the effects of such programs on student outcomes." | 1stness | The Dallas School Accountability and Incentive Program: An Evaluation of its Impacts on Student Outcomes | Economics of Education Review, 1999 | |||
| 45 | Daniel D. Goldhaber | Dominic J. Brewer | Ò There are good reasons to believe that many educational production function studies, particularly those completed in the 1970s, had major deficiencies in empirical methodology and available data.Ó p. 4 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | |
| 46 | Daniel D. Goldhaber | Dominic J. Brewer | Ò For example, many early studies were unable to control for prior achievement using a "pre-test" score to net out individual ability, as is now generally accepted to be important (Boardman and Murnane, 1979; Hanushek, 1979; Hedges et al., 1994).Ó p. 5 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | |
| 47 | Daniel D. Goldhaber | Dominic J. Brewer | Ò Variables representing school and teacher ÔqualityÕ that are used in most production function studies are typically very crude.Ó p. 5 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | |
| 48 | Daniel D. Goldhaber | Dominic J. Brewer | ÒData deficiencies may also have led to significant measurement error problems in previous studies.Ó p. 6 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | |
| 49 | Daniel D. Goldhaber | Dominic J. Brewer | ÒThis link enables us to avoid problems with aggregation that may have plagued earlier studies.Ó `p. 20 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | |
| 50 | Daniel D. Goldhaber | Ronald G. Ehrenberg, Dominic J. Brewer | ÒMost of this research [on the effectiveness of minority teachers in educating minority students] has not addressed the students' educational outcomes; has failed to control for other teacher characteristics, such as verbal ability, experience, and degree levels; and has not investigated the effects that under-represented minority teachers have on non-minority students.Ó p. 548 | Denigrating | Do TeachersÕ Race, Gender, and Ethnicity Matter? Evidence From the National Education Longitudinal Study of 1988 | Industrial and Labor Relations Review, Vol. 48, No. 3 (April 1995) | http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1626&context=articles | |
| 51 | Daniel D. Goldhaber | Ronald G. Ehrenberg, Dominic J. Brewer | ÒThe few studies that do address outcomes focus on the correlation between teacher gender and students' test scores at a point in time, Ò p. 548 | Denigrating | Do TeachersÕ Race, Gender, and Ethnicity Matter? Evidence From the National Education Longitudinal Study of 1988 | Industrial and Labor Relations Review, Vol. 48, No. 3 (April 1995) | http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1626&context=articles | |
| 52 | Daniel D. Goldhaber | Ronald G. Ehrenberg, Dominic J. Brewer | ÒIn contrast to the previous literature, we focus both on how teachers subjectively relate to and evaluate their students and on how much their students learn, as measured by standardized tests.Ó p. 548 | 1stness | Do TeachersÕ Race, Gender, and Ethnicity Matter? Evidence From the National Education Longitudinal Study of 1988 | Industrial and Labor Relations Review, Vol. 48, No. 3 (April 1995) | http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1626&context=articles | |
| IRONIES: | ||||||||
| Frederick M. Hess | "É would-be reformers err when they presume to know a lot more than they do." | Twitter tweet Feb 12, 2018 | ||||||
| Author cites (and accepts as fact without checking) someone elses dismissive review | ||||||||
| Cite selves or colleagues in the group, but dismiss or denigrate all other work | ||||||||
| Falsely claim that research has only recently been done on topic. |