| HOME: Dismissive Reviews in Education Policy Research | |||||||||
| Author | Co-author(s) | Dismissive Quote | type | Title | Source | Funders | Link1 | ||
| 1 | Catherine Welch, Stephen Dunbar, Eds. | Morgan Polikoff | "Polikoff (this issue) would argue that given the importance of alignment in policy decisions, research related to alignment is surprisingly limited." | Dismissive | Alignment and Implications for Test Takers, Abstract | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
| 2 | Morgan Polikoff | "And there are not many important measurement topics that draw less scholarly attention than alignment." p.1 | Dismissive | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 3 | Morgan Polikoff | "There are just a handful of researchers regularly doing research on alignment, and the core alignment methods have seen, at most, modest technical advances since their creation. There is a need for a broader alignment agenda—one that tackles interesting substantive alignment questions from multiple angles and that advances alignment methodologies to make them more rigorous and more approachable for a wider array of researchers" p.1 | Dismissive, Denigrating | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 4 | Morgan Polikoff | "the alignment methods have not advanced much at all since the earliest instantiations of this approach—the main alignment indices, for example, are largely unchanged since the original methods were proposed." p.1 | Denigrating | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 5 | Morgan Polikoff | "But these two approaches clearly dominate. And for a concept as central to policy as alignment, the science of alignment methods is surprisingly limited and weak." p.1 | Dismissive, Denigrating | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 6 | Morgan Polikoff | "a quick scan through the table of contents of major educational measurement journals reveals that there are very few articles focused on the development or application of alignment methods (by my count seven since 2010 in this journal and just one apiece in Applied Measurement in Education and Journal of Educational Measurement). As compared to articles on, say, item response theory, standard setting, or scale validation, these counts are quite scant." p.1 | Dismissive | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 7 | Morgan Polikoff | "Why has there been so little peer-reviewed research on alignment methods, and why has the science advanced in such a limited fashion?" p.1 | Dismissive, Denigrating | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 8 | Morgan Polikoff | "Furthermore, there has obviously not been demand for improving alignment methods—the federal peer review requirements have not pushed for more rigor in these methods." p.1 | Dismissive, Denigrating | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 9 | Morgan Polikoff | "As a result, the alignment methods have not changed much, and the science of alignment research remains weak relative to other areas of test construction and validation" p.1 | Dismissive, Denigrating | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 10 | Morgan Polikoff | "Claims of alignment go unverified, because there is no standard way to verify them (nor the resources to do the verification)." pp.1-2 | Dismissive, Denigrating | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 11 | Morgan Polikoff | "There is also a great need for descriptive alignment research along a number of different dimensions. Perhaps, the most important (and most difficult) is a serious investigation of the instructional alignment of teachers’ actual instruction. To date, the only such investigations have used the SEC (not counting survey- or interview-based studies with obvious methodological challenges, see Polikoff, 2012a, for an example and discussion of this topic). Furthermore, these studies are dated—drawing on data from the middle part of the No Child Left Behind era. There is a need to update this research to get a revised look at instructional alignment under Common Core and other college- and career-ready standards. In particular," | Dismissive, Denigrating | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 12 | Morgan Polikoff | "Another important and unexplored area is the alignment of supplemental curriculum materials with standards" | Dismissive | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 13 | Morgan Polikoff | "Supplemental materials are a massive phenomenon— virtually all teachers source materials from Google, Pinterest, and TeachersPayTeachers (Kaufman, Thompson, & Opfer, 2016)—yet we know close to nothing about these materials and their content." | Dismissive | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 14 | Morgan Polikoff | "I have argued here that the last decade and a half has seen disappointing effort to advance alignment methods and apply them to answer important policy questions." | Denigrating | The Present and Future of Alignment | Educational Measurement: Issues and Practice, May 23, 2020 | journal of the National Council on Measurement in Education | https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12332 | Polikoff
claims (p. 1) to have conducted "a quick scan" of the post-2010
editions of three US education measurement journals and found nine articles
on alignment, which he asserts represents a "dated,"
"few," and "limited and weak" collection. This
"quick scan" in just three journals in just one decade in just one
country apparently justified to the editors publishing his many global
declarations of an alleged lack of research. No matter that there exist
hundreds of other countries, a century's worth of research prior to 2010,
literally thousands of other journals that might publish such a article, and
a large "grey literature" of alignment studies conducted as routine
parts of test development. Virtually any standards-based, large-scale test
development includes an alignment study, not to be found in a scholarly
journal. A short time before this journal published Polikoff's commentary
consisting of nothing but his thinly substantiated claims of a lack of
research, I proposed a commentary criticising dismissive reviews; it was
rejected. Some notable alignment
studies: with NRTs: Freeman, Kuhs,
Porter, Floden, Schmidt, Schwille (1983); Debra P. v. Turlington (1984);
Cohen, Spillane (1993); La Marca, Redfield, Winter, Bailey, and Despriet
(2000); Wainer (2011) with Standards: Archbald (1994); Buckendahl, Plake, Impara, Irwin (2000); Bhola, Impara, Buckendahl (2003); Phelps (2005) with RTs: Massell, Kirst, Hoppe (1997); Wiley, Hembry, Buckendahl, Forte,Towles Nebelsick-Gullett (2015) |
|
| 15 | Mike Petrilli | "To be sure, education research improved dramatically starting in the early 2000s with the creation of the Institute of Education Sciences, the federal mandate for annual tests in grades three through eight, and the concurrent development of longitudinal data systems in most states." | Denigrating | Practicing humility when it comes to evidence-based practice | Thomas B. Fordham Institute Flypaper, Jan. 16, 2019 | (6) Fordham funders | https://edexcellence.net/articles/practicing-humility-when-it-comes-to-evidence-based-practice | There's little humulity apparent in these three statements. | |
| 16 | Mike Petrilli | "Whereas the world outside of our schools has been transformed by information technology, the data we collect on classroom practices is somewhere between nonexistent and laughably rudimentary. In other words, we know almost nothing about almost everything that matters." | Dismissive, Denigrating | Practicing humility when it comes to evidence-based practice | Thomas B. Fordham Institute Flypaper, Jan. 16, 2019 | (6) Fordham funders | https://edexcellence.net/articles/practicing-humility-when-it-comes-to-evidence-based-practice | There's little humulity apparent in these three statements. | |
| 17 | Mike Petrilli | "the evidence around educational effectiveness is extremely limited. The number of areas where we have strong science to guide classroom practice is tiny. | Dismissive | Practicing humility when it comes to evidence-based practice | Thomas B. Fordham Institute Flypaper, Jan. 16, 2019 | (6) Fordham funders | https://edexcellence.net/articles/practicing-humility-when-it-comes-to-evidence-based-practice | There's little humulity apparent in these three statements. | |
| 18 | Mike Petrilli | "In fact, it may be sui generis: early reading. There we do have a scientific consensus, or close to it, around “what works,”But that’s a rare case. Much more common are parts of the curriculum and the educational experience where we hardly have any scientific evidence at all." | Dismissive | Practicing humility when it comes to evidence-based practice | Thomas B. Fordham Institute Flypaper, Jan. 16, 2019 | (6) Fordham funders | https://edexcellence.net/articles/practicing-humility-when-it-comes-to-evidence-based-practice | "A
simple search for academic journals with the word “curriculum” in their title
unearths more than a dozen journals with around 6,000 articles dating back to
1968: Curriculum Inquiry, from 1968, >2,500 articles; Journal of
Curriculum Studies, from 1968, >1,500 articles; Language, Culture, and
Curriculum, from 1988, >450 articles; Curriculum Journal, from 1990,
>700 articles; Curriculum and Teaching, from 1995, ~500 articles; Teachers
and Curriculum, from 1997, >200 articles; Curriculum and Teaching
Dialogue, from 1998, >500 articles; Journal of Curriculum and Pedagogy,
from 2004, >300 articles; Curriculum Matters, from 2005, ~100 articles;
Journal of Curriculum and Instruction, 2007–2014, >50; International
Journal of Curriculum and Instruction, from 2008, >150 articles; and
Journal of Curriculum and Teaching, from 2013, >300 articles. All that represents just a small proportion of research articles on curriculum that one would find if one searched all of the many thousands of education journals dating back a century, as well as the “grey literature” of graduate student theses, program evaluations, and governmental reports. A search on “curriculum” in ERIC gets 215,139 hits, in Google Scholar, 3,440,000 hits. To believe the dismissive review, one must believe that none among these many thousands of sources conducted a curricular evaluation or comparison. " |
|
| 19 | Mike Petrilli | "What’s the “best” way to teach U.S. history? Civics? Biology? Welding? If there are rigorous studies on these topics, that’s news to me." | Dismissive | Practicing humility when it comes to evidence-based practice | Thomas B. Fordham Institute Flypaper, Jan. 16, 2019 | (6) Fordham funders | https://edexcellence.net/articles/practicing-humility-when-it-comes-to-evidence-based-practice | ||
| 20 | Mayowa Aina, National Public Radio (NPR) | Constance Lindsey [interviewee] | "This is the first study of its kind to show that there are long-run effects." | 1stness | To Get To College, It Helps Black Students To Have A Black Teacher Early On | NPR.org, January 11, 2019 | NPR, Urban Institute funders | ||
| 21 | Mike Petrilli | "People have been trying to figure out what works in education for at least fifty years. But we still haven’t come close to cracking this nut, and if we want to make progress, we need to figure it out." | Dismissive | Identifying "what works" is still a work in progress | Thomas B. Fordham Institute Flypaper, Dec. 12, 2018 | (6) Fordham funders | https://edexcellence.net/articles/identifying-what-works-is-still-a-work-in-progress | ||
| 22 | Mike Petrilli | "The best part about these questions is that their answers are knowable. In an ideal world, it would go something like this: Educators identify key instructional questions for which they would like empirical answers—like those above. (Morgan Polikoff and Carrie Conaway have ideas on how to solicit those questions.)" | Dismissive | Identifying "what works" is still a work in progress | Thomas B. Fordham Institute Flypaper, Dec. 12, 2018 | (6) Fordham funders | https://edexcellence.net/articles/identifying-what-works-is-still-a-work-in-progress | ||
| 23 | Morgan Polikoff | "However, research about textbooks has been hampered by a lack of good data. Simply put, very few states have collected useful data on which textbooks are used where." | Dismissive | How Good Statewide Data and Careful Research on School Textbooks Can Improve Student Learning | Scholars Strategy Network | https://scholars.org/brief/how-good-statewide-data-and-careful-research-school-textbooks-can-improve-student-learning | "A
simple search for academic journals with the word “curriculum” in their title
unearths more than a dozen journals with around 6,000 articles dating back to
1968: Curriculum Inquiry, from 1968, >2,500 articles; Journal of
Curriculum Studies, from 1968, >1,500 articles; Language, Culture, and
Curriculum, from 1988, >450 articles; Curriculum Journal, from 1990,
>700 articles; Curriculum and Teaching, from 1995, ~500 articles; Teachers
and Curriculum, from 1997, >200 articles; Curriculum and Teaching
Dialogue, from 1998, >500 articles; Journal of Curriculum and Pedagogy,
from 2004, >300 articles; Curriculum Matters, from 2005, ~100 articles;
Journal of Curriculum and Instruction, 2007–2014, >50; International
Journal of Curriculum and Instruction, from 2008, >150 articles; and
Journal of Curriculum and Teaching, from 2013, >300 articles. All that represents just a small proportion of research articles on curriculum that one would find if one searched all of the many thousands of education journals dating back a century, as well as the “grey literature” of graduate student theses, program evaluations, and governmental reports. A search on “curriculum” in ERIC gets 215,139 hits, in Google Scholar, 3,440,000 hits. To believe the dismissive review, one must believe that none among these many thousands of sources conducted a curricular evaluation or comparison. " |
||
| 24 | Katharine O. Strunk | Dan Goldhaber, David S. Knight, Nate Brown | "...we provide the first evidence about the impact of the layoff process on teacher productivity." p.755 | 1stness | Are There Hidden Costs Associated With Conducting Layoffs? The Impact of Reduction-in-Force and Layoff Notices on Teacher Effectiveness | Journal of Policy Analysis and Management, Vol. 37, No. 4, 755–782 (2018) | "We gratefully acknowledge ... funding for this study from the Center for Analysis of Longitudinal Data in Education Research (CALDER), which is funded through Award #R305C120008 by the Institute of Education Sciences, U.S. Department of Education, and an anonymous foundation." | ||
| 25 | Katharine O. Strunk | Dan Goldhaber, David S. Knight, Nate Brown | "And, to our knowledge, there has been no assessment of whether the threat of job loss affects worker productivity." p.756 | Dismissive | Are There Hidden Costs Associated With Conducting Layoffs? The Impact of Reduction-in-Force and Layoff Notices on Teacher Effectiveness | Journal of Policy Analysis and Management, Vol. 37, No. 4, 755–782 (2018) | "We gratefully acknowledge ... funding for this study from the Center for Analysis of Longitudinal Data in Education Research (CALDER), which is funded through Award #R305C120008 by the Institute of Education Sciences, U.S. Department of Education, and an anonymous foundation." | ||
| 26 | Katharine O. Strunk | Dan Goldhaber, David S. Knight, Nate Brown | "This paper is the first that we know of to assess the impact of the layoff process on employee productivity." p.777 | 1stness | Are There Hidden Costs Associated With Conducting Layoffs? The Impact of Reduction-in-Force and Layoff Notices on Teacher Effectiveness | Journal of Policy Analysis and Management, Vol. 37, No. 4, 755–782 (2018) | "We gratefully acknowledge ... funding for this study from the Center for Analysis of Longitudinal Data in Education Research (CALDER), which is funded through Award #R305C120008 by the Institute of Education Sciences, U.S. Department of Education, and an anonymous foundation." | ||
| 27 | Mike Petrilli | "Students who learn dramatically more at school, as measured by valid and reliable assessments, will go on to graduate from high school, enroll in and complete postsecondary education, and earn more as adults than similar peers who learn less. You would think that there would be lots of studies looking at students’ learning gains in elementary or middle school and how that impacts their high school graduation or college enrollment rates. Yet to my knowledge none exist. (Academics: Let’s change that please!)" | Dismissive | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | (6) Fordham funders | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 28 | Mike Petrilli | "What we do have is the famous Raj Chetty et al. study examining teacher value-added, which found that students who learn more in elementary school earn more as adults. It’s just one study, but it’s a remarkable finding, one that might be hard to replicate unless more scholars can gain access to the tax data Chetty and his colleagues have." | 1stness | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | (6) Fordham funders | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 29 | Mike Petrilli | "Elementary and middle schools that dramatically boost the achievement of their students should also boost their long-term outcomes, including high school graduation, postsecondary enrollment, performance, and completion, as well as later earnings. Here we have a bit more to go on, at least if we look at studies that examine both individual schools and programs that are focused at least in part on elementary or middle schools. Remember that we’re interested in schools or programs that make a significant impact on achievement, for good or ill. According to Hitt, McShane, and Wolf’s review, there are four of those." | Dismissive | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | (6) Fordham funders | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 30 | Mike Petrilli | "High schools that dramatically boost the achievement of their students should also boost their long-term outcomes, including postsecondary enrollment, performance, and completion, and earnings. Here the research base is a tad larger. We can start with a 2016 study of Texas’s accountability system by all-stars David J. Deming, Sarah Cohodes, Jennifer Jennings, and Christopher Jencks, ... | Dismissive | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | (6) Fordham funders | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 31 | Mike Petrilli | "The research base is very thin—too thin for a serious meta-analysis. With only nine relevant studies, this is clearly a field still in its infancy." | Dismissive | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | (6) Fordham funders | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 32 | Mike Petrilli | "No doubt this debate will continue; we plainly need a lot more empirical evidence to inform it." | Dismissive | The evidence on test scores and long-term outcomes: Limited but encouraging | Thomas B. Fordham Institute Flypaper, May 8, 2018 | (6) Fordham funders | https://edexcellence.net/articles/the-evidence-on-test-scores-and-long-term-outcomes-limited-but-encouraging | ||
| 33 | Morgan Polikoff | "This talk summarizes the growing evidence on the importance of curriculum. Long neglected by policy researchers, curriculum (including both traditional textbooks and online and technology-enhanced materials) is emerging as a potentially promising lever for improving students learning and instructional quality." | Dismissive | Curriculum Matters | ResearchED New York 2017 Conference Program, p.5 | ResearchED | A
simple search for academic journals with the word “curriculum” in their title
unearths more than a dozen journals with around 6,000 articles dating back to
1968: Curriculum Inquiry, from 1968, >2,500 articles; Journal of
Curriculum Studies, from 1968, >1,500 articles; Language, Culture, and
Curriculum, from 1988, >450 articles; Curriculum Journal, from 1990,
>700 articles; Curriculum and Teaching, from 1995, ~500 articles; Teachers
and Curriculum, from 1997, >200 articles; Curriculum and Teaching
Dialogue, from 1998, >500 articles; Journal of Curriculum and Pedagogy,
from 2004, >300 articles; Curriculum Matters, from 2005, ~100 articles;
Journal of Curriculum and Instruction, 2007–2014, >50; International
Journal of Curriculum and Instruction, from 2008, >150 articles; and
Journal of Curriculum and Teaching, from 2013, >300 articles. All that represents just a small proportion of research articles on curriculum that one would find if one searched all of the many thousands of education journals dating back a century, as well as the “grey literature” of graduate student theses, program evaluations, and governmental reports. A search on “curriculum” in ERIC gets 215,139 hits, in Google Scholar, 3,440,000 hits. To believe the dismissive review, one must believe that none among these many thousands of sources conducted a curricular evaluation or comparison. |
||
| 34 | Morgan Polikoff | "David Figlio and Susanna Loeb's 2011 review of [test-based accountability] research summarizes this literature comprehansively." | Dismissive | Why accountability matters, and why it must evolve. | Education Next, SUMMER 2017 / VOL. 17, NO. 3 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | No, it does not. See, for example, Test Frequency, Stakes, and Feedback in Student Achievement: A Meta-Analysis https://journals.sagepub.com/doi/abs/10.1177/0193841X19865628#abstract | ||
| 35 | Matt Barnum | Max Eden [interviewee] | "Using anecdotal evidence and surveys, critics claim that restricting suspensions may have a deleterious effect on school safety and climate, particularly without support, resources, or broader structural reforms. That’s the argument put forth in a report focusing on New York City by Max Eden, a senior fellow at the Manhattan Institute," | Denigrating | The School Discipline Revolution: How Policy and Rhetoric Outstrip Hard Evidence | The 74, May 2, 2017 | The 74 funders (2) | https://www.the74million.org/article/the-school-discipline-revolution-how-policy-and-rhetoric-outstrip-hard-evidence/ | |
| 36 | Matt Barnum | Morgan Polikoff [interviewee] | "One problem is simply a lack of information, according to Morgan Polikoff, a professor at the University of Southern California and co-author of one of the California studies. 'Very few states keep track of which districts adopt which books,' he said." | Dismissive | New Studies Suggest Choice of Curriculum and Textbooks Can Make a Big Difference for Students | The 74, May 1, 2017 | The 74 funders (7) | https://www.the74million.org/article/new-studies-suggest-choice-of-curriculum-and-textbooks-can-make-a-big-difference-for-students/ | A
simple search for academic journals with the word “curriculum” in their title
unearths more than a dozen journals with around 6,000 articles dating back to
1968: Curriculum Inquiry, from 1968, >2,500 articles; Journal of
Curriculum Studies, from 1968, >1,500 articles; Language, Culture, and
Curriculum, from 1988, >450 articles; Curriculum Journal, from 1990,
>700 articles; Curriculum and Teaching, from 1995, ~500 articles; Teachers
and Curriculum, from 1997, >200 articles; Curriculum and Teaching
Dialogue, from 1998, >500 articles; Journal of Curriculum and Pedagogy,
from 2004, >300 articles; Curriculum Matters, from 2005, ~100 articles;
Journal of Curriculum and Instruction, 2007–2014, >50; International
Journal of Curriculum and Instruction, from 2008, >150 articles; and
Journal of Curriculum and Teaching, from 2013, >300 articles. All that represents just a small proportion of research articles on curriculum that one would find if one searched all of the many thousands of education journals dating back a century, as well as the “grey literature” of graduate student theses, program evaluations, and governmental reports. A search on “curriculum” in ERIC gets 215,139 hits, in Google Scholar, 3,440,000 hits. To believe the dismissive review, one must believe that none among these many thousands of sources conducted a curricular evaluation or comparison. |
| 37 | Matt Barnum | David Steiner [interviewee] | "'To date, research on the curriculum effect has told us little about what makes a particular curriculum or genre of curriculum especially effective or not,” wrote Steiner." | Dismissive | New Studies Suggest Choice of Curriculum and Textbooks Can Make a Big Difference for Students | The 74, May 1, 2017 | The 74 funders (7) | https://www.the74million.org/article/new-studies-suggest-choice-of-curriculum-and-textbooks-can-make-a-big-difference-for-students/ | A
simple search for academic journals with the word “curriculum” in their title
unearths more than a dozen journals with around 6,000 articles dating back to
1968: Curriculum Inquiry, from 1968, >2,500 articles; Journal of
Curriculum Studies, from 1968, >1,500 articles; Language, Culture, and
Curriculum, from 1988, >450 articles; Curriculum Journal, from 1990,
>700 articles; Curriculum and Teaching, from 1995, ~500 articles; Teachers
and Curriculum, from 1997, >200 articles; Curriculum and Teaching
Dialogue, from 1998, >500 articles; Journal of Curriculum and Pedagogy,
from 2004, >300 articles; Curriculum Matters, from 2005, ~100 articles;
Journal of Curriculum and Instruction, 2007–2014, >50; International
Journal of Curriculum and Instruction, from 2008, >150 articles; and
Journal of Curriculum and Teaching, from 2013, >300 articles. All that represents just a small proportion of research articles on curriculum that one would find if one searched all of the many thousands of education journals dating back a century, as well as the “grey literature” of graduate student theses, program evaluations, and governmental reports. A search on “curriculum” in ERIC gets 215,139 hits, in Google Scholar, 3,440,000 hits. To believe the dismissive review, one must believe that none among these many thousands of sources conducted a curricular evaluation or comparison. |
| 38 | Matt Barnum | David Steiner [interviewee] | "'The paucity of evidence upon which sound instructional, purchasing, and policy decisions can be made is a matter of deep concern and urgent need,' wrote Steiner." | Dismissive | New Studies Suggest Choice of Curriculum and Textbooks Can Make a Big Difference for Students | The 74, May 1, 2017 | The 74 funders (7) | https://www.the74million.org/article/new-studies-suggest-choice-of-curriculum-and-textbooks-can-make-a-big-difference-for-students/ | A
simple search for academic journals with the word “curriculum” in their title
unearths more than a dozen journals with around 6,000 articles dating back to
1968: Curriculum Inquiry, from 1968, >2,500 articles; Journal of
Curriculum Studies, from 1968, >1,500 articles; Language, Culture, and
Curriculum, from 1988, >450 articles; Curriculum Journal, from 1990,
>700 articles; Curriculum and Teaching, from 1995, ~500 articles; Teachers
and Curriculum, from 1997, >200 articles; Curriculum and Teaching
Dialogue, from 1998, >500 articles; Journal of Curriculum and Pedagogy,
from 2004, >300 articles; Curriculum Matters, from 2005, ~100 articles;
Journal of Curriculum and Instruction, 2007–2014, >50; International
Journal of Curriculum and Instruction, from 2008, >150 articles; and
Journal of Curriculum and Teaching, from 2013, >300 articles. All that represents just a small proportion of research articles on curriculum that one would find if one searched all of the many thousands of education journals dating back a century, as well as the “grey literature” of graduate student theses, program evaluations, and governmental reports. A search on “curriculum” in ERIC gets 215,139 hits, in Google Scholar, 3,440,000 hits. To believe the dismissive review, one must believe that none among these many thousands of sources conducted a curricular evaluation or comparison. |
| 39 | David Steiner | "Research comparing one curriculum to another is very rare and, therefore, not usually actionable." | Dismissive | Curriculum Research: What We Know and Where We Need to Go | March 2017, p.1 | (5) StandardsWork funders | https://standardswork.org/wp-content/uploads/2017/03/sw-curriculum-research-report-fnl.pdf | A
simple search for academic journals with the word “curriculum” in their title
unearths more than a dozen journals with around 6,000 articles dating back to
1968: Curriculum Inquiry, from 1968, >2,500 articles; Journal of
Curriculum Studies, from 1968, >1,500 articles; Language, Culture, and
Curriculum, from 1988, >450 articles; Curriculum Journal, from 1990,
>700 articles; Curriculum and Teaching, from 1995, ~500 articles; Teachers
and Curriculum, from 1997, >200 articles; Curriculum and Teaching
Dialogue, from 1998, >500 articles; Journal of Curriculum and Pedagogy,
from 2004, >300 articles; Curriculum Matters, from 2005, ~100 articles;
Journal of Curriculum and Instruction, 2007–2014, >50; International
Journal of Curriculum and Instruction, from 2008, >150 articles; and
Journal of Curriculum and Teaching, from 2013, >300 articles. All that represents just a small proportion of research articles on curriculum that one would find if one searched all of the many thousands of education journals dating back a century, as well as the “grey literature” of graduate student theses, program evaluations, and governmental reports. A search on “curriculum” in ERIC gets 215,139 hits, in Google Scholar, 3,440,000 hits. To believe the dismissive review, one must believe that none among these many thousands of sources conducted a curricular evaluation or comparison. |
|
| 40 | David Steiner | "The paucity of evidence upon which sound instructional, purchasing, and policy decisions can be made is a matter of deep concern and urgent need." | Dismissive | Curriculum Research: What We Know and Where We Need to Go | March 2017, p.1 | (5) StandardsWork funders | https://standardswork.org/wp-content/uploads/2017/03/sw-curriculum-research-report-fnl.pdf | A
simple search for academic journals with the word “curriculum” in their title
unearths more than a dozen journals with around 6,000 articles dating back to
1968: Curriculum Inquiry, from 1968, >2,500 articles; Journal of
Curriculum Studies, from 1968, >1,500 articles; Language, Culture, and
Curriculum, from 1988, >450 articles; Curriculum Journal, from 1990,
>700 articles; Curriculum and Teaching, from 1995, ~500 articles; Teachers
and Curriculum, from 1997, >200 articles; Curriculum and Teaching
Dialogue, from 1998, >500 articles; Journal of Curriculum and Pedagogy,
from 2004, >300 articles; Curriculum Matters, from 2005, ~100 articles;
Journal of Curriculum and Instruction, 2007–2014, >50; International
Journal of Curriculum and Instruction, from 2008, >150 articles; and
Journal of Curriculum and Teaching, from 2013, >300 articles. All that represents just a small proportion of research articles on curriculum that one would find if one searched all of the many thousands of education journals dating back a century, as well as the “grey literature” of graduate student theses, program evaluations, and governmental reports. A search on “curriculum” in ERIC gets 215,139 hits, in Google Scholar, 3,440,000 hits. To believe the dismissive review, one must believe that none among these many thousands of sources conducted a curricular evaluation or comparison. |
|
| 41 | David Steiner | "To date, research on the curriculum effect has told us little about what makes a particular curriculum or genre of curriculum especially effective or not. We encounter only occasional, anecdotal observations on this in the research. | Denigrating | Curriculum Research: What We Know and Where We Need to Go | March 2017, p.7 | (5) StandardsWork funders | https://standardswork.org/wp-content/uploads/2017/03/sw-curriculum-research-report-fnl.pdf | A
simple search for academic journals with the word “curriculum” in their title
unearths more than a dozen journals with around 6,000 articles dating back to
1968: Curriculum Inquiry, from 1968, >2,500 articles; Journal of
Curriculum Studies, from 1968, >1,500 articles; Language, Culture, and
Curriculum, from 1988, >450 articles; Curriculum Journal, from 1990,
>700 articles; Curriculum and Teaching, from 1995, ~500 articles; Teachers
and Curriculum, from 1997, >200 articles; Curriculum and Teaching
Dialogue, from 1998, >500 articles; Journal of Curriculum and Pedagogy,
from 2004, >300 articles; Curriculum Matters, from 2005, ~100 articles;
Journal of Curriculum and Instruction, 2007–2014, >50; International
Journal of Curriculum and Instruction, from 2008, >150 articles; and
Journal of Curriculum and Teaching, from 2013, >300 articles. All that represents just a small proportion of research articles on curriculum that one would find if one searched all of the many thousands of education journals dating back a century, as well as the “grey literature” of graduate student theses, program evaluations, and governmental reports. A search on “curriculum” in ERIC gets 215,139 hits, in Google Scholar, 3,440,000 hits. To believe the dismissive review, one must believe that none among these many thousands of sources conducted a curricular evaluation or comparison. |
|
| 42 | David Steiner | "The rapid growth of online, personalized learning platforms will likely change classroom instruction further. As of yet, there exists no high-quality research on the impact of such platforms…" | Denigrating | Curriculum Research: What We Know and Where We Need to Go | March 2017, p.6 | (5) StandardsWork funders | https://standardswork.org/wp-content/uploads/2017/03/sw-curriculum-research-report-fnl.pdf | A
simple search for academic journals with the word “curriculum” in their title
unearths more than a dozen journals with around 6,000 articles dating back to
1968: Curriculum Inquiry, from 1968, >2,500 articles; Journal of
Curriculum Studies, from 1968, >1,500 articles; Language, Culture, and
Curriculum, from 1988, >450 articles; Curriculum Journal, from 1990,
>700 articles; Curriculum and Teaching, from 1995, ~500 articles; Teachers
and Curriculum, from 1997, >200 articles; Curriculum and Teaching
Dialogue, from 1998, >500 articles; Journal of Curriculum and Pedagogy,
from 2004, >300 articles; Curriculum Matters, from 2005, ~100 articles;
Journal of Curriculum and Instruction, 2007–2014, >50; International
Journal of Curriculum and Instruction, from 2008, >150 articles; and
Journal of Curriculum and Teaching, from 2013, >300 articles. All that represents just a small proportion of research articles on curriculum that one would find if one searched all of the many thousands of education journals dating back a century, as well as the “grey literature” of graduate student theses, program evaluations, and governmental reports. A search on “curriculum” in ERIC gets 215,139 hits, in Google Scholar, 3,440,000 hits. To believe the dismissive review, one must believe that none among these many thousands of sources conducted a curricular evaluation or comparison. |
|
| 43 | Blake Heller | Matthew Davis | "And although there is a robust positive correlation between test performance and college enrollment, there is little existing evidence as to whether schools that increase test scores the most also help their students succeed at the next level." | Dismissive | Raising more than test scores | Education Next, WINTER 2017 / VOL. 17, NO. 1 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/raising-more-than-test-scores-noble-charter-no-excuses/ | |
| 44 | Adam Edgerton | Morgan Polikoff, Laura Desimone | Five-plus years into the experiment with new “college- and career-ready standards”, we know little about teachers’ implementation and the ways policy can support that implementation. This paper… | Dismissive | How is policy affecting classroom instruction?, p.1 | Brookings Institution, Evidence Speaks Reports, Vol 2, #14, May 11, 2017 | Laura and John Arnold Foundation, USA Funds | https://www.brookings.edu/research/how-is-policy-affecting-classroom-instruction/ | |
| 45 | Adam Edgerton | Morgan Polikoff, Laura Desimone | We see little evidence that teachers’ beliefs about state policy are associated with their instructional choices. Certainly, there is some evidence that the accountability pressures that typically come with standards-based reforms can induce student learning gains. | Dismissive | How is policy affecting classroom instruction?, p.2 | Brookings Institution, Evidence Speaks Reports, Vol 2, #14, May 11, 2017 | Laura and John Arnold Foundation, USA Funds | https://www.brookings.edu/research/how-is-policy-affecting-classroom-instruction/ | |
| 46 | Morgan Polikoff | Tenice Hardaway | Whereas most of the energy in the school choice debates has focused on vouchers and charter schools, relatively little attention has been paid to another important choice model that serves as many students as charters and has been in existence for longer—magnet schools. | Dismissive | Don't forget magnet schools when thinking about school choice, p.1 | Brookings Institution, Evidence Speaks Reports, Vol 2, #8, March 16, 2017 | Laura and John Arnold Foundation, USA Funds | https://www.brookings.edu/research/dont-forget-magnet-schools-when-thinking-about-school-choice/ | |
| 47 | Robert Pondiscio | "Six years after Common Core’s debut, these critics have produced enough books to collapse a sturdy bookshelf. Few of them make any earnest attempt to persuade readers to reject Common Core on its merits or lack thereof. Some barely take up the content of the standards at all. Instead, they mainly traffic in fear mongering and paranoid conspiracy theories about corporate greed." | Denigrating | Lessons on Common Core: Critical books offer more folly than wisdom | Education Next, January 5, 2017 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | https://www.educationnext.org/lessons-on-common-core-critical-books-pondiscio/ | ||
| 48 | Robert Pondiscio | "For Common Core’s excitable enemies, there is no such thing as overreach." | Denigrating | Lessons on Common Core: Critical books offer more folly than wisdom | Education Next, January 5, 2017 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | https://www.educationnext.org/lessons-on-common-core-critical-books-pondiscio/ | ||
| 49 | Robert Pondiscio | "Sadly, the paranoia that infuses the anti–Common Core literature is particularly prominent in books written by teachers. ... Schneider’s true intent is not to evaluate the standards but to expose the “power grab” behind education reform. The roundup of usual suspects includes Bill Gates, Arne Duncan, the testing company Pearson Education, and even the Fordham Institute." | Denigrating | Lessons on Common Core: Critical books offer more folly than wisdom | Education Next, January 5, 2017 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | https://www.educationnext.org/lessons-on-common-core-critical-books-pondiscio/ | ||
| 50 | Brian Jacob | Susan Dynarski, Kenneth Frank, Barbara Schneider | "A large body of research has found that increasing high school graduation requirements (whether in the form of additional courses or exit exams) results in lower graduation rates among the most disadvantaged students. There is less evidence on how such policies affect student achievement, but the existing research generally does not find large gains in student performance." p.2 | Dismissive | Are Expectations Alone Enough? Estimating the Effect of a Mandatory College-Prep Curriculum in Michigan | EPI Working Paper 01-2016, January, 2016 & NBER Working Paper No. 22013 | "The U.S. Department of Education’s Institute of Education Sciences provided generous support through Grant R305E100008." & NBER funders | http://edpolicy.umich.edu/files/01-2016_are-expectations-alone-enough.pdf | See, for example, https://www.tandfonline.com/doi/full/10.1080/15305058.2011.602920 https://nonpartisaneducation.org/Review/Resources/QuantitativeList.htm ; https://nonpartisaneducation.org/Review/Resources/SurveyList.htm ; https://nonpartisaneducation.org/Review/Resources/QualitativeList.htm |
| 51 | Brian Jacob | Susan Dynarski, Kenneth Frank, Barbara Schneider | "In addition, a small but growing body of research shows that taking certain core courses, especially those in math and science, can have significant, positive effects on long-term labor-market outcomes (Betts & Rose, 2004; Cortes, Goodman, & Nomi, 2015; Goodman, 2012; Levine & Zimmerman, 1995)." p.3 | Dismissive | Are Expectations Alone Enough? Estimating the Effect of a Mandatory College-Prep Curriculum in Michigan | EPI Working Paper 01-2016, January, 2016 & NBER Working Paper No. 22013 | "The U.S. Department of Education’s Institute of Education Sciences provided generous support through Grant R305E100008." & NBER funders | http://edpolicy.umich.edu/files/01-2016_are-expectations-alone-enough.pdf | See, for example, https://www.tandfonline.com/doi/full/10.1080/15305058.2011.602920 https://nonpartisaneducation.org/Review/Resources/QuantitativeList.htm ; https://nonpartisaneducation.org/Review/Resources/SurveyList.htm ; https://nonpartisaneducation.org/Review/Resources/QualitativeList.htm |
| 52 | Brian Jacob | Susan Dynarski, Kenneth Frank, Barbara Schneider | "Moreover, there is substantial evidence that high school exit exams, a closely related policy, increase drop-out rates, particularly among low-income students (Dee & Jacob, 2007; Jacob, 2001; Jenkins, Kulick, & Warren, 2006; Papay, Murnane, & Willett, 2010), and little evidence that they improve student achievement (Dee & Jacob, 2007; Grodsky, Kalogrides, & Warren, 2009)." p.4 | Dismissive | Are Expectations Alone Enough? Estimating the Effect of a Mandatory College-Prep Curriculum in Michigan | EPI Working Paper 01-2016, January, 2016 & NBER Working Paper No. 22013 | "The U.S. Department of Education’s Institute of Education Sciences provided generous support through Grant R305E100008." & NBER funders | http://edpolicy.umich.edu/files/01-2016_are-expectations-alone-enough.pdf | See, for example, https://www.tandfonline.com/doi/full/10.1080/15305058.2011.602920 https://nonpartisaneducation.org/Review/Resources/QuantitativeList.htm ; https://nonpartisaneducation.org/Review/Resources/SurveyList.htm ; https://nonpartisaneducation.org/Review/Resources/QualitativeList.htm |
| 53 | Brian Jacob | Susan Dynarski, Kenneth Frank, Barbara Schneider | "… with the exception of the working paper by Buddin and Croft (2014), this article is (to the best of our knowledge) the first analysis of a state attempt to require college-preparatory courses for all students." p.5 | 1stness | Are Expectations Alone Enough? Estimating the Effect of a Mandatory College-Prep Curriculum in Michigan | EPI Working Paper 01-2016, January, 2016 & NBER Working Paper No. 22013 | "The U.S. Department of Education’s Institute of Education Sciences provided generous support through Grant R305E100008." & NBER funders | http://edpolicy.umich.edu/files/01-2016_are-expectations-alone-enough.pdf | See, for example, https://www.tandfonline.com/doi/full/10.1080/15305058.2011.602920 https://nonpartisaneducation.org/Review/Resources/QuantitativeList.htm ; https://nonpartisaneducation.org/Review/Resources/SurveyList.htm ; https://nonpartisaneducation.org/Review/Resources/QualitativeList.htm |
| 54 | Susan M. Dynarski | Steven W. Hemelt, Joshua M. Hyman | NSC [National Student Clearinghouse] data are relatively new to academic researchers and policymakers. A growing number of papers make use of NSC data for research purposes (e.g., Bettinger, Long, Oreopoulos, & Sanbonmatsu, 2012; Chingos & Peterson, 2012; Deming, Hastings, Kane, & Staiger, 2014; Dynarski, Hyman, & Schanzenbach, 2013; Goldrick-Rab, Harris, Kelchen, & Benson, 2012; Hemelt, Roth, & Eaton, 2013; Hyman, 2013; Kane, 2003; Richburg-Hayes et al., 2009). | Dismissive | The Missing Manual: Using National Student Clearinghouse Data to Track Postsecondary Outcomes, p.54 | Educational Evaluation and Policy Analysis, May 2015, Vol. 37, No. 1S, pp. 53S–79S | Institute of Education Sciences, U.S. Department of Education, through Grants R305B110001 and R305E100008 to the University of Michigan, as well as through Grant R305C110011-11A to the Teachers College, Columbia University | http://journals.sagepub.com/doi/pdf/10.3102/0162373715576078 | |
| 55 | Susan M. Dynarski | Steven W. Hemelt, Joshua M. Hyman | NSC [National Student Clearinghouse] data are relatively new to academic researchers and policymakers. A growing number of papers make use of NSC data for research purposes (e.g., Bettinger, Long, Oreopoulos, & Sanbonmatsu, 2012; Chingos & Peterson, 2012; Deming, Hastings, Kane, & Staiger, 2014; Dynarski, Hyman, & Schanzenbach, 2013; Goldrick-Rab, Harris, Kelchen, & Benson, 2012; Hemelt, Roth, & Eaton, 2013; Hyman, 2013; Kane, 2003; Richburg-Hayes et al., 2009). | Dismissive | The Missing Manual: Using National Student Clearinghouse Data to Track Postsecondary Outcomes, p.54 | Educational Evaluation and Policy Analysis, May 2015, Vol. 37, No. 1S, pp. 53S–79S | Institute of Education Sciences, U.S. Department of Education, through Grants R305B110001 and R305E100008 to the University of Michigan, as well as through Grant R305C110011-11A to the Teachers College, Columbia University | http://journals.sagepub.com/doi/pdf/10.3102/0162373715576078 | |
| 56 | Ann Huff Stevens | Michal Kurlaender, Michel Grosz | "While career technical education (CTE) programs have often been mentioned as an attractive alternative to four-year colleges for some students, very little systematic evidence exists on the returns to specific vocational certificates and degrees." | Dismissive | Career technical education and labor market outcomes: Evidence from California Community Colleges | National Bureau of Economic Research, NBER Working Paper #21137, Issued in April 2015, ' | (1) NBER supporters | ||
| 57 | Daniel D. Goldhaber | "Empirical evidence now clearly buttresses intuition that teachers differ significantly from one another in terms of their impacts on student learning and shows that these differences have long-term consequences for students’ later academic (Goldhaber and Hansen, 2010; Jackson and Bruegmann, 2009; Jacob andLefgren, 2008; Kane and Staiger, 2008)." | Dismissive | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 58 | Daniel D. Goldhaber | "Empirical evidence now clearly buttresses intuition that teachers differ significantly from one another in terms of their impacts on student learning and shows that these differences have long-term consequences for students’ later academic (Goldhaber and Hansen, 2010; Jackson and Bruegmann, 2009; Jacob and Lefgren, 2008; Kane and Staiger, 2008)." | Dismissive | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 59 | Daniel D. Goldhaber | "Research on school leaders is far less extensive, ..." | Dismissive | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 60 | Daniel D. Goldhaber | "Yet, there is very little empirical evidence linking pre-service training to workforce outcomes (National Research Council, 2010)." | Dismissive | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 61 | Daniel D. Goldhaber | "However, we do not know the extent to which this finding reflects differences in potential teachers’ backgrounds (i.e., who is selected into a program or pathway) versus differences in potential educators’ experiences in programs." | Dismissive | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 62 | Daniel D. Goldhaber | "Only a few studies connect the features of teacher training to the outcomes of teachers in the field. That said, evidence is mounting that some types of pre-service teaching experiences and pedagogical coursework are associated with better teacher outcomes." | Dismissive | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 63 | Daniel D. Goldhaber | "To my knowledge, only one study connects principals’ training to student outcomes (Clark et al., 2009), and it doesn’t substantiate a relationship between the two." | Dismissive | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 64 | Daniel D. Goldhaber | "Taken together, studies like these begin to point toward ways to improve teacher preparation. But with such a thin evidentiary base, we are just beginning to understand what makes teacher preparation effective." | Dismissive | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 65 | Daniel D. Goldhaber | "Unfortunately, most research on PD is not terribly rigorous, and few studies suggest that it systematically improves teaching." | Denigrating | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 66 | Daniel D. Goldhaber | "Several large-scale, well-designed, federally funded experimental studies do tend to confirm that PD has little or mixed impacts on student achievement." | Denigrating | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 67 | Daniel D. Goldhaber | "Another way that policymakers have tried to improve educator effectiveness is by providing explicit incentives for teacher performance. Unfortunately, much of the highest quality randomized control trial evidence on this avenue of reform also suggests that it has limited impacts on student achievement (Yuan et al., 2013)." | Denigrating | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 68 | Daniel D. Goldhaber | "... we now pay far more policy and research attention to the effects of schools and educators on student learning – an outcome focus – rather than making judgments about the quality of education students receive." | Denigrating | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 69 | Daniel D. Goldhaber | "But we are now just on the cusp of learning about how these changes affect the quality of the educator workforce and sound policy must rest on such knowledge." | Denigrating | Written Statement before Full Committee Hearing: Fixing No Child Left Behind: Supporting Teachers and School Leaders | U.S. Senate, Committee on Health, Education, Labor, and Pensions, January 27, 2015 | CALDER funder, U.S. Institute for Education Sciences | |||
| 70 | Grover "Russ" Whitehurst | "...in 1999 the National Academies of Science came to the conclusion that: 'One striking fact is that the complex world of education - unlike defense, health care, orindustrial production - does not rest on a strong research base.'" | Dismissive | Testimony of Grover J. (Russ) Whitehurst | The U.S. Department of Education’s Institute of Education Sciences | ||||
| 71 | Grover "Russ" Whitehurst | "...in 1999 the National Academies of Science came to the conclusion that: '...In no other field are personal experience and ideology so frequently relied on to make policy choices, and in no other field is the research base so inadequate and little used.'" | Denigrating | Testimony of Grover J. (Russ) Whitehurst | The U.S. Department of Education’s Institute of Education Sciences | ||||
| 72 | Grover "Russ" Whitehurst | "Since the National Academies report and as a direct result of Education Sciences Reform Act we have seen considerable progress in the quality and relevance of education research." | Denigrating | Testimony of Grover J. (Russ) Whitehurst | The U.S. Department of Education’s Institute of Education Sciences | ||||
| 73 | Grover "Russ" Whitehurst | "We know much more about what works and what doesn’t in education than we did 15 years ago as a result of advances in research, but our level of ignorance dwarfs our understanding by orders of magnitude. It has been so in the early years of the transformation of other fields to evidence-based practice." | Denigrating | Testimony of Grover J. (Russ) Whitehurst | The U.S. Department of Education’s Institute of Education Sciences | ||||
| 74 | C.J. Libassi | "The research base on full-day kindergarten has been both sparse and, until now, exclusively contained to non-experimental studies." | Dismissive, Denigrating | Best Research Yet on the Effects of Full-Day Kindergarten | EdCentral, Dec. 5, 2014 | (4) New America funders | |||
| 75 | Grover "Russ" Whitehurst | "The takeaway from my testimony is that we know very little from research, or even from practical experience, that can inform the dozens of important decisions that should be on the table for government officials responsible for the design and implementation of early childhood programs and services." | Dismissive | Testimony of Grover J. (Russ) Whitehurst | The Health, Education, Labor, and Pensions Committee of the U.S. Senate, April 10, 2014 | The U.S. Department of Education’s Institute of Education Sciences | |||
| 76 | Grover "Russ" Whitehurst | "We should not to be hobbled by consensus views that arise largely in an evidence.free zone grounded on little more than high hopes. We need to acknowledge how much we don’t know…" | Denigrating | Testimony of Grover J. (Russ) Whitehurst | The Health, Education, Labor, and Pensions Committee of the U.S. Senate, April 10, 2014 | The U.S. Department of Education’s Institute of Education Sciences | |||
| 77 | Daniel D. Goldhaber | "There is relatively little quantitative research on the features of TPPs (Teacher Preparation Programs) that are associated with student achievement but what does exist offers suggestive evidence that some features may matter." p.3 | Dismissive | WHAT DO VALUE-ADDED MEASURES OF TEACHER PREPARATION PROGRAMS TELL US? | Carnegie
Knowledge Network, Knowledge Brief 12 November 2013 |
Carnegie Foundation for the Advancement of Teaching | |||
| 78 | Daniel D. Goldhaber | "As noted, there are a few studies that connect the features of teacher training to the effectiveness of teachers in the field, but this research is in its infancy." p.5 | Dismissive | WHAT DO VALUE-ADDED MEASURES OF TEACHER PREPARATION PROGRAMS TELL US? | Carnegie
Knowledge Network, Knowledge Brief 12 November 2013 |
Carnegie Foundation for the Advancement of Teaching | |||
| 79 | Daniel D. Goldhaber | "most of the evidence from the studies cited above is based on teaching at the elementary and middle school levels; we know very little about how graduates from different preparation programs compare at the high school level." p.5 | Dismissive | WHAT DO VALUE-ADDED MEASURES OF TEACHER PREPARATION PROGRAMS TELL US? | Carnegie
Knowledge Network, Knowledge Brief 12 November 2013 |
Carnegie Foundation for the Advancement of Teaching | |||
| 80 | Daniel D. Goldhaber | "But aside from research that shows a correlation between value-added and training effects across subjects, we do not know how much estimates of training effects from programs within an institution correspond with one another" p.5 | Dismissive | WHAT DO VALUE-ADDED MEASURES OF TEACHER PREPARATION PROGRAMS TELL US? | Carnegie
Knowledge Network, Knowledge Brief 12 November 2013 |
Carnegie Foundation for the Advancement of Teaching | |||
| 81 | Daniel D. Goldhaber | "we know even less about what goes on inside training programs, the criteria for recruitment and selection of candidates, and the features of training itself. The absence of this research is significant" p.5 | Dismissive | WHAT DO VALUE-ADDED MEASURES OF TEACHER PREPARATION PROGRAMS TELL US? | Carnegie
Knowledge Network, Knowledge Brief 12 November 2013 |
Carnegie Foundation for the Advancement of Teaching | |||
| 82 | Daniel D. Goldhaber | "Research, for instance, has just begun to assess the degree to which training programs or their particular features relate to outcomes as fundamental as the probability of a graduate’s getting a teaching job and of staying in the profession" p.6 | Dismissive | WHAT DO VALUE-ADDED MEASURES OF TEACHER PREPARATION PROGRAMS TELL US? | Carnegie
Knowledge Network, Knowledge Brief 12 November 2013 |
Carnegie Foundation for the Advancement of Teaching | |||
| 83 | Daniel D. Goldhaber | "It is surprising how little we know about the impact of TPPs on student outcomes given the important role these programs could play in determining who is selected into them and the nature of the training they receive." p.7 | Dismissive | WHAT DO VALUE-ADDED MEASURES OF TEACHER PREPARATION PROGRAMS TELL US? | Carnegie
Knowledge Network, Knowledge Brief 12 November 2013 |
Carnegie Foundation for the Advancement of Teaching | |||
| 84 | Eric S. Taylor | John H. Tyler | "...very little is known about how the availability of new information, or the experience of being evaluated, might change teacher effort and effectiveness." | Dismissive | Can teacher evaluation improve teaching? Evidence of systematic growth in the effectiveness of midcareer teachers | Education Next, FALL 2012 / VOL. 12, NO. 4 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/can-teacher-evaluation-improve-teaching/ | |
| 85 | Eric S. Taylor | John H. Tyler | "In short, there are good reasons to expect that well-designed teacher-evaluation programs could have a direct and lasting effect on individual teacher performance. To our knowledge, however, ours is the first study to test this hypothesis directly. | 1stness | Can teacher evaluation improve teaching? Evidence of systematic growth in the effectiveness of midcareer teachers | Education Next, FALL 2012 / VOL. 12, NO. 4 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/can-teacher-evaluation-improve-teaching/ | |
| 86 | Finley Edwards | "Despite this attention, there is little rigorous evidence directly linking school start times and academic performance." | Denigrating | Do schools begin too early? The effect of start times on student achievement | Education Next, Summer 2012 / VOL. 12, NO. 3 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/do-schools-begin-too-early/ | ||
| 87 | Sa Bui | Steven Craig, Scott Imberman | "Three million students in the United States are classified as gifted, yet little is known about the effectiveness of traditional gifted and talented (G&T) programs." | Dismissive | Poor results for high achievers: New evidence on the impact of gifted and talented programs | Education Next, Winter 2012 / Vol. 12, No. 1 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/poor-results-for-high-achievers/ | Many psychometricians work in the field of gifted testing. Indeed, some specialize in it, and have created a large, robust research literature. One can find much of it at web sites such as "Hoagie's Gifted" and those for the gifted education research centers such as: Belin-Blank (in Iowa); Josephson (in Nevada); Johns Hopkins Center for Talented Youth (in Maryland); and Duke University's Talent Identification Program. |
| 88 | Sa Bui | Steven Craig, Scott Imberman | "To our knowledge, no existing studies offer convincing evidence on the causal effect of G&T programs on student achievement. Our research begins to fill this gap with…" | Dismissive | Poor results for high achievers: New evidence on the impact of gifted and talented programs | Education Next, Winter 2012 / Vol. 12, No. 1 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/poor-results-for-high-achievers/ | Many psychometricians work in the field of gifted testing. Indeed, some specialize in it, and have created a large, robust research literature. One can find much of it at web sites such as "Hoagie's Gifted" and those for the gifted education research centers such as: Belin-Blank (in Iowa); Josephson (in Nevada); Johns Hopkins Center for Talented Youth (in Maryland); and Duke University's Talent Identification Program. |
| 89 | Thomas S. Dee | Brian A. Jacob | "In a recent review of this diverse evaluation literature, Figlio and Ladd (2008) suggest that three studies (Carnoy & Loeb, 2002; Jacob, 2005; Hanushek & Raymond, 2005) are the “most methodologically sound” (Ladd, 2007)." | Denigrating | The Impact of No Child Left Behind on Student Achievement | Journal of Public Policy Analysis & Management, 30(3), 418–446 (2011) | |||
| 90 | Daniel D. Goldhaber | Michael Hansen | "In spite of the popularity of these testing policies (all but three states require teachers to pass some kind of licensure test) and the increased emphasis of testing teachers under the No Child Left Behind Act of 2001 (U.S. Department of Education, 2006), surprisingly little empirical evidence is available about the predictive validity of teachers’ performance on these tests as an indicator of classroom effectiveness." | Dismissive | Race, Gender, and Teacher Testing: How Informative a Tool Is Teacher Licensure Testing?, p.219 | American Educational Research Journal, March 2010, Vol. 47, No. 1, pp. 218–251 | |||
| 91 | Daniel D. Goldhaber | Michael Hansen | "...there is relatively little empirical work linking teacher licensure test scores to student achievement." | Dismissive | Race, Gender, and Teacher Testing: How Informative a Tool Is Teacher Licensure Testing?, p.219 | American Educational Research Journal, March 2010, Vol. 47, No. 1, pp. 218–251 | |||
| 92 | Daniel D. Goldhaber | Michael Hansen | "Furthermore, no study that we are aware of has explicitly analyzed the relationship between teacher licensure test performance for specific populations of teachers." | Dismissive | Race, Gender, and Teacher Testing: How Informative a Tool Is Teacher Licensure Testing?, p.219 | American Educational Research Journal, March 2010, Vol. 47, No. 1, pp. 218–251 | |||
| 93 | Daniel D. Goldhaber | Michael Hansen | "Most prior studies implicitly assume that the information provided through these avenues is uniform across teachers, which may not necessarily be the case. | Denigrating | Race, Gender, and Teacher Testing: How Informative a Tool Is Teacher Licensure Testing?, p.220 | American Educational Research Journal, March 2010, Vol. 47, No. 1, pp. 218–251 | |||
| 94 | Daniel D. Goldhaber | Michael Hansen | "Three recent studies examining this relationship find a consistent, albeit small, relationship between teacher performance on licensure exams (the Praxis II tests used in North Carolina) and student achievement." | Dismissive | Race, Gender, and Teacher Testing: How Informative a Tool Is Teacher Licensure Testing?, p.221 | American Educational Research Journal, March 2010, Vol. 47, No. 1, pp. 218–251 | |||
| 95 | David E. Marcotte | Benjamin Hansen | "This is not to say that there is no interest in extending the school year. While there has been little solid evidence that doing so will improve learning outcomes, the idea is often endorsed." | Denigrating | Time for school? When the snow falls, test scores also drop | Education Next, Winter 2010, Vol. 10, No. 1 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/time-for-school/ | |
| 96 | David E. Marcotte | Benjamin Hansen | "...in 1994 included not one study on the impact of additional instruction on learning. Researchers at that time simply had little direct evidence to offer." | Dismissive | Time for school? When the snow falls, test scores also drop | Education Next, Winter 2010, Vol. 10, No. 1 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/time-for-school/ | |
| 97 | David E. Marcotte | Benjamin Hansen | "Among the first researchers to try to identify the impact of variation in instructional time were economists studying the effect of schooling on labor market outcomes such as earnings." | 1stness | Time for school? When the snow falls, test scores also drop | Education Next, Winter 2010, Vol. 10, No. 1 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/time-for-school/ | |
| 98 | Daniel D. Goldhaber | Matthew G. Springer, Ed. | "Although a number of districts have at times used alternatives to the single salary schedule, ...little is known about how to structure other plans to make them effective." p.35 | Dismissive | The Politics of Teacher Pay Reform | Chapter 2 in Performance Incentives: Their Growing Impact on American K-12 Education, Brookings Institution | Brookings Institution funders | ||
| 99 | Daniel D. Goldhaber | Matthew G. Springer, Ed. | " ...there are few credible quantitative studies on their effects ...the available evidence base from which to draw inferences about diferent pay reform designs remains thin." p.35 | Denigrating | The Politics of Teacher Pay Reform | Chapter 2 in Performance Incentives: Their Growing Impact on American K-12 Education, Brookings Institution | Brookings Institution funders | ||
| 100 | Daniel D. Goldhaber | Matthew G. Springer, Ed. | "Moreover, as discussed above, the lack of evidence on how pay reform affects students surely plays into the willingness of policymakers to engage in reform." p.35 | Dismissive | The Politics of Teacher Pay Reform | Chapter 2 in Performance Incentives: Their Growing Impact on American K-12 Education, Brookings Institution | Brookings Institution funders | ||
| 101 | Helen F. Ladd | "Given what a decade of research tells us about test-based accountability it seems reasonable to think about policy changes… Here is my vision...." | Dismissive | Education Week, January 23, 2008, p.27. | (3) Education Week funders | Relevant studies of the effects of tests and/or accountability program on motivation and instructional practice: Goslin (1967), *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | |||
| 102 | Daniel D. Goldhaber | "Few studies link principal attributes directly to student achievement,... This report includes new empirical research...." | 1stness | Principal compensation - More research needed on a promising reform | Center for American Progress, December 4, 2007 | (2) Center for American Progress funders | |||
| 103 | Cecelia Elena Rouse | Jane Hannaway, Dan Goldhaber, & David Figlio | "…there has been little attention paid to substantive changes in instructional policies and practices resulting from school accountability. The lack of research is primarily due to the unavailability of appropriate data to carry out such analysis. This paper brings to bear new evidence from a remarkable five-year survey..." | 1stness | Feeling the Florida heat? How low-performing schools respond to voucher and accountability pressures, Abstract | National Center for Analysis of Longitudinal Data in Educational Research (CALDER), Working Paper 13, November 2007 | US Education Department (USED) | Relevant studies of the effects of tests and/or accountability program on motivation and instructional practice: Goslin (1967), *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | |
| 104 | Cecelia Elena Rouse | Jane Hannaway, Dan Goldhaber, & David Figlio | "Surprisingly, there has been little systematic effort to determine the substantive ways in which schools alter their methods of delivering education in response to school accountability and school choice pressures (see Hannaway and Hamilton, 2007, for a review). | Dismissive | Feeling the Florida heat? How low-performing schools respond to voucher and accountability pressures, Abstract | National Center for Analysis of Longitudinal Data in Educational Research (CALDER), Working Paper 13, November 2007 | US Education Department (USED) | Relevant studies of the effects of tests and/or accountability program on motivation and instructional practice: Goslin (1967), *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | |
| 105 | David Steiner | "Daniel C. Humphrey and Marjorie E. Weschsler have stated the problem succinctly: We currently know very little about how a teacher candidate’s educational background, previous classroom experience, course work, clinical practice, mentoring and school placement interact to produce ateacher with the skills and knowledge to meet the academic needs of diverse students. (Humphrey and Weschsler, 2006)" | Dismissive | "Preparing Teachers to Teach the Liberal Arts" p.119 | Beyond the Basics: Achieving a Liberal Education for All Children, Chester E. Finn, Jr., and Diane Ravitch (Eds.), Thomas B. Fordham Institute, July 2007, pp.118–126 | (3) Fordham Institute funders | https://files.eric.ed.gov/fulltext/ED498008.pdf | ||
| 106 | David Steiner | "This problem would be far easier to tackle if we knew more about what makes a good teacher a good teacher." | Dismissive | "Preparing Teachers to Teach the Liberal Arts" p.119 | Beyond the Basics: Achieving a Liberal Education for All Children, Chester E. Finn, Jr., and Diane Ravitch (Eds.), Thomas B. Fordham Institute, July 2007, pp.118–126 | (3) Fordham Institute funders | https://files.eric.ed.gov/fulltext/ED498008.pdf | ||
| 107 | T.R. Stinebrickner | R. Stinebrickner | "Despite the large amount of attention that has been paid recently to understanding the determinants of educational outcomes, knowledge of the causal effect of the most fundamental input in the education production function - student study time and effort - has remained virtually non-existent. In this paper..." | Dismissive | The causal effect of studying on academic performance. | NBER Working Paper No. 13341, 2007 | (1) NBER supporters | ||
| 108 | Mike Petrilli | "... solid RFT [randomized field trials] studies exist in only a handful of areas in education—class-size reduction, early reading, vouchers ..." | Dismissive | The Key to Research Influence: Quality data and sound analysis matter, after all | Education Next, Spring 2007 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | https://www.educationnext.org/files/ednext_20072_77.pdf | ||
| 109 | Tom Loveless | "Scant research has methodically examined the politics of NCLB or marshaled empirical evidence to investigate support and opposition to the act at the state level." p.2 | Dismissive | The Peculiar Politics of No Child Left Behind | Brookings Report, August 1, 2006 | Brookings Institution funders | |||
| 110 | Daniel D. Goldhaber | "...there is relatively little empirical work linking teacher test scores to student achievement." | Dismissive | EVERYONE’S DOING IT, BUT WHAT DOES TEACHER TESTING TELL US ABOUT TEACHER EFFECTIVENESS?, p.2 | Center for Policy Research in Education (CPRE), U. Washington, Urban Institute | "The author gratefully acknowledges the Carnegie Corporation of New York and the Ewing Marion Kauffman Foundation for providing financial support for this project and the North Carolina Education Research Data Center and the Educational Testing Service for providing the confidential data." | |||
| 111 | Daniel D. Goldhaber | "In the absence of evidence about the relationship between these teacher tests and measures of teacher effectiveness in the classroom, there is no way to judge the extent to which states’ use of these tests allows ineffective teachers into the workforce or screens potentially effective teachers out of the workforce." | Dismissive | EVERYONE’S DOING IT, BUT WHAT DOES TEACHER TESTING TELL US ABOUT TEACHER EFFECTIVENESS?, p.2 | Center for Policy Research in Education (CPRE), U. Washington, Urban Institute | "The author gratefully acknowledges the Carnegie Corporation of New York and the Ewing Marion Kauffman Foundation for providing financial support for this project and the North Carolina Education Research Data Center and the Educational Testing Service for providing the confidential data." | |||
| 112 | Tom Loveless | "Most analysts date the standards and accountability movement to the early 1990s..." | Dismissive | Test-based Accountability: The Promise and the Perils, Chapter 1 in | Brookings Papers in Education Policy, 2005 | Brookings Institution funders | Relevant studies of the effects of tests and/or accountability program on motivation and instructional practice: Goslin (1967), *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | ||
| 113 | Tom Loveless | "Because many states did not have systems in place before 2000, studies of accountability are relatively recent." | 1stness | Test-based Accountability: The Promise and the Perils, Chapter 1 in | Brookings Papers in Education Policy, 2005 | Brookings Institution funders | Relevant studies of the effects of tests and/or accountability program on motivation and instructional practice: Goslin (1967), *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | ||
| 114 | Tom Loveless | "Scant empirical evidence has been produced to document the problem [of manipulating the pool of test-takers] nationally." | Dismissive | Test-based Accountability: The Promise and the Perils, Chapter 1 in | Brookings Papers in Education Policy, 2005 | Brookings Institution funders | |||
| 115 | Tom Loveless | "Few studies have surveyed large numbers of schools on how accountability shapes the school curriculum. Fewer yet have attempted to untangle the effects of testing in general from the effects of tests linked to accountability." | Dismissive | Test-based Accountability: The Promise and the Perils, Chapter 1 in | Brookings Papers in Education Policy, 2005 | Brookings Institution funders | Relevant studies of the effects of tests and/or accountability program on motivation and instructional practice: Goslin (1967), *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | ||
| 116 | Tom Loveless | "Studies that offer the best guidance on accountability policies are relatively recent." | Denigrating | Test-based Accountability: The Promise and the Perils, Chapter 1 in | Brookings Papers in Education Policy, 2005 | Brookings Institution funders | |||
| 117 | Tom Loveless | "Effective ways of boosting high school achievement have proved elusive. ... Whether the beneficial effects of accountability systems extend to subjects other than math or to students beyond the eighth grade is currently unknown." | Dismissive | Test-based Accountability: The Promise and the Perils, Chapter 1 in | Brookings Papers in Education Policy, 2005 | Brookings Institution funders | Relevant studies of the effects of tests and/or accountability program on motivation and instructional practice: Goslin (1967), *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | ||
| 118 | Tom Loveless | "With the exception of John Bishop, researchers also have not yet teased apart the impact of different types of accountability." | Dismissive | Test-based Accountability: The Promise and the Perils, Chapter 1 in | Brookings Papers in Education Policy, 2005 | Brookings Institution funders | Relevant studies of the effects of tests and/or accountability program on motivation and instructional practice: Goslin (1967), *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | ||
| 119 | Tom Loveless | "Research has only scratched the surface on assessing the independent effects of student and school accountability, and within those two categories, little work has been done ...." | Dismissive | Test-based Accountability: The Promise and the Perils, Chapter 1 in | Brookings Papers in Education Policy, 2005 | Brookings Institution funders | Relevant studies of the effects of tests and/or accountability program on motivation and instructional practice: Goslin (1967), *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | ||
| 120 | Jane Hannaway | Andrew J. Rotherham | "We then compared notes about what we knew from research about collective bargaining and who was pursuing the topic in an analytic way. Despite the work of a few well-known figures, the landscape was sparsely populated.", p.1 | Dismissive | Collective Bargaining in Education: Negotiating Change in Today’s Schools | Harvard Education Press, Cambridge, Massachusetts | |||
| 121 | ? | NYSED, FARU | "Nevertheless there is not a lot of information in the research literature on student perceptions of the effects of high-stakes tests on their motivations and attitudes." | Dismissive | The Impact of High-Stakes Exams on Students and Teachers, p.9 | New York State Education Department, Fiscal Analysis Research Unit | http://www.oms.nysed.gov/faru/TheImpactofHighStakesExams_files/The_Impact_of_High-Stakes_Exams.pdf | At least twelve pre-2004 student surveys were included here: https://www.nonpartisaneducation.org/Review/Resources/SurveyList.htm. See also: https://richardphelps.net/DemandForStandardizedTesting.pdf | |
| 122 | ? | NYSED, FARU | "Little systemic, national and comprehensive research of the change in teacher curriculum, instruction and other behaviors engendered by high-stakes testing has been conducted however." | Dismissive | The Impact of High-Stakes Exams on Students and Teachers, p.11 | New York State Education Department, Fiscal Analysis Research Unit | http://www.oms.nysed.gov/faru/TheImpactofHighStakesExams_files/The_Impact_of_High-Stakes_Exams.pdf | Nonsense. Just some of the relevant studies of the effects of tests and/or accountability program on motivation and instructional practice include those of the *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | |
| 123 | ? | NYSED, FARU | "Of the various questions that this policy brief attempts to answer, the question of the cost of implementing high-stakes high school exams has probably the weakest research base behind it. Few studies have examined this issue." | Dismissive, Denigrating | The Impact of High-Stakes Exams on Students and Teachers, p.13 | New York State Education Department, Fiscal Analysis Research Unit | http://www.oms.nysed.gov/faru/TheImpactofHighStakesExams_files/The_Impact_of_High-Stakes_Exams.pdf | No. See, for example, Phelps, R.P. (2000, Winter). Estimating the cost of systemwide student testing in the United States. Journal of Education Finance, 25(3) 343–380; Danitz, T. (2001, February 27). Special report: States pay $400 million for tests in 2001. Stateline.org. Pew Center for the States; Hoxby, C.M. (2002). The cost of accountability, in W. M Evers & H.J. Walberg (Eds.), School Accountability, Stanford, CA: Hoover Institution Press; U.S. GAO. (1993, January). Student testing: Current extent and expenditures, with cost estimates for a national examination. GAO/PEMD-93-8. Washington, DC: US General Accounting Office; Picus, L.O., & Tralli, A. (1998, February). Alternative assessment programs: What are the true costs? CSE Technical Report 441, Los Angeles: CRESST; Phelps, R.P. (1998). Benefit-cost analysis of systemwide student testing, Paper presented at the annual meeting of the American Education Finance Association, Mobile, AL. | |
| 124 | Manhattan Institute | Christian Bourge (journalist), interviewer | "But because there is little empirical evidence either way, the debate on this question has long featured arguments from both sides that are supported mostly by anecdotes." | Dismissive, Denigrating | "Experts Differ about High-stakes Testing," February 11, 2003. | UPI | https://www.upi.com/Top_News/2003/02/13/Experts-differ-about-high-stakes-testing/60271045180206/ | Nonsense. Just some of the relevant studies of the effects of tests and/or accountability program on motivation and instructional practice include those of the *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | |
| 125 | Tom Loveless | ""It is important to keep in mind the limited body of data on the subject." | Dismissive | quoted in “New Report Confirms,” February 11, 2003. | U.S. Congress: Committee on Education and the Workforce, news release | also: UPI, Feb. 13, 2003 "Experts differ about high-stakes testing" | https://www.upi.com/Top_News/2003/02/13/Experts-differ-about-high-stakes-testing/60271045180206/ | ||
| 126 | Tom Loveless | "We are just getting started in terms of solid research on standards, testing and accountability.” | Denigrating | quoted in “New Report Confirms,” February 11, 2003. | U.S. Congress: Committee on Education and the Workforce, news release | also: UPI, Feb. 13, 2003 "Experts differ about high-stakes testing" | https://www.upi.com/Top_News/2003/02/13/Experts-differ-about-high-stakes-testing/60271045180206/ | ||
| 127 | Daniel D. Goldhaber | “[R]elatively few studies that address students’ outcomes observe the professional norm of having detailed controls for students’ background characteristics (including previous academic achievement).” p. 53 | Denigrating | The Mystery of Good Teaching | Education Next, Spring 2002 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 128 | Daniel D. Goldhaber | “Hoover Institution senior fellow Eric Hanushek showed that only a small proportion of studies find these teacher characteristics to be statistically significant in the expected direction.” p. 53 | Dismissive | The Mystery of Good Teaching | Education Next, Spring 2002 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 129 | Daniel D. Goldhaber | “There are also statistical shortcomings in many of the studies cited by both Hanushek and Greenwald et al.” p. 53 | Denigrating | The Mystery of Good Teaching | Education Next, Spring 2002 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/files/ednext20021_50.pdf | ||
| 130 | Daniel D. Goldhaber | “But only four of these studies were based on students’ outcomes and most of them were more than 25 years old, which means they predated the ‘value added’ methodology of assessing educational effects that is now standard practice.” p. 54 | Denigrating | The Mystery of Good Teaching | Education Next, Spring 2002 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 131 | Daniel D. Goldhaber | “[T]here is little research directly assessing the influence of pedagogical training on student outcomes, . . . .” p. 54 | Dismissive | The Mystery of Good Teaching | Education Next, Spring 2002 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 132 | Daniel D. Goldhaber | “As Carolyn Evertson and her colleagues write, ‘Investigations of teacher education do not represent a strong body of research.’” p. 54 | Denigrating | The Mystery of Good Teaching | Education Next, Spring 2002 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | http://educationnext.org/files/ednext20021_50.pdf | ||
| 133 | Helen F. Ladd | "Given the current attention to educational accountability, surprisingly little is known about how accountability progams affect student achievement." p. 395 | Dismissive | School-Based Educational Accountability Systems: The Promise and the Pitfalls | National Tax Journal 54(2). May 2001 DOI: 10.17310/ntj.2001.2.08 | " commissioned by the Committee on Education Finance of the National Academy of Sciences." | Nonsense. Just some of the relevant studies of the effects of tests and/or accountability program on motivation and instructional practice include those of the *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | ||
| 134 | Helen F. Ladd | "Even if we do not know much about the impacts of school-based accountability systems on student achievement, we do have evidence from North Carolina ... This evidence comes from a recent study of the responses of elementary school principals ... (Ladd and Zelli 2001)" p.396 | Dismissive | School-Based Educational Accountability Systems: The Promise and the Pitfalls | National Tax Journal 54(2). May 2001 DOI: 10.17310/ntj.2001.2.08 | " commissioned by the Committee on Education Finance of the National Academy of Sciences." | Nonsense. Just some of the relevant studies of the effects of tests and/or accountability program on motivation and instructional practice include those of the *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | ||
| 135 | Henry Levin | in G. Orfield & M. Kornhaber, (Eds. | “…disinterested appraisals of the research on the predictive validity of test scores conclude that there is only a very modest connection between test scores and productivity ratings by supervisors. Indeed, an overall summary of the potential economic gains from using test scores for employment selection suggests that the economic claims of industrial psychologists are flawed and highly exaggerated.” | Denigrating | High-stakes testing and economic productivity | Raising standards or raising barriers? Inequality and high-stakes testing in public education, New York: Century Foundation | |||
| 136 | Dan Goldhaber | "There is really no quantitative evidence to help answer these questions [on "the supply side effects of private schooling"]." p.30 | Dismissive | The Interface between Public and Private Schooling: Market Pressure and the Impact on Performance | National Invitational Conference on Improving Educational Productivity: Lessons from Economics | Laboratory for Student Success, Temple U. | |||
| 137 | Dan Goldhaber | "There is also relatively little evidence on the impact that vouchers might have on funding for public schools." p.30 | Dismissive | The Interface between Public and Private Schooling: Market Pressure and the Impact on Performance | National Invitational Conference on Improving Educational Productivity: Lessons from Economics | Laboratory for Student Success, Temple U. | |||
| 138 | Julian Betts | Jeff George | "Despite recent theoretical work and proposals from educational reformers, there is little empirical work on the effects of higher grading standards. In this paper…" | Dismissive | The impact of grading standards on student achievement, educational attainment, and entry-level earnings, abstract | National Bureau of Economic Research, Working Paper 7875 | This research was supported by a grant from the American Educational Research Association which receives funds for its "AERA Grants Program" from the National Science Foundation and the National Center for Education Sttistics under NSF Grant #RED-9452861. | http://www.nber.org/papers/w7875 | See a review of
hundreds of studies: Brookhart et al.
(2016) A Century of Grading Research: A Century of Grading Research: Meaning
and Value in the Most Common Educational Measure. Brookhart, S. M., Guskey,
T. R., Bowers, A. J., McMillan, J. H., Smith, J. K., Smith, L. F., Stevens,
M.T., Welsh, M. E. (2016). A Century of Grading Research: Meaning and Value
in the Most Common Educational Measure. Review of Educational Research,
86(4), 803-848. doi: 10.3102/0034654316672069 http://doi.org/10.3102/0034654316672069 |
| 139 | Julian Betts | Jeff George | "...it is surprising how little empirical work has been devoted to understanding how other aspects of the educational environment affect student behavior. In particular, given economists’ general interests in incentive schemes, it is surprising how little empirical work has focused on educational incentives." | Dismissive | The impact of grading standards on student achievement, educational attainment, and entry-level earnings, p.1 | National Bureau of Economic Research, Working Paper 7875 | This research was supported by a grant from the American Educational Research Association which receives funds for its "AERA Grants Program" from the National Science Foundation and the National Center for Education Sttistics under NSF Grant #RED-9452861. | http://www.nber.org/papers/w7875 | See
a review of hundreds of studies:
Brookhart et al. (2016) A Century of Grading Research: A Century of
Grading Research: Meaning and Value in the Most Common Educational Measure.
Brookhart, S. M., Guskey, T. R., Bowers, A. J., McMillan, J. H., Smith, J.
K., Smith, L. F., Stevens, M.T., Welsh, M. E. (2016). A Century of Grading
Research: Meaning and Value in the Most Common Educational Measure. Review of
Educational Research, 86(4), 803-848. doi: 10.3102/0034654316672069 http://doi.org/10.3102/0034654316672069 |
| 140 | Julian Betts | Jeff George | "The only other empirical study that we know of that addresses similar questions is Lillard and DeCicca (forthcoming)." | Dismissive | The impact of grading standards on student achievement, educational attainment, and entry-level earnings, p.1 | National Bureau of Economic Research, Working Paper 7875 | This research was supported by a grant from the American Educational Research Association which receives funds for its "AERA Grants Program" from the National Science Foundation and the National Center for Education Sttistics under NSF Grant #RED-9452861. | http://www.nber.org/papers/w7875 | See
a review of hundreds of studies:
Brookhart et al. (2016) A Century of Grading Research: A Century of
Grading Research: Meaning and Value in the Most Common Educational Measure.
Brookhart, S. M., Guskey, T. R., Bowers, A. J., McMillan, J. H., Smith, J.
K., Smith, L. F., Stevens, M.T., Welsh, M. E. (2016). A Century of Grading
Research: Meaning and Value in the Most Common Educational Measure. Review of
Educational Research, 86(4), 803-848. doi: 10.3102/0034654316672069 http://doi.org/10.3102/0034654316672069 |
| 141 | Helen F. Ladd, Thomas Sobol, Janet S. Hansen | "To our knowledge, these are the only systematic studies of the effect of [accountability] programs on achievement." | Dismissive | Making Money Matter: Financing America's Schools, p.183 | National Research Council, Committee on Education Finance, 1999 | US Education Department (USED) | Nonsense. Just some of the relevant studies of the effects of tests and/or accountability program on motivation and instructional practice include those of the *Southern Regional Education Board (1998); Johnson (1998); Schafer, Hultgren, Hawley, Abrams Seubert & Mazzoni (1997); Miles, Bishop, Collins, Fink, Gardner, Grant, Hussain, et al. (1997); Tuckman & Trimble (1997); Clarke & Stephens (1996); Zigarelli (1996); Stevenson, Lee, et al. (1995); Waters, Burger & Burger (1995); Egeland (1995); Prais (1995); Tuckman (1994); Ritchie & Thorkildsen (1994); Brown & Walberg, (1993); Wall & Alderson (1993); Wolf & Rapiau (1993); Eckstein & Noah (1993); Chao-Qun & Hui (1993); Plazak & Mazur (1992); Steedman (1992); Singh, Marimutha & Mukjerjee (1990); *Levine & Lezotte (1990); O’Sullivan (1989); Somerset (1988); Pennycuick & Murphy (1988); Stevens (1984); Marsh (1984); Brunton (1982); Solberg (1977); Foss (1977); *Kirkland (1971); Somerset (1968); Stuit (1947); and Keys (1934). *Covers many studies; study is a research review, research synthesis, or meta-analysis. | ||
| 142 | Helen F. Ladd | "Given the widespread interest in school-based recognition and reward programs, it is surprising how little evaluation has been done of their impacts." | Dismissive | The Dallas School Accountability and Incentive Program: An Evaluation of its Impacts on Student Outcomes, p.1 | Economics of Education Review, 1999 | This paper is part of a larger project on performance based accountability that the author initiated as a Visiting Fellow at the Brookings Institution. She thanks the Brookings Institution and its Brown Center for Education Policy for financial and collegial support. In addition, she is indebted to the Spencer Foundation and an anonymous donor for financial support of the larger project. | |||
| 143 | Helen F. Ladd | "...several states and a few local districts have introduced school-based incentive programs. This paper provides one of the few evaluations of the effects of such programs on student outcomes." | 1stness | The Dallas School Accountability and Incentive Program: An Evaluation of its Impacts on Student Outcomes, p.1 | Economics of Education Review, 1999 | This paper is part of a larger project on performance based accountability that the author initiated as a Visiting Fellow at the Brookings Institution. She thanks the Brookings Institution and its Brown Center for Education Policy for financial and collegial support. In addition, she is indebted to the Spencer Foundation and an anonymous donor for financial support of the larger project. | |||
| 144 | Daniel D. Goldhaber | Dominic J. Brewer | “ There are good reasons to believe that many educational production function studies, particularly those completed in the 1970s, had major deficiencies in empirical methodology and available data.” p. 4 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 145 | Daniel D. Goldhaber | Dominic J. Brewer | “ For example, many early studies were unable to control for prior achievement using a "pre-test" score to net out individual ability, as is now generally accepted to be important (Boardman and Murnane, 1979; Hanushek, 1979; Hedges et al., 1994).” p. 5 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 146 | Daniel D. Goldhaber | Dominic J. Brewer | “Variables representing school and teacher ‘quality’ that are used in most production function studies are typically very crude.” p. 5 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 147 | Daniel D. Goldhaber | Dominic J. Brewer | “The few studies which have had measures of teacher (verbal) ability, for example in the form of a test score, have found a much more robust positive relationship to student achievement . . . than those using other teacher characteristics.” pp. 5-6 | Dismissive | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 148 | Daniel D. Goldhaber | Dominic J. Brewer | “ With a few exceptions most educational production function studies use simple linear ordinary least squares (OLS) regression” p. 5, note 7 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 149 | Daniel D. Goldhaber | Dominic J. Brewer | “Data deficiencies may also have led to significant measurement error problems in previous studies.” p. 6 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 150 | Daniel D. Goldhaber | Dominic J. Brewer | “Of particular concern is that ‘teacher characteristics’ used in many studies are not in fact those of any individual teacher but rather a school level variable such as mean years of experience, percentage of teachers with at least a Masters [sic] degree, or mean teacher salary.” p. 6 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 151 | Daniel D. Goldhaber | Dominic J. Brewer | “Before adding teacher behavior variables it is worth considering more closely the teacher characteristic results which contrast with many earlier studies which have found little relationship between such variables and student achievement” p. 14 | Dismissive | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 152 | Daniel D. Goldhaber | Dominic J. Brewer | “Few attempts have been made to utilize such concepts in multivariate analysis. . . . ” p. 15 | Dismissive | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 153 | Daniel D. Goldhaber | Dominic J. Brewer | “This link enables us to avoid problems with aggregation that may have plagued earlier studies.” `p. 20 | Denigrating | Why Don't Schools and Teachers Seem to Matter? Assessing the Impact of Unobservables on Educational Productivity | Revision of paper presented at meetings of the Econometric Society (San Francisco, Calif.), January 1996 | http://files.eric.ed.gov/fulltext/ED400237.pdf | ||
| 154 | Daniel D. Goldhaber | Ronald G. Ehrenberg, Dominic J. Brewer | “Most of this research [on the effectiveness of minority teachers in educating minority students] has not addressed the students' educational outcomes; has failed to control for other teacher characteristics, such as verbal ability, experience, and degree levels; and has not investigated the effects that under-represented minority teachers have on non-minority students.” p. 548 | Denigrating | Do Teachers’ Race, Gender, and Ethnicity Matter? Evidence From the National Education Longitudinal Study of 1988 | Industrial and Labor Relations Review, Vol. 48, No. 3 (April 1995) | http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1626&context=articles | ||
| 155 | Daniel D. Goldhaber | Ronald G. Ehrenberg, Dominic J. Brewer | “The few studies that do address outcomes focus on the correlation between teacher gender and students' test scores at a point in time, “ p. 548 | Denigrating | Do Teachers’ Race, Gender, and Ethnicity Matter? Evidence From the National Education Longitudinal Study of 1988 | Industrial and Labor Relations Review, Vol. 48, No. 3 (April 1995) | http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1626&context=articles | ||
| 156 | Daniel D. Goldhaber | Ronald G. Ehrenberg, Dominic J. Brewer | “In contrast to the previous literature, we focus both on how teachers subjectively relate to and evaluate their students and on how much their students learn, as measured by standardized tests.” p. 548 | 1stness | Do Teachers’ Race, Gender, and Ethnicity Matter? Evidence From the National Education Longitudinal Study of 1988 | Industrial and Labor Relations Review, Vol. 48, No. 3 (April 1995) | http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1626&context=articles | ||
| 157 | 1 | Ronald G. Ehrenberg, Dominic J. Brewer | “[N]o evidence exists, however, on whether that RGE match influences either how well employees perform or the level of their earnings.” p. 548 | Dismissive | Do Teachers’ Race, Gender, and Ethnicity Matter? Evidence From the National Education Longitudinal Study of 1988 | Industrial and Labor Relations Review, Vol. 48, No. 3 (April 1995) | http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1626&context=articles | ||
| 158 | Daniel D. Goldhaber | Ronald G. Ehrenberg, Dominic J. Brewer | “As in the research on teachers and students, however, there is virtually no evidence on whether the degree of correspondence in RGE between supervisors and employees influences how well employees perform. Similarly, there is no evidence on whether the match, or lack thereof, between supervisor and employee characteristics influences an employee's long-run earnings and productivity at a firm.” p. 560 | Dismissive | Do Teachers’ Race, Gender, and Ethnicity Matter? Evidence From the National Education Longitudinal Study of 1988 | Industrial and Labor Relations Review, Vol. 48, No. 3 (April 1995) | http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1626&context=articles | ||
| IRONIES: | |||||||||
| Frederick M. Hess | Michael J. Petrilli, Martin R. West | "Instead of more cheerleading, what is desparately needed is more humility.", p. 65 | "Taking stock of a decade of reform" | Education Next, Spring 2011 | Harvard Kennedy School; Thomas B. Fordham Foundation and Institute | ||||
| Daniel D. Goldhaber | "... any debate about the use of test scores in educational accountability (1) should be framed by use of all relevant empirical evidence ..." abstract | How Much Should We Rely on Student Test Achievement as a Measure of Success? | Educational Researcher, September 6, 2019 | https://journals.sagepub.com/doi/abs/10.3102/0013189X19874061?journalCode=edra | |||||
| Frederick M. Hess | "… would-be reformers err when they presume to know a lot more than they do." | Twitter tweet Feb 12, 2018 | |||||||
| Michael Petrilli | "Hey Team EWA: Please consider putting this on your list serve. It might make for an uncomfortable but important conversation." in reference to National Review article, "We’re Plagued by a Partisan Press. Here’s One Cure: Bring Idealogical Diversity to the Newsroom" | Twitter tweet Jan 25, 2019 | |||||||
| Michael Petrilli | "we need to be careful of lapsing into morality plays. We need to be particularly mindful of not villainizing our opponents. And we need to be humble enough to acknowledge the technical challenges in what we’re trying to achieve." | Reformers’ push to improve education is worthy of celebration — but is there a dark side, too? | Hechinger Report, Nov.10, 2015 | Hechinger Report funders | |||||
| Michael Petrilli | "what if the reform movement needs more “science” and less “religion”? More openness to trial-and-error and a greater commitment to using evidence to guide our decisions?" | Reformers’ push to improve education is worthy of celebration — but is there a dark side, too? | Hechinger Report, Nov.10, 2015 | Hechinger Report funders | |||||
| Michael Petrilli | "It’s always been a good idea for us to check our egos at the door. Let’s check our halos there, too." | Reformers’ push to improve education is worthy of celebration — but is there a dark side, too? | Hechinger Report, Nov.10, 2015 | Hechinger Report funders | |||||
| Author cites (and accepts as fact without checking) someone elses dismissive review | |||||||||
| Cite selves or colleagues in the group, but dismiss or denigrate all other work | |||||||||
| Falsely claim that research has only recently been done on topic. | |||||||||
| (1) [as of January, 2019] Some NBER funders [https://www.nber.org/CorporateSupporters2018.pdf] Contributing $20,000 - $25,000: AIG; Bank for International Settlements; Brevan Howard; Capital Group Companies; ExxonMobil; Fidelity Management & Research; General Motors Foundation; Goldman Sachs; Google, Inc.; Johnson & Johnson; JP Morgan Chase Institute; Koret Foundation; Pfizer, Inc.; Vanguard; Anonymous (2). Contributing $10,000 - $19,999: Fuller & Thaler Asset Management; Insurance Information Institute. Contributing $5,000 - $9,999: Central Bank Research Associates; Norges Bank Investment Management. Contributing Less Than $5,000: Allen Sinai; Board of Governors of Federal Reserve System; Federal Reserve District Banks (12); Francis Schott. Contributions to Support the NBER Summer Institute: Contributing $50,000 to $75,000: Mohamed El-Erian. Contributing $10,000 to $19,999: Bank of England; Bank of France; Bank of Germany; Bank of Italy; Bank of Japan; Bank of Netherlands; Monetary Authority of Singapore; Reserve Bank of India. Contributions to Support the NBER Digitization Initiative: Contributing $25,000 - $50,000: Amazon; Tides Foundation. | |||||||||
| (2) [as of January, 2019] Center for American Progress funders [https://www.americanprogress.org/about/c3-our-supporters/] $1,000,000 or more: Anonymous (5); Democracy Forward; Fidelity Charitable Gift Fund; Ford Foundation; Bill & Melinda Gates Foundation; The William and Flora Hewlett Foundation; The Hutchins Family Foundation; W.K. Kellogg Foundation; Jonathan and Jeannie Lavine Family Fund; Open Society Foundations; Sandler Foundation; TomKat Charitable Trust. $500,000 to $999,999: Anonymous (2); Carnegie Corporation of New York; Embassy of the United Arab Emirates; Amy P. Goldman Foundation; Joyce Foundation; John D. and Catherine T. MacArthur Foundation; National Philanthropic Trust; The David and Lucile Packard Foundation; The Rockefeller Foundation; Siegel Family Endowment; Silicon Valley Community Foundation; Walton Family Foundation. $100,000 to $499,999: Anonymous (4); American Federation of State, County and Municipal Employees (AFSCME); Apple Inc.; The Arcus Foundation; Stewart Bainum Jr.; Bloomberg Philanthropies; Paul Boskind; William K. Bowes, Jr. Foundation; The California Endowment; Annie E. Casey Foundation; Consolidated Contractors Company; Quinn Delaney and Wayne Jordan; Blair Effron; Paul & Joanne Egerman Family Charitable Foundation; Dr. Anita Friedman; First Five Years Fund; Foundation for the Greatest Good; Mark Gallogly and Lise Strickler; Gill Foundation; Goldman-Sonnenfeldt Foundation; H&R Block; Hagedorn Foundation; Irving Harris Foundation; Heising-Simons Foundation; HR&A Advisors; The Kendeda Fund; The Kresge Foundation; Lumina Foundation; Mai Family Foundation; Microsoft Corporation; Eric Mindich; New Venture Fund; New York Community Trust; Open Philanthropy Project; Robert W. Roche; Robert E. Rubin; Schlosstein-Hartley Family Foundation; Stephen M. Silberstein Foundation; Stiftung Mercator; Vanguard Charitable Endowment Program; The WhyNot Initiative. $50,000 to $99,999: Anonymous (5); 444S Foundation; Robert Abernethy; American Federation of Teachers (AFT); William and Bonnie Apfelbaum; AT&T; Bank of America; The Bauman Foundation; Blackstone; Campion Foundation; CareFirst BlueCross BlueShield; Coalition for Public Safety; Common Counsel Foundation; Covanta; Embassy of Japan; Marc Fasteau and Anne G. Fredericks; Google; Sanjay Govil; Evelyn & Walter Haas, Jr. Fund; The Nick and Leslie Hanauer Foundation; Fred P. Hochberg and Thomas P. Healy; James Hormel; Infinite Computer Solutions Inc.; Tony James; Johnson Family Foundation; Altman Kazickas Foundation; LaSalle Adams Fund; Dale P. Mathias; Rebecca and Nathan Milikowsky; Ken Miller and Lybess Sweezy; Rockefeller Family Fund; Schwab Charitable Fund; Taipei Economic and Cultural Representative Office in the United States, or TECRO; United Minds for Progress; Henry van Ameringen; Jon F. Vein; Wallace Global Fund; Walmart; Wilburforce Foundation. $5,000 to $49,999: Anonymous (14); A. L. Mailman Family Foundation; Wendy and Jim Abrams; ADARA Charitable Fund; Madeleine K. Albright; The Albright Stonebridge Group; American Association for Justice (AAJ); American Beverage Association; The American Express Company; Greg and Anne Avis; Bank of America Charitable Gift Fund; B.W. Bastian Foundation; Nina Beattie and Michael Eberstadt; Carol and Frank Biondi; Adam Blumenthal; Brownstein Hyatt Farber Schreck, LLP; C.J.L. Charitable Foundation; California Community Foundation; James Capalino; Capricorn Management, LLC; Chan Zuckerberg Initiative; Dana Chasin; Simon Clark; Steven Cohen; David Colden; Combined Federal Campaign; Connecticut Street Foundation; CVS Health; Raj Date; Defenders of Wildlife; Discovery Communications; Eileen Donahoe; East Bay Community Foundation; Charles Leonard Egan; Elmo Foundation; Everytown for Gun Safety Action Fund; Express Scripts; Facebook; Federal Foreign Office of the Federal Republic of Germany; Joseph and Marie Field Foundation; Geoffrey Garin; General Electric; Heinrich Böll Foundation; Ann and Gordon Getty Foundation; Lisa and Douglas Goldman Fund; William Goldman; Joshua Greer; Garrett Gruener and Amy Slater Family Fund; Estate of Vincent Gulisano; Margaret and Shashi Gupta; Craig and Kathryn Hall; Heinrich-Böll-Stiftung North America; Joe Henderson; The Heyday Foundation; Belle Horwitz and Jonathan Weiner; Institute of International Education; Invariant; Joan and Irwin Jacobs; Japan Bank for International Cooperation; Jewish Community Foundation; Michael W. Kempner; Ed Kissam; Lebowitz-Aberly Family Foundation; Lefkofsky Family Foundation; Leonardo DRS; Damon & Heidi Lindelof; Hani Masri; Master Your Card; James Mauch; McLarty Associates; The Herbert McLaughlin Children’s Trust; Al Mottur; Kristin Mugford; The Philip and Tammy Murphy Family Foundation; Nicole Mutchnik; MWW; Shekar Narasimhan; New Silk Route Advisors LP; Joyce Newstat and Susan Lowenberg; NVG LLC; Peter Orszag; Pacific Gas and Electric Company; Alan Patricof; Pearson Education; PepsiCo Inc.; Anne Peretz; Peter G. Peterson Foundation; Andrew Pincus; Heather Podesta; Portlight Inclusive Disaster Strategies; The Pritzker Children’s Initiative; Quest Diagnostics; Deepak Raj; Steven Rattner and Maureen White; Robert Raymar; Francene and Charles Rodgers; Marti and Greg Rosenbaum; Laura Ross; Samsung; Parag Saxena; Alan & Susan Lewis Solomont Family Foundation; The Summers/New Family; Temasek; The Travelers Indemnity Company; Trehan Foundation Inc.; Tom and Janet Unterman; Philippe and Katherine Villers; Jeffrey C. Walker; Hope Warschaw; Herbert S. Winokur Jr; Robert Wolf. | |||||||||
| (3) [as of January, 2019] Education Week funders [https://www.edweek.org/info/about/philanthropy.html] "Our Funders": Bill & Melinda Gates Foundation; Carnegie Corporation of New York; Eli and Edythe Broad Foundation; Leona M. and Harry B. Helmsley Charitable Trust; Jack Kent Cooke Foundation; Joyce Foundation; Charles Stewart Mott Foundation; NoVo Foundation; Noyce Foundation; Raikes Foundation; Schott Foundation for Public Education; Wallace Foundation; Walton Family Foundation. | |||||||||
| (4) (as of 3rd quarter, 2018) New America funders [https://www.newamerica.org/our-funding/] (counting only those divisions with education or learning in their titles): Alliance for Early Success; Annie E. Casey Foundation; Ballmer Group; Bill and Melinda Gates Foundation; Brookings Institution; Charles Stewart Mott Foundation;Citi Foundation; Daniel B. Solomon; David and Lucile Packard Foundation; Education First; Foundation for Child Development; George Wasserman Family Foundation; Heising-Simons Foundation; Institute for Higher Education Policy; Jacobs Foundation; Joan Ganz Cooney Center; Joyce Foundation; JPB Foundation; JPMorgan Chase; Karen Alden; Kresge Foundation; Laura and John Arnold Foundation; Lumina Foundation; Mark Levin; New Venture Fund/Trust for Learning; Siemens Foundation; uAspire; W. Clement & Jessie V. Stone Foundation; Walton Family Foundation; William and Flora Hewlett Foundation. | |||||||||
| (5) Some StandardsWork funders over the years: Skillman Foundation; USED Office of Career and Technical Education; National Endowment for the Humanities; US Department of Defense Advanced Distributed Learning (ADL) Initiative; Texas Education Agency; DC Public Schools; National Assessment Governing Board. | |||||||||
| (6) [as of January, 2019] Fordham funders: Recent Funders* ( https://edexcellence.net/about-us/funding-and-finances.html ): American Federation for Children; Bloomberg Philanthropies; Bill and Melinda Gates Foundation; Carnegie Corporation of New York; Charles and Helen Schwab Foundation; Charles and Lynn Schusterman Foundation;; Collaborative for Student Success; Doris and Donald Fisher Fund; Exxon Mobil Corporation; Hastings Education Fund; Jack Kent Cooke Foundation; JPMorgan Chase Foundation; Kern Family Foundation; Leona B. and Harry M. Helmsley Trust; National Alliance for Public Charter Schools; Nord Family Foundation; Roger and Susan Hertog; Smith Richardson Foundation; Strada Education Network; The Achelis and Bodman Foundation; The Bernard Lee Schwartz Foundation; The Eli and Edythe Broad Foundation; The George Gund Foundation; The Joyce Foundation; The Kovner Foundation; The Louis Calder Foundation; The Lovett & Ruth Peters Foundation; The Lynch Foundation; Walton Family Foundation; William and Flora Hewlett Foundation; William E. Simon Foundation; * Does not include individual funders | |||||||||
| (7) [as of January, 2019] See https://www.the74million.org/supporters/ "Partners" include: Triad Foundation, The Eli and Edythe Broad Foundation, Park Avenue Charitable Trust, Bill & Melinda Gates Foundation, Carnegie Corporation of New York, Charles and Lynn Schusterman Family Foundation, Bloomberg Philanthropies, California Community Foundation, Doris & Donald Fisher Fund, Gen Next Foundation, Karsh Family Foundation, Jon Sackler, William E. Simon Foundation, Charles Strauch, Walton Family Foundation. | |||||||||