Nonpartisan Education Review /Essays: Volume 2 Number 5
Access this article in pdf format
An Analysis of Merit Pay Reforms in Educational Institutions
Jared Honeycutt, Steven Loomis, and Andrew Brulle
With roots in behaviorist philosophy, performance pay for teachers is often linked to accountability regimes in school reform. The theory girding such programs suggests that pay as an economic incentive can help cause teachers to increase student outcomes as measured by standardized test scores. What is little noticed by many educationists, but particularly by policy makers, is how programmatic effects affect the ontology of educational environment. There are several ways to approach the viability of such programs. In this study of three pay-for-performance programs, two in the U.S. and one in the UK, we provide theoretic insights in light of three variables: (i) their psychological framework, (ii) teacher efficacy and the teacher-student relationship, and (iii) how the psychological impact of such programs coincides with larger institutional forces. Using theory to examine pay-for-performance is necessary in order to get beneath mere data and secure more thorough understandings of the phenomenological impacts of performance pay. And better understanding of these foundational features is necessary, even critical, in order to fully appreciate the economic and informational trade-offs in implementation. Our study suggests that as a small-scale reform measure and when it specifically accounts for complexities of educational production, performance pay may be a viable reform option.
Performance pay is often touted as a systemic solution that will institute accountability for teachers, with the goal of ensuring student learning. Historically, many attempts have been made to develop a system of performance pay, particularly recently amidst high-profile use of standardized measures of student achievement. Performance pay reforms have been proposed and implemented since at least the 1850s (Holt, 2001). In the 1980s, a variety of incentive plans were implemented by state and local school agencies in order to improve the attractiveness and quality of careers in teaching, retain the best teachers and theoretically improve teacher performance and effectiveness (CTAC, 2004).
One's initial impression might be that performance pay is a monolithic reform strategy. Actually, performance pay is a nebulous term that garners much of its rhetorical weight from the popularized form's simple logic. The common version espoused by politicians is to fasten compensation for teachers to their students' achievement, generally as defined by student performance on standardized tests. A general statement of this logical relationship—that teachers are responsible for their students' learning—is a useful, self-evidently moral conclusion to draw. David Cohen (1996) examined the assumptions of this proposed policy initiative, noting that “much of the appeal of performance rewards lies in the idea that they would be a relatively simple and cheap solution to a complex and difficult problem” (p.62). However, as David Labaree (2004) has noted, dangers lurk in stating this relationship of accountability so simply.
Pay for performance is promoted as initiating fundamental changes in the classroom that will lead to increased attainment of outcomes. This paper will seek to analyze the integration of teacher compensation and student achievement (defined variously) from several grounds. We will examine the psychological framework underpinning some approaches to performance pay, both regarding the professional teacher and his or her sense of efficacy, and upon student-teacher interaction. The psychological impact of the reform coincides with larger, related issues of organizational structure and the sociological setting within which schools operate.
From the discussion of the reform's possible psychological outcomes (a common source of criticism against performance pay), we will move to an investigation of several performance pay programs that have already been implemented. From amongst the vast literature, we chose three evaluations of performance pay for review: 1) the report from the recent Denver Pay for Performance (PFP) pilot (CTAC, 2004); 2) an analysis of the professional climate surrounding the United Kingdom's recent performance management initiative (Storey, 2000); and 3) a study of the Tennessee Career Ladder Evaluation System and the Project STAR class-size experiment (Dee and Keys, 2003).
We will draw conclusions regarding these insights into performance pay and compare them to what we consider a more appropriate understanding of accountability and, subsequently, how performance pay might function in a limited role in aligning organizational goals with the bureaucratic structure. It will become apparent through our study that performance pay can only function well so long as the ontological relationships of institutionalized schooling are held intact during policy formation.
Psychological Background of Performance Pay
Behaviorism. Performance pay, popularly conceived, is undergirded by a behaviorist framework. The popular conception defines teaching as a highly technical endeavor wherein the teacher implements procedures and activities in the classroom that result in student obtainment of knowledge, skills, and certain traits of behavior. Under such a technical model, tying teacher compensation to the work expected of them (outcomes of student capability and knowledge) is logical and, further, morally appropriate in a utilitarian sense.
However, significant problems appear to lie within this technical model, as one simply has to ask any teacher to discover that good or effective teaching involves more than selecting from an available bank of pedagogical techniques and applying them to cause student learning. Upon reflection, most thoughtful persons will concede this point, but the popular weight of the behaviorist argument is significant and operates foundationally in many discussions of “accountability” and “meeting standards.”
We do not intend to absolve teachers of responsibility for their necessary (but insufficient) role in student obtainment of skills, knowledge and (often overlooked) moral character. Rather, using the behaviorist/technical model of learning, we suggest an insufficiency to acknowledge complex components of education and, consequently, the failure of this behaviorist framework to sustain a viable performance pay policy.
Many critics aligned against performance pay criticize merit pay's psychological implications, particularly in its deficient view of human motivation (Kohn, 1993; Ramirez, 2001). Alfie Kohn (1993), relying primarily on experimental and theoretical work by Edward L. Deci, argues that the behaviorist approach of posing an artificial motivator (an extrinsic reward) actually limits and possibly destroys intrinsic motivation within students. A substantial number of empirical studies and policy advisers have held similar ideas regarding the “destructive nature of extrinsic awards” (Deming, 1993 in Ramirez, 2001). These conclusions originate from the fact that extrinsic motivators become integral to all subsequent performances of the behavior and preclude individually determined will to affect behavior, or (similarly) the extrinsic motivator facilitates the development of a desire to avoid desired risk-taking behaviors because the praise or reward becomes preferential to risk-taking and autonomy.
Kohn's primary critique of the behaviorist model is that it overlooks (or takes for granted) the relationship between rewarder and the person being rewarded (1993). The behaviorist assumes that the only relationship of relevance is between the reward and the rewarded when, in fact, the rewarded is cognizant of the power being exercised by the rewarder. In essence, the behaviorist model ignores the internal states relevant in complex human activities and formulates all behavior as if it were occurring subconsciously or out of the supervision of any operation of an individual's will.
Kohn’s critique of the behavioral model may be too simplified. Behaviorists would classify the relationship between the rewarder and the person being rewarded as a social reinforcer (e.g., Nutter and Reid, 1978), which is often affected by the status that exists between the rewarder and the person being rewarded (e.g., Bandura, 1969). Still, the logic of Kohn’s critique leads to several conclusions. First, the person may feel controlled or powerless to define her behaviors; she is always under the direction of a rewarding/punishing authority. In the context of compensation based on attainment of performance criteria, the process of developing the organizational and procedural design of a viable performance compensation scheme is vital to its eventual feasibility.
Second, the rewarded person is likely to become dependent upon the incentive (or advancements in the reward) to maintain the desired behavior. A performance pay scheme probably will not be exempt from some current compensatory discrepancies, particularly since the pay structure is both a means of maintaining organizational commitment and facilitating recruitment. The expectation that performance pay will certainly pay off in efficient distribution of resources and control costs is an oversimplification of the interaction of psychological and economic realities.
Third, when a reward is posed at the outset, the conditions to obtain the reward circumscribe the activities the rewarded undertake(s), resulting in suboptimization (Ramirez, 2001). Since a performance pay scheme must utilize criteria for obtaining various levels of compensation, it is likely that teaching, as a profession, will become increasingly defined by the criteria of the evaluation tool. For instance, basing compensation on student performance on standardized tests will lead to a narrowing of the school curriculum and neglect of other widely-held functions a school is expected to perform. This occurs due to the cost of information (rules), requiring lower-in-cost production and a vicious process of the standardization of information. The process is initiated by an expanding institution with the demand for sameness adjoining universal or standardized information. This has two primary characteristics that join it to the logic of expansion or large-scale production; these are: (1) its cost effectiveness—that is, its ease (lower relative cost) of handling or processing; and (2) its capacity for developing cooperation and trade on an impersonal level. Expanding institutions like education (or, say, health care) prefer universal information because it has the characteristic and function of lowering the cost of production; it consists of properties that are mostly measurable, predictable, consistent, and order generating; all attributes that tend to make communication easier, that enable calculation and trade to move forward toward impersonal exchange (Rodriguez, Loomis, and Weeres, forthcoming).
One of a school's functions is to socially and morally educate youth, ideally in the context of other societal supports to this learning (Ingersoll, 2003; Powell, 1996; Commission on Children at Risk, 2003). A fourth danger of this behavioral approach to motivation is that coupling a significant amount of compensation to student achievement as measured by test scores can run the risk of devaluing students in teachers' eyes: the students become mere barriers to salary attainment. Brulle (1984) has suggested that curriculum-based assessments linked to individualized school districts might be used to determine the academic progress of students. However, while this procedure might alleviate some of the concerns about “teaching to the test,” it is both very expensive and not applicable to current federal testing requirements. Teachers often hold that working with students is an intrinsic motivator for teaching, and the addition of any external mechanism of motivation addressing this critical relationship should be approached with caution lest it supplant and degrade the complex interaction between students and teachers. Related concerns include newer forms of corruption in regards to meeting criteria and “working the system” to obtain financial benefits.
Motivation. Kohn's heavy-handed method of argumentation assumes that intrinsic and extrinsic motivators are somehow mutually exclusive, when in fact these motivators often coincide along a continuum and are not always in opposition. For example, an employee who enjoys his/her work and is compensated well would feel more satisfaction and overall reduction in stress than if either the intrinsic motivation of the nature of the work or the extrinsic motivator of salary or wages were low. Both intrinsic motivation and extrinsic supports are often needed, but the goal is to enable intrinsic motivation to drive apprehension of any external supports a person may need to continue the valued work (Powell, 1996).
These criticisms offered against the behaviorist model of human motivation apply to those performance pay schemes that adopt the behavioral model in total. Insofar as it conceives of human motivation as described by behaviorism, a performance pay reform will likely be ineffective, perhaps even damaging. The nuances of various performance pay schemes qualify the impact of the psychological argument against them, as some adopt a more robust view of human motivation and take effort to acknowledge the ontological, psycho-social relations of human activity within schools.
Many organizational theorists think alongside the differing, though not necessarily opposing, motivational theories proposed by Abraham Maslow and Bernard Weiner. Weiner's theory of attribution (Gredler, 2001) has particularly strong implications for performance related pay because the system is explicitly concerned with improving the motivation and achievement of teachers. Individuals have concepts of self-efficacy in regards to situations that are cultivated by experiences, for good or ill. The danger of a performance related pay system is that it may overlook other factors equally relevant to teachers' perceptions of probable success in their tasks, such as organizational, resource and social conditions within which they work. In a Maslowian sense, the additional pay available to teachers might not have strong motivational force. Consideration of the UK's performance based management system will provide opportunity to discuss in more detail the dominant theories of motivation employed by organizational theorists.
With these considerations on the psychological factors involved, we should expand our deliberation to the wider context, which is the playing field and delimiter for these psychological realities.
The General Nature of the Educational Task
Education as an institution is fundamentally the process and goal-directed relationship between teachers, students and support staff (locally and more broadly), with the aim being to enable students to obtain knowledge and skills in thinking and doing, by supporting the family and student's pursuit of growth in the moral, physical, cognitive and spiritual realms. In other words, the aim of education is the complex creation of a whole and effective person. This characterization of education is grounded upon the ontological identity of educators and students as human beings who think and are meant to be empowered to exercise their autonomy in responsible ways in order to achieve competence and, as Aristotle suggested, excellence.
Presently, education is operationally defined in human capital terms (Becker 1964) as the obtainment of knowledge and skills by individuals attending schools that contribute to individual economic success and the greater political and economic good. It is decidedly outcome-oriented.
The former, holistic view of education of the Progressive era was concerned with both inputs and outputs, both human and social capital (Coleman 1988); that is, the complex process of education (its social capital) was at least as important as the academic skill outcomes (its human capital). The modern crisis of childhood within the U.S., UK, and other western societies could be the result of educational institutions, such as school and family, relinquishing responsibility and involvement in children's lives to a narrow range of technical activities whose intentions are responding to a false signal to what children really need. The evident disintegration of formative human community has detrimental impact upon children's lives. The process and form of educational communities, centered within a network of relationships like family and civil society, are crucial elements to the eventual “outcome” of psychologically and physically healthy, moral and contributing human beings (Coleman 1988; Commission on Children at Risk, 2003).
The collapsation of the education good into narrow outcomes such as test scores risks reduction of institutional identity to banal production and the eventual elimination of a whole set of inputs that can—in a first order sense—enable the desired outcomes anyway. Said colloquially, information is inherently costly, and due to the cost of information the proverbial cart (lower-in-cost outputs) has been placed in front of the horse (higher-in-cost inputs), creating an asymmetrical relationship between educational attainment (e.g., the obtaining of certificates, diplomas and degrees) and the commensurate level of knowledge and skills that attainment once implied. Over time, the lower production costs associated with educational attainment trumps the actual development of higher-in-cost knowledge and skills development; scale and scarcity require that inputs are traded off against outputs. Standardization of competition, acting as the great leveler, heightens the importance of attainment (perhaps the most extreme effect of standardization) above actual knowledge and skills acquisition. This cost-induced emphasis in turn leads to a distorted signal to the individual teacher and student, as well as to society (labor market, etc.), generating the belief to an individual or society that progress is being made: expansion of educational attainment is a solver of inequality, whereas it may well be the case that it is a generator of inequality. This standardized direction in schools causes a loss of ability for students to connect at a personally and ontologically nurturing level, a critical input for the obtaining of any desired outcomes, test scores or otherwise.
Properly understood, the process of education, then, is a highly complex endeavor with rich interests in the development of human beings. Schooling plays a fundamental role, yet remains nested within other factors and institutions that have a responsibility and, in many cases, capability to contribute. In order for any reform to take place, it must logically account for the identity and purposes of the institution(s) it seeks to reform. This consideration, conjoined with the psychological phenomenon previously discussed, provides a robust ontological standard by which to evaluate potential effectiveness and applicability of performance related pay systems within an educational context.
Denver Pay for Performance (PFP) Pilot
Context of the Denver (Colorado) PFP Pilot. The largest, most comprehensive study to be released regarding performance pay for educators is the recent analysis by the Community Training and Assistance Center of a four-year pilot project in the Denver Public Schools (CTAC, 2004). The performance pay pilot emerged out of a desire on the part of the school board and the teachers' union to improve instruction for students and the profession of teaching. The structure of the pay system was determined by a board composed of various school officials, union members and parents. Teachers wanted the chance to have input into a system they sensed was coming: “Oregon has imposed PFP [pay for performance] through legislation. If Colorado imposes it, we’ll be glad that we tried it out on our own and that we have data” (CTAC, 2004, p.21). The pilot's compensation architecture and study design was characterized by: (1) teacher-set objectives as the basis for performance-pay compensation; (2) scale of implementation limited to an initial 12 schools; (3) letting schools opt in; and (4) promoting the involvement of an independent outside organization to conduct the study (CTAC, 2004). Classroom educators remained in significant control of the design and adaptation of the PFP policies and assessment procedures.
The Denver pilot focused on ensuring student growth and designed the work of teachers to enable attainment of that goal. The teachers individually or in groups set performance objectives which were approved by the school principal. The teacher chose the measurement device and obtained baseline data before setting the goal for end-of-year student performance in order to choose appropriate goals for the students that wouldn't be unduly confounded by previous experiences or socio-economic status. Bonuses were riders on the in-place salary schedule, consisting of US$500 stipends for undertaking the work of goal-setting. Teachers were required to set two objectives. Per objective obtained, teachers were given bonuses of US$500 the first year, followed by US$750 per objective in the second year (CTAC, 2004). Determining the amount of bonuses which teachers will find commensurate with the work required to obtain them is one difficult component of designing a performance pay salary structure (Odden, 2000). The teacher input in the Denver pay architecture helped to make the bonuses of significant value.
Research Design and Considerations. The study sought to answer the following research questions: (1) How does pay for performance affect student achievement?; (2) What is the impact of teacher developed objectives?; (3) To what extent do school, teacher and student factors impact and/or explain student achievement?; and (4) What broader institutional factors have influenced the implementation of the pilot, and how have these factors affected student achievement?
The evaluation of the pilot involved complex analysis due to diversity across the school system. Researchers relied upon triangulation to verify and clarify the quantitative and qualitative results of the pilot. Using comprehensive surveys, interviews (individual and group), documentary data, observations, and student achievement data in a value-added approach based on results from the Iowa Test of Basic Skills and the Colorado Student Assessment program, the researchers hoped to uncover statistically hearty data describing the impact of PFP on the Denver School System.
The research design included groups of control schools and (self-selected) experimental schools at all levels from kindergarten through grade 12. The evaluators could not realistically obtain standardization of organizational procedure across the various schools, nor equivocate other relevant factors in school performance, though their statistical analyses attempted to minimize the impact of those factors on interpreting the data.
The study was unable to eliminate difficulties arising from self-selection on the part of schools participating, plus the control schools had lower performance on assessment, hence regression towards the mean over time hindering easy identification of a positive result for PFP (CTAC, 2004). Amongst other confounding factors were that control school participation in ITBS testing was not as high as desired; some schools were uninformed that they needed to continue administering the test for the purposes of the study. Some control schools also took advantage of protocols being used in the pilot schools, such as web-based objective setting software and access to online assessment results databases, reducing the identification of definitive effects of the PFP compensation and organizational scheme.
Results of the Denver PFP Pilot. The Denver pilot's key component was to be found in the teacher-set objectives. Meeting the year's objectives was associated with higher mean scores on external assessments, particularly when the teacher met both of the objectives set. The study also reviewed all objectives set and ranked them on a scale of (1) too little to evaluate; (2) needs improvement; (3) acceptable; (4) excellent. It was found that meeting objectives classified at higher levels was associated with higher student achievement on external assessments (CTAC, 2004). The association of the bonuses with objective obtainment seems to logically implicate those bonuses in facilitating as an incentive to meet objectives. We need additional research to determine if the bonus incentive is a necessary condition for these objectives to be met at higher levels, but this may be moot if we, at the outset, believe that teachers who work to meet objectives and obtain them should receive additional monetary compensation for their effort. At the same time, we must keep in mind our earlier discussion of psychological implications of bonuses regarding work with human persons and exercise thoughtful caution in making the bonuses anything beyond supplementary.
In the Denver pilot, teachers maintained significant, primary control over how they were assessed and the content of the performance goals that they set for their students. Administrators became informed of the efforts of the teacher and could more appropriately support teachers in their classroom. Interaction amongst colleagues, particularly those who set collective goals, increased as assessment data was shared and strategies discussed. The report issued by CTAC (2004) names performance pay as a “catalyst for change.” In the Denver case, PFP opened up the discussion amongst teachers and administrators about the classroom and the type of work teachers were undertaking. Survey results suggested that, overall, competition amongst teachers was unaffected or in some cases decreased and was replaced by an improved atmosphere of collegiality (CTAC, 2004). These relationships were facilitated by the demand of generating quality information about the learning process inherent in a PFP scheme. Indeed, from surveys and interviews, the Denver PFP pilot gave teachers improved access to student data, helped them set informed objectives, and created a culture of goal-setting for students that was shared across the school, from the principal to the classroom (CTAC, 2004). Many shared the opinion of one of the pilot teachers: “ I would rather any day to have good leadership, professional development, and time with my colleagues than $1500” (CTAC, 2004, p. 79).
The current systemic accountability measures embodied in the U.S. 2002 No Child Left Behind reauthorization of the Elementary and Secondary Education Act developed shortly after the Denver pilot began (CTAC, 2004). The high stakes testing environment, if eventually melded with performance pay, could be incommensurate with the approach taken in Denver, i.e. enabling teachers themselves to set objectives and determine the assessment strategy at a local level by which their students will be evaluated for learning. Since these components are viewed as critical to allay the fears of teachers that the PFP system would be ultimately arbitrary and result in unfair distribution of compensation, tying PFP to an imposed assessment device by an authority such as a federal government would likely be counterproductive.
Effects of the Performance Management Initiative (PMI) in the Educational System of the United Kingdom
The United Kingdom's recent governmental reorganization efforts have been to adopt performance related pay across civil service divisions to increase the gains obtained from tax expenditures in regards to services delivered to the citizenry. In the case of education, the expectation is that higher standards will be obtained and the subsequent reform of the system towards excellency will place the UK at a competitive advantage within the world economy (Storey, 2000).
The policy being implemented in the UK is multifaceted and contains relatively few 'pure' performance related pay elements (Storey, 2000). Like its counterpart in the United States, the United Kingdom's predominant system has been a salary schedule with various lanes or spines which people fall into and are given compensation based on experience and responsibilities undertaken. The salary schedule has benefits of transparency and ease of administration, but lacks the desired capacity to differentiate rewards between high and low performers within the organization (Storey, 2000; Odden, 2000).
The performance related pay reforms contain three primary focal points: improved teacher performance, target setting for improved student performance, and professional development. The resulting system, developed under advisement of a management consulting group, includes a significant increase in tying incentives directly to teachers' work, making professional development and assessment of teacher performance and capabilities the primary criterion for moving upwards in the pay schedule (Storey, 2000).
Storey's (2000) evaluation is based upon the volumes of responses submitted to the Department for Education and Employment (DfEE) at Whitehall during the comment period after the reforms were proposed, not necessarily the actual outcomes of the reform. Given that most of the comments came from parties who have a direct role in education, it is relevant to observe their concerns because the reform system will fail (at least politically) if it neglects addressing those concerns. By systematically sampling the written responses to gain a wide scope of organizational, personal and geographic representation, Storey intended to cull insights into the major issues regarding the performance related pay reforms.
The major concerns fell into two camps: recruitment, retention and motivation of the workforce and the type of assessment used to determine performance.
Recruitment, Retention and Motivation. The performance pay system would not adequately address the United Kingdom's reliance on teachers from Australia, New Zealand and South Africa to sustain replacement of retirees, particularly with the future mass departure of an aging workforce. Retaining employees requires transparency and fairness, in addition to a relative guarantee that resources for compensation will be available. Many expressed doubts that the system could be straightforward and not eventually become subject to mechanisms such as pay level quotas in order to shore up budgets (Storey, 2000). In a similar situation, the United States' urban school districts increasingly rely on teachers who are not certified or are alternatively certified. Lowering trade barriers to certification arguably eases access to increasingly difficult to find quality labor. Performance pay may not address these issues of recruitment and retention effectively.
Three dominant motivational theories were mentioned in reference to the performance related pay policy reform: goal-setting theory, which demands that clear, specific goals must be set for motivation to translate to action; expectancy theory, a lá Weiner, that individuals must have a sense of efficacy regarding the task at hand; and equity theory, which asserts that persons must be treated equitably to have a relative guarantee that they will receive recognition or compensation commensurate with effort expended. The numerous stipulations regarding teacher competence within the proposed design, though specific, become overwhelmingly detailed when taken collectively. Teachers are likely to ignore most of them out of pragmatic concerns. Additionally, teachers' expectancies that they will receive the posited reward and that the reward is sufficiently desirable are not apparent in the responses made to the DfEE, granted that the responses are self-selected (Storey, 2000). A sizable number of responses doubted the ability of an educational organization to facilitate such reforms.
Evaluation of teacher performance. Whilst the proponents of the proposal assert that evaluation will be based on objective, clear criteria, many doubt the ability of any assessment method to evaluate teaching in those terms. Respondents consistently pointed to the fact of the complexity of schools as being incommensurate with a standardized assessment procedure upon which it is proposed to base distribution of compensation; that is, internal and external validity of the evaluation tool is doubtful. Even when taking into account the observation of supervisors (head teachers) and inclusion of particular, context-acknowledging information into the assessment, many distrust the intrusion of non-teaching issues (interoffice politics, racism, sexism, etc.) as becoming unduly potent through the new distribution of pay. Integrating student performance into the evaluation scheme is feared because it may consider the relationship between teacher 'performance' and student achievement as too definitive (Storey, 2000).
Performance pay brings to the surface the issues surrounding evaluation of teachers, even when compensation is not necessarily at stake. Many evaluation tools are often incommensurate with the nature of an open, ontologically complex system inherent to a classroom. We can expect that exhaustive evaluation tools will never be obtained finally, though we can approximate to a sufficient evaluation or measurement device so long as it is widely understood that the tool is limited in its efficacy to be thorough, objective and fail-proof.
Sidenote: Legal Considerations. Particularly in the United States, the legal ramifications of a merit pay reform policy emerge as critical implementation issues, particularly in regards to due process. Sufficient systems of documentation, feedback, succinct declaration of criterion and agreed-upon standards and procedures for evaluators are requisite for the merit pay to be distributed in a manner that can legally substantiate decisions managers make in regards to paying their employees. These procedures need to be spelled out within and remain respectful of collective bargaining agreements, which will likely remain a feature of compensation in education, operating within the statutes of labor law. Court rulings (see A.A. Moris, 1974 for a summary) in the U.S. have given school authorities significant leeway in establishing criteria for teacher credentials and performance, but inclusion of unions will be legally (and pragmatically) crucial to the implementation of pay for performance on a deep organizational scale (Desander, 2000; Kerchner, 2003).
Recognizing institutional identity. Failing to take into consideration the complex nature of schools, the performance related pay reform cannot be applied either generically or wholesale. In certain contexts within the school, a performance pay system may work to obtain limited goals, and probably only those goals that are temporary and relatively simple (Storey, 2000). As evidenced by the Denver pay-for-performance pilot, performance pay can function in a limited role when trying to obtain clear, limited objectives that are well-understood by the teacher and contextually relevant (CTAC, 2004).
Tennessee Career Ladder Evaluation System and Project STAR
The Tennessee Career Ladder Evaluation system consisted of multiple measures of teacher quality and performance linked with substantial professional rewards. Data from Tennessee classrooms provided a unique opportunity to investigate the effects of performance pay due to a contemporaneous classroom size-reduction experiment: Project STAR (Student Teacher Achievement Ratio). The STAR class-size experiment resulted in largely random sorting of students and teachers, which helps to drastically reduce the confounding biases that complicate interpretation in other studies, such as the Denver PFP pilot (Dee & Keys, 2003). Project STAR included assessment using the Stanford Achievement Tests, which provided a data set for analyzing student achievement.
The Tennessee Career Ladder System was implemented in the mid-1980s during the tenure of state Governor Lamar Alexander as a component of a package of school reforms. Teachers progressed (voluntarily and with high participation) through the career ladder after an initial 3 year apprenticeship. Movement into Level I (of three levels) was implemented by local review under oversight by the state. Moving into Levels II and III required demonstrating superior performance to the state board of education. Entering levels brought pay increases, from US$1000 in addition to base pay upon entry to Level I, up to US$7000 in Level III (Dee & Keys, 2003).
Adequacy of Evaluation Measurements. The career ladder evaluation was multi-faceted, consisting of portfolio review, observations, student surveys, teacher interview and testing in several domains of teaching competence such as planning, communication, classroom management, strategies, etc. Of note is that the reform did not include reference to student performance by direct measure. All evaluations were conducted by individuals trained extensively by the state. In the case of Level I evaluations, trained principals were the evaluators, while Levels II and III were mediated by teachers external to the school who had achieved Level III status. Though the evaluators were trained, some teachers were critical of the adequacy and validity of the evaluation procedures.
Study results. The subjects of study were students involved in Project STAR from kindergarten through their progression to grade three. Using regression models, Dee & Keys (2003) analyzed the effect of presence in a career-ladder teacher classroom on gains in achievement as measured by the Stanford Achievement Tests. They found that mathematics scores increased by 3.9% (p<.05) over the course of the “treatment” when students were in classrooms with teachers in the career ladder. However, no significant increase was identified for reading scores (Dee & Keys, 2003).
Conclusions. The analysis of the Tennessee Career Ladder's effects on student achievement demonstrates a key point: teacher competency is linked to student achievement and exists amongst a network of other interacting elements. The Tennessee Career Ladder is decidedly different from the Denver PFP scheme because there was no explicit reference in advancement criteria or compensatory reward to teachers' effectiveness in raising student achievement. An improved evaluation system results in some apparent increase in student achievement when compared to the traditional system (Dee & Keys, 2003).
Based on these three studies, we can expect that performance pay may or may not function productively in educational settings; its effectiveness is contingent upon other factors of equal or surpassing weight. These other factors, which we discuss presently, must guide the design of a performance pay system that is viable and supports organizational goals.
Performance Pay and Accountability
Interaction of Organizational Characteristics with Compensation. Implementing a standardized system of teacher professional assessment finds itself torn between two apparently irreconcilable demands. On the one hand, a straightforward, simple assessment device would keep costs lower and be easy to administer, but it would be difficult to justify (legally or conceptually) such a device as being reflective of the complex educational task being evaluated. Conversely, having multiple criteria and nuanced assessment tools (interview, observation, authentic assessments of student learning) would be prohibitively expensive but justifiably more accurate in regards to teacher evaluation (Cohen, 1996).
Other compounding factors for the evaluation instruments include student mobility (which students are legitimate to include in the measures), justifying established, standardized cutoff points as nonarbitrary, as well as the inherent unpredictability of the teacher-student relationship and outcomes arising from that relationship (Cohen, 1996).
Cohen, a prominent scholar in this area, recognizes the critical organizational features of schools. However, he implies that the organization must be changed to accommodate and respond to a rewards scheme. Despite the inherent difficulties in implementing a performance pay system in schools, he insists that “the capacity to respond to performance rewards should be understood as a problem of organization and culture as well as a matter of individual knowledge and values” (1996, p. 86). Further, “rewards and penalties are essential in our imperfect world, for they help us decide what is most important out of all that we need and want, and they help us influence others' decisions” (1996, p. 96). The manipulative, behaviorist stance he exhibits toward this unresponsive system seems to misunderstand the set of connections and relationships that actually comprise education and, in so doing, flattens accountability to verifiable compliance to externalities—in this case, demands by policy makers and, in theory, those on whose behalf the policy makers act. This tends to be a top-down ordering of rules and incentives, not a bottom up one.
Accountability as broad, multi-relational support of education. We would suggest that Cohen's flat view of accountability severely interrupts and incapacitates an institution whose work is better founded upon a community's collaborative effort to provide an educative experience for its children, rather than service delivery to a waiting customer (students, like teachers, should be considered co-producers of the education good). The rewards system that could be used to puppeteer the various employees to meet desired goals is inappropriate to the widely held aims of schooling, and any ineffectiveness is a fault of the reform measure, not the system itself.
Schools are coupled to a network of interrelated parties who all have a stake in providing a functional, child-nurturing education. In democracies, the school is formally accountable to the electorate in both obtaining funding and through school boards (for most districts). Teachers are responsible as employees to the school and to the greater institution of which they are a part, and districts are responsible, acting on behalf of the citizenry, to provide teachers with the resources and institutional supports to complete their jobs. Teachers are also directly accountable to students and families to provide quality instruction and to work with parents to face challenges in developing in students’ human and social capital. Families and the students in turn are responsible with cooperating and being proactive in supporting the work of the teacher. Other community institutions participate by making legitimate demands of the school system, such as providing a quality workforce and responsible citizens, while retaining responsibility to support the efforts of the school to meet such requirements. All parties participate in the educational endeavor as part of a sociological/ecological network.
To this point we have assumed, along with a substantial portion of the policy literature, that increasing performance of schools means increasing the academic and intellectual achievement (human capital) of the students who (hopefully) graduate from those schools. This view tends to narrow the education good that society has traditionally expected from schools. Schools are also expected to reliably perform both an intellectual and a moral function by instilling socially productive behaviors, valuations, and relations (social capital) in students' lives in addition to providing an intellectual education (Ingersoll, 2003).
Pragmatically, performance pay focuses on the intellectual attainment of students because of the propensity to opt for simplified assessment tools by which to determine pay. Including the social component of schooling into the equation would increase complexity of assessment, drive up costs, and run the risk of overstating teachers' influence in affecting the decisions students make regarding how to behave and what to value. This lack of control on the part of the teacher in the moral sphere manifests itself in the intellectual tasks of schooling as well; teachers are not often supported by students' families or the wider culture in supporting activities and policies that enable student learning (Powell, 1996). Additionally, society is conflicted to a degree about what values to teach and how those should be taught.
Accountability is an absolutely critical component of school reform, but it only has long-term effectiveness when functioning as a network of viable, information-sharing relationships. To focus accountability as an institution-to-teacher mechanism risks absolving others of their responsibility in supporting the success of education for children. A performance pay scheme could play a part in a reform effort, but only if the requisite relationships and information economy are functional in aspects of accountability, cooperation and active facilitation of the work of schools. If parties agree that performance pay is a productive form for systematic accountability, the structural issues dictate that the form of the performance pay must enable teachers to have considerable say in how and what they provide in their relationship to students and families. However, merit pay simply cannot force on its own the broad, motivated and sustained accountability in relationship amongst all parties that the aims of a school require.
Attempts at aligning organization with the identity of the educational task. Linda Darling-Hammond (1996) outlines a series of New York City schools that move toward providing a collaborative institution that enables teachers to do their work and demands of parents, students and community institutions that they support and work toward those same goals. Individual schools are small, within which are contained houses of 70-85 students who all are taught by a team of 4-5 teachers. “Taught” in this school means students receive instruction, advisement and active parental consultation from a group of teachers and perhaps a few support staff, such as a guidance counselor/internship coordinator. Students stay in these houses for 2 year periods of time starting in the equivalent of seventh grade. Many middle schools operate, at least in part, under similar principles of students' class organization.
These schools all remain small and structure high involvement teacher-student interaction for stability over the long-term. Each school is designed to operate at a personal level, rather than through the dominant institutional production model commonly utilized by most schools, particularly secondary schools (Darling-Hammond, 1996; Mohrman & Lawler, 1996; Meier 1998). Students receive benefits by being bonded to important adult members in a learning community and by having more opportunities for extracurricular involvement and experience in leadership capacities. By the school being smaller, more is demanded of teachers and students for its functioning, which provides a context supporting high-quality learning. Indeed, putting responsibility and the tools to fulfill the task into the hands of those involved directly in the task of education increases their expectancy of success, in line with the expectancy theory of motivation (Mohrman & Lawler, 1996).
These schools “build strong relationships, rather than rely solely on rules for governing behaviors” (Darling-Hammond, 1996, p.146). The small overall size of schools and institutional preference for hiring educators who will share the responsibilities and organizational operation in an atmosphere of teamwork drives down class sizes for students and enables the institution as a whole to be able to function well without teachers being forced to implement overly specific procedural directives. The collapsation of administration and teaching (at least at the instructional and behavioral level) enables the teachers in these schools to structure the school day and classroom interaction, to function with the necessary information to be effective, and to realign their practice toward student achievement and learning, broadly defined. By structuring their interaction with students to get to know them better, the teachers are able to be better teachers (1996).
Of course, problems of politics and interpersonal strife or misunderstanding will certainly occur in this system. However, it is reasonable to expect no more in this alternative and responsive environment than what current institutions experience because information flows more freely between the decision-makers and implementers, who are often one and the same.
Information diversity and accessibility (both in mere availability and capability for infusion into decision-making) emerges as a crucial issue in organizations such as schools. When schools (and institutions in general) ascend in scale, the tendency is to adopt specialized roles because necessary tasks of coordination and decision-making become increasingly time-consuming and require greater expertise. To a certain degree, this is necessary and beneficial (many people get what they want). However, by specializing decision-making, those involved in the critical aspects of highly unpredictable work cannot pass insight effectively to the management. As information becomes costlier, the division of labor becomes intensified in order to allay cost. Hence, disconnect surfaces between institutional demands and the control teachers’ exercise over their required tasks and perceived or real capabilities to meet students' needs. The small school approach, with tight-linkage between administration and instruction, avails information to practitioners, parents and students to continuously structure and adapt learning toward agreed-upon goals.
Qualifying Performance Pay: Compensation Aligned with Organizational Form. An adaptive, organic form of organization (Darling-Hammond, 1996; Meier, 1998) is a tendency in some modern organizations. Within this context, a form of ‘performance pay’ may take shape that is distinct from some common systemic reform approaches. Odden (1996), for example, finds the current system of teacher compensation to be inappropriate for the changing goals of education, particularly the demands that teachers in the decentralized, organic organization are likely to hold. The current pay structure rewards teachers for obtaining additional coursework (attainment), but often teachers choose courses that are somewhat disconnected from improving teaching practice or subject matter knowledge (e.g., enrolling in administrative programs to matriculate out of the teaching field). By linking pay to teaching competencies and demonstrable knowledge and skills acquisition (some of which will be administrative), Odden believes that the stated aims of educational reform (in the mid-90s context, “school-based management”) would be supported by teacher's decisions in training and professional development selection (1996, 2000). This appears to be verified by the work of Kerchner, Koppich and Weeres (1997).
Unfortunately, the other incentive (the standardized information environment) often mentioned alongside such decentralization efforts have worked against emergence of this organizational structure on a broad scale. Demanding higher standards and—the crux of the issue—developing mechanisms and rules to enforce those standards causes a re-centralizing of administrative discretion at hierarchical organizational units progressively distant from teachers and students (see Weeres, 1993; Rowan, 1996).
New pay structures are a reflection of changes in the organization, not the genesis of change. To accommodate the new demands and tasks required of educators, the compensation system should be brought into line with the new responsibilities given to teachers. Odden finds that skill-based pay is a systemic incentive embedded within a decentralized workforce; blocks of skills upon which to base a pay structure, particularly when administrative and teaching skills are conjoined, would support more realistically a decentralization of the administrative task as teachers assume those roles and responsibilities (Odden, 1996).
Critical to success is universal involvement in designing pay structures—those who have a stake in the issue must be incorporated into the design process lest the teaching and learning tasks become decontextualized and unreal. This partnership includes not only initial planning, but perpetual involvement in evaluation and modification. Otherwise, the teachers it affects are increasingly likely to become alienated and incapacitated. Teachers, their unions, administrators, parents, students, and politicians all must be incorporated into the redesign initiative (AFT, 2001; Odden, 1996; Mohrman & Lawler, 1996; Kerchner 2001).
Indeed, the major teacher's unions in the U.S. have endorsed reforming the pay system to exclude the phenomenon of career advancement due to obtaining credits or other professional development units that have little to do with learning and teaching. The preference would be to orchestrate pay structures that reflect the changing nature of education—stipends/bonuses for (U.S.) National Board Professional Teaching Standards (NBPTS) certified teachers and those who can teach in multiple fields at the secondary level. However, the unions warn that the equity, particularly in regards to gender, achieved in the single salary schedule not be lost (AFT, 2001; NEA, 1997). Further, the unions disdain linking pay directly to student performance on standardized tests because of fears of a narrowing curriculum and loss of professional discretion (AFT, 2001).
The organic form of organization only seems to function effectively when the work, in this case very personal and complex work, is done on a relatively small scale. Small schools can enable teachers to plan, evaluate and implement in conjunction to providing instruction. The larger the school, the less resources (time, expertise, etc.) teachers have at their disposal to put toward managing the school; scale and expansion requires a division of labor. Hence, specialization and delegation result, distancing teachers from some of the structures that they need input and control over to become ideally effective in the classroom Bureaucracy imbues an organization (an individual school) or institution (say, education) to bring order and enable coordination and communication in organizational processes, yet may disable teachers in critical ways from doing the kind of work expected of them (Ingersoll, 2003).
The Potential of Performance Pay in Policy Reform
The broad range of reform strategies that can be classified as performance/merit pay are girded by an equally diverse landscape of psychological, sociological and philosophical assumptions. These assumptions lead to a variety of pay for performance or merit pay designs which, up to the present, could not be implemented successfully due to either: a contradiction in components of the design derived from competing or contradicting assumptions, or the system became prohibitively difficult to manage.
Broader accountability. A linkage of compensation to teacher performance, however defined, systematizes accountability in only one aspect of the social network within which educational institutions are situated. Figure 1 is a conceptual scheme similar to ecological models of a schools' relationship to society at-large and the parties involved within the institution itself. In the figure, the red line indicates the relationship addressed by performance pay schemes: the accountability between teacher and institution is intensified because teachers are (regardless of the scheme) compensated according to institutional aims, with the institution providing the supports to meet those aims. This accountability places evaluative weight on the circled interaction, that between student and teacher. However, if the performance pay accountability mechanism does not recognize that the classroom is an open system, it will fail to facilitate constructive accountability.
This ecological model shows the limitation of a performance pay scheme to effect the type of systemic change often touted in political discourse as an effective way to bring student academic achievement into line with society's needs. The performance or merit pay scheme can become dysfunctional whenever the relationship becomes one-sided, such as external demands and objectives being placed on teachers. Though teachers contain to retain considerable control over academic instruction in their own classroom (Ingersoll, 2003), removing this discretion for the sake of “accountability” is likely to be counterproductive, both in the understanding of schools according to organizational theory and psychological models. Performance pay schemes are further limited by their tendency to overlook the complex moral and social function of schools, which society still expects schools to perform, even when these expectations may not be explicitly communicated or dominate discussions of accountability.
Performance pay is concurrently suggested as a means of staffing schools with highly qualified teachers. The monetary incentives are hypothesized to make the teaching profession more attractive and enable schools to retain teachers whom they currently employ. Again, if the larger institutional and social contexts of the school are not concurrently reformed to make accountability multi-dimensional, the efficacy and satisfaction teachers experience in their work will detract from or possibly eliminate any benefit performance pay could have in teacher recruitment and retention. Further, if the larger reforms aren't instituted, pay could spiral beyond sustainable levels because larger and larger amounts of compensation will be required to attract and retain people in work that (if unreformed) is not inherently satisfactory (Mohrman & Lawler, 1996).
Broader incentives. The larger problem is how to develop a set of overarching incentives to move the educational system forward (Powell, 1996). These valuations by all parties involved should revolve on similar goals, such as student growth to maturity, responsibility and skill in thought and action. Until the wider culture (or at least the local community surrounding the school) values education in this sense, the system will not be able to move forward because all conjoined parties, not merely students and teachers, are required to move the organization forward in an efficient manner.
Information Sharing in Relationship. The form of relationship between teacher and institution that could be achieved in performance pay may function well when lodged within policies and praxis that acknowledge and facilitate communicative relationship and accountability amongst all parties involved in the public educational enterprise. Teachers can meet the demands of communities only when they are given the discretion to exercise professional expertise and are given clear information about their students (one source being from norm- and criterion-referenced standardized tests) and the larger needs of the society. The analysts of the Denver program framed PFP as a catalyst for system-wide change. Indeed, a performance related pay structure absolutely requires a flow of quality information regarding learning in the classroom. The Denver pilot project demonstrates how information sharing amongst individual teachers and administration can be facilitated by a PFP compensation scheme.
The limited scope of the Denver PFP pilot is probably the key to its general success: it recognizes that multiple factors are involved in student learning and enables teachers to discuss with each other and the administration about the task of enabling students to access and use content knowledge and skills. We must hold to the broad concept of education, recognizing its unique and multi-dimensional character, in both our day-to-day interactions and policy formulations. Doing this requires the continued work of qualified, dedicated people who understand academics as viable only in extension of more foundational commitments to students, their families, and society as a whole. Teachers and administrators must work with the wider community and students as strong leaders, with the goal of cultivating widely-held, positive valuations of education and responsibility to youth (Mohrman & Lawler, 1996). These relationships in the community are the context within which education is constrained to occur. Enabling real accountability would appear to require discussion and collaboration in these relations. To forget this is to ensure the eventual breakdown of several cardinal aims of education.
Citation: Honeycutt, J., Loomis, S., and Brulle, A. (2006). An Analysis of Merit Pay Reforms in Educational Institutions, Nonpartisan Education Review / Essays, 2(5). Retrieved [date] from http://www.nonpartisaneducation.org/Review/Essays/v2n5.pdf
Access this article in pdf format
American Federation of Teachers (2001). AFT on the Issues: Merit Pay, “Pay-for-Performance” and Professional Teacher Compensation. <http://www.aft.org/issues/meritpay/meritpay.html> Accessed on 20 April 2004.
Bandura, A. (1969). Principles of Behavior Modification. New York: Holt, Rinehart & Winston.
Brulle, A. R. (1984). “Teacher evaluation: Let’s do it right.” MATE Viewpoints, 6:1, 12-13.
Cohen, D.K. (1996). “Rewarding Teachers for Student Performance.” Fuhrman, S.H. & O'Day, J.A., (Eds). Rewards and Reform. San Francisco: Jossey-Bass Publishers.
Coleman, J. (1988). “Social Capital in the Creation of Human Capital.” The American Journal of Sociology. 94:95-120.
Commission on Children at Risk. (2003). Hardwired to Connect: The New Scientific Case for Authoritative Communities. New York: Institute for American Values.
Community Training and Assistance Center (CTAC). (2004). Catalyst for Change: Pay for performance in Denver, Final Report. <http://www.ctacusa.com/denver-vol3-final.pdf> Accessed 30 Mar 2004.
Darling-Hammond, L. (1996). “Restructuring Schools for High Performance.” Fuhrman, S.H. & O'Day, J.A., Eds. Rewards and Reform. San Francisco: Jossey-Bass Publishers.
Dee, T.S. & Keys, B.J (2003). Does merit pay reward good teachers? Evidence from a randomized experiment. Swarthmore College Department of Economics. <www.swarthmore.edu/socsci/tdee1/Research/merit0803.pdf> Accessed 2 April 2004.
Deming, W.E. (1993). The new economics for industry, government, education. Cambridge, MA: Massachusetts Institute of Technology Center for Advanced Engineering Study.
Desander, M. (2000). “Teacher evaluation and merit pay: Legal considerations, practical concerns.” Journal of Personnel Evaluation in Education 14(4): 307-317.
Gredler, M. (2001). Learning and Instruction: Theory into Practice. Upper Saddle River, NJ: Prentice Hall.
Holt, M. (2001). “Performance pay for teachers: The standards movement's last stand?” Phi Delta Kappan 83(4): 312-317.
Ingersoll, R.M. (2003) Who Controls Teachers Work? Cambridge, MA: Harvard University Press.
Kohn, A. (1993). Punished by Rewards: The trouble with gold stars, incentive plans, A's, praise and other bribes. Boston: Houghton Mifflin.
Kerchner, C. (2001). “Deindustrialization” in Education Next. Fall, No. 3.
Kerchner, C., Koppich, J., and Weeres, J. (1997). United Mind Workers: Unions and Teaching in the Knowledge Society. San Francisco: Jossey-Bass.
Labaree, D. (2004). The Trouble with Ed Schools. New Haven: Yale University Press.
McCollum, S. (2001). “How merit pay improves education.” Educational Leadership. 58(5): 21-24.
Meier, D. (1998). “Can the Odds Be Changed?” Phi Delta Kappa. January.
Mohrman, S.A. & Lawler, E.E. III. (1996). “Motivation for School Reform.” Fuhrman, S.H. & O'Day, J.A., (Eds). Rewards andReform. San Francisco: Jossey-Bass Publishers.
Morris, A. A. (1974). The Constitution and American education. St. Paul, MN: West Publishing.
National Education Association (NEA). (1997). Advocate. Online Dialogue: Gender Equity in Higher Education Merit Compensation. <http://www.nea.org/he/head9697/advo9705/dialog.html> Accessed 20 April 2004.
Nutter, D. and Reid, D. H. (1978). “Teaching retarded women a clothing selection skill using community norms.” Journal of Applied Behavior Analysis, 11, 475-487.
Odden, A. (2000). “New and better forms of teacher compensation are possible.” Phi Delta Kappan 81(5): 361-366.
Odden, A. (1996). “Incentives, School Organization, and Teacher Compensation.” Fuhrman, S.H. & O'Day, J.A., (Eds). Rewards and Reform. San Francisco: Jossey-Bass Publishers.
Powell, A.G. (1996). “Motivating Students to Learn: An American Dilemma.” Fuhrman, S.H. & O'Day, J.A., (Eds). Rewards andReform. San Francisco: Jossey-Bass Publishers.
Ramirez, A. (2001). “How merit pay undermines education.” Educational Leadership. 58(5): 16-20.
Rodriguez, J., Loomis, S., & Weeres, J. (forthcoming). The Cost of Institutions.
Rowan, B. (1996). “Standards as Incentives for Structural Reform.” Fuhrman, S.H. & O'Day, J.A., (Eds). Rewards and Reform. San Francisco: Jossey-Bass Publishers.
Storey, A. (2000). “A leap of faith: Performance pay for teachers?” Journal of Education Policy. 15(5): 509-523.
Weeres, J. (1993). “The Organizational Structure of Urban Educational Systems: Bureaucratic Practices in Mass Societies” in Rothstein, S. (Ed.). Handbook of Schooling in Urban America. Westport, CT: Greenwood Press.