Teacher evaluation in Portugal: persisting challenges and perceived effects

Abstract This paper reports on findings from a broader piece of research aimed at investigating the ways in which head teachers experience a new teacher evaluation policy in Portugal, particularly in regard to the challenges and perceived effects of the policy on school and on teacher development. Data were collected through a questionnaire with open and closed-ended questions. In total, 134 head teachers participated in the study. Findings suggest a number of tensions and problems, most of which are related to the key features of the model of teacher evaluation itself. Others issues pertain to the lack of recognition of the appraisers and to the emergence of tensions amongst staff leading to the deterioration of the school climate. Persisting challenges to policy implementation, perceived effects on teachers and schools as well as the dilemmas of head teachers are analysed. The article concludes with the discussion of ways forward.


Introduction
The implementation of a new policy is problematic (Timperley & Robinson, 1997), dynamic and complex, particularly if it requires deep change in teachers' professional cultures and in their workplace (Flores, 2010a(Flores, , 2012)).Implementing a new teacher evaluation policy within the context of increased performativity and accountability is particularly problematic (Midddlewood & Cardno, 2001;Avalos, 2004;Avalos & Assael, 2006;Asseal & Pavez, 2008;Flores, 2010a).In many cases, policy initiatives have been influenced by concerns for raising the standards of teaching and by an increasing emphasis on student achievement within the context of testing regimes and international assessments.
Literature suggests that policy is mediated by head teachers' and teachers' perceptions and experiences in the workplace and, therefore, a better understanding of their implementation experiences with and perceptions of evaluation is required (Ovando & Ramirez, 2007;Vekeman et al., 2015).In a recent review, Hallinger, Heck, and Murphy (2014, p. 22) concluded that 'research has highlighted the complexity of achieving the desired "fidelity of implementation" of the new teacher evaluation models in schools' .According to the OECD (2013), constraints in regard to successful implementation of teacher evaluation systems include resistance, lack of evaluation feedback, limited sharing of practice, appraisers' lack of expertise, poor understanding of evaluation's purposes, excessive workload and lack of resources, including time.
This article reports on findings from a broader piece of research aimed at investigating issues related to the implementation of the new policy in Portugal from the perspective of head teachers, particularly in regard to the challenges and perceived effects of the policy on school and on teacher development during the first four years of its implementation.

Teacher evaluation: purposes and context
Teacher evaluation is complex.It may entail different purposes, it may involve multiple processes and it may result in intended and unintended effects.Existing literature has identified tensions between formative and summative purposes (Chow, Wong, Yeung, & Mo, 2002;Avalos & Assael, 2006;Stronge, 2006;Su, Feng, & Hsu, 2017) as well as different factors that may enhance or hinder its implementation (Vaillant, 2008).Some authors argue that formative and summative evaluation purposes are incompatible, whilst others advocate for the need to incorporate them into the same system of teacher evaluation (Chow et al., 2002;Avalos & Assael, 2006).Stronge (2010, p. 31), reinforcing the need to create constructive climates for the process of teacher evaluation, identifies the 3 C's: communication, commitment and collaboration, which he considers as key elements in developing the synergy 'that can elevate evaluation to a meaningful dialogue about quality instruction for students' .In order for this to happen, another 'C' needs to be added, i.e. the confidence in the evaluation system itself from the part of various stakeholders, as it may undermine its quality and effectiveness and, more importantly, its successful implementation in the school context (see Flores, 2010a).
Understanding the complexity and comprehensiveness of evaluation systems consequently requires an analysis of the conceptions of teaching and teacher professionalism underpinning them (Darling-Hammond, Wise, & Pease, 1983;Day, Flores, & Viana, 2007) as well as the ways in which they are translated into evaluation criteria and standards (Avalos & Assael, 2006).Martinez, Taut, and Schaaf (2016) have identified, amongst other issues, variation in terms of how different teacher evaluation systems in different countries operationalised good teaching as well as the degree of standardisation of the classroom observation process.
Moreover the importance of teachers' perceptions and the complexity of the social dimension in the implementation process have been documented (Fullan, 2001;Tuytens & Devos, 2009;Van der Vegt, Smyth, & Vandenberghe, 2001).Therefore, both the content of the evaluation system and the context in which the system will be used should be taken into account if it is to be effective and successful (Peterson & Comeaux, 1990).In particular, issues such as the meaning of the actors involved in the implementation of a given policy, their values and emotions as well as the social interactions and the contexts in which such change is going to be implemented have been highlighted (Timperley & Robinson, 1997;Spillane, Reiser, & Reimer, 2002;Van der Berg, Vandenberghe, & Sleegers, 1999).Based on a review of research, Hallinger et al. (2014, p. 21) concluded that the ''policy logic' driving teacher evaluation remains considerably stronger than empirical evidence of positive results' .Thus, it is crucial to examine the views and experiences of head teachers in enacting policies in context and the ways in which they perceive the policy and make sense of it.

The key role of school leadership in enacting policies in context
Head teachers are important in an effective implementation process when they communicate clear purposes of teacher evaluation and encourage teacher participation (Flores, 2009;Vekeman et al., 2015).In fact, head teachers have the responsibility to successfully implement a policy in its complexity and within the context of the dynamic process developed at the local level (Retallick & Fink, 2002).For instance, when head teachers attend to teachers' professional development needs and to organisational requirements for accountability the supervisory relationship between them and the teachers can be affected (Tuytens & Devos, 2010).In a recent study, Flores and Derrington (2017) have identified the problems and strategies developed by head teachers in the USA and in Portugal to deal with the implementation of teacher evaluation policies.The authors identified the need for head teachers to balance conflicting goals, to minimise the negative effects of evaluation, to manage tensions of implementation and to make sense of the new policy and its effects at school.Similarly, Vekeman et al. (2015) highlight the importance of sense making from the part of the head teachers in the process of discrepancy reduction between initial standards they set and teachers' expectations.Writing in the context of Italy, Barzanò and Grimaldi (2013) have shown the uncertainty, unpredictability and disputability of policy enactment.Braun, Maguire, and Ball (2011, p. 549) assert that ''enactment' refers to an understanding that policies are interpreted and 'translated' by diverse policy actors in the school environment, rather than simply implemented' .
For instance, Tuytens and Devos (2011) argued for the relevance of school leadership for effective evaluation and its direct influence on feedback utility and indirect influence on teacher professional learning.They also have reported the importance of school organisational characteristics and of an integrated leadership approach for the feedback response of teachers (Tuytens & Devos, 2017).Another study (Tuytens & Devos, 2010) has demonstrated that school leadership influences teachers' policy perceptions.The authors stress that the structure the school leader provides and the amount of trust teachers have in the school leader have a significant impact on teachers' perception of the practicality of the new teacher evaluation policy.
Also, Flores and Derrington (2017) identified three key dimensions to enhance the quality and success of a teacher evaluation policy: the existence of supportive school structures for teacher evaluation; the need to interpreting and managing policy in context whilst dealing with its mediating factors; and the relevance of the formative dimension of evaluation, particularly the role of supervision.

Teacher Evaluation (TE) in Portugal: key features of a contested model
The publication of the Teacher Career Statute in January 2007 (Decree-Law 15/2007), introduced, amongst other features, teacher performance appraisal for all teachers in Portuguese public schools.The model was contested by teachers and teacher unions since its implementation in 2008 (Decree-Law 2/2008), a process marked by resistance and controversy (see Flores, 2010bFlores, , 2012)).The imposition of the model, an increase in workload and bureaucracy, an emphasis on the summative dimension made it difficult for schools to manage and implement the new system which was perceived as a burden for both schools and teachers (see Flores, 2009).
Before 2007, all teachers would follow the same path in order to progress to the top of their career.In general, progression was understood as a 'matter of time' in so far as it was dependent upon years of experience, a number of credits obtained for attending in-service education courses and the writing-up of a critical reflection on one's own practice.Teachers had to write-up a report (self-assessment report) in which they stated the activities they had undertaken, the teaching they had done over a given period of time (depending on the stage of the career in which they were, usually for 4 or 5 years, except for the one year contract teachers who had to do it annually).The report was to be assessed by the leadership team (the Executive Council) of the school in which they worked.It was an administrative and bureaucratic model for progression in the teaching career within a teacher evaluation system which 'did not evaluate' as literature has highlighted (see Silva & Conboy, 2004;Flores, 2010b).
The new model has, therefore, introduced important changes in teacher evaluation (TE) both in its content and form.The original model of TE, published in 2008, was subject to various processes of 'simplification' and 'adaptation' due to a number of critical issues that were identified during the process of its implementation and to teachers' resistance.Thus, two key features of the Portuguese TE system are the wide array of legal texts that have been issued since 2008 and the various adjustments made to the original model in order to deal with some of the most critical issues related to its implementation (see Flores, 2009Flores, , 2010bFlores, , 2012)).
In the OECD report focusing on teacher evaluation in Portugal, a number of issues were identified: the contentious nature of the model; resistance to implementation; difficulties in operationalising a comprehensive model within a short time span and a number of unintended consequences (see Santiago, Roseveare, van Amelsvoort, Manzi, & Matthews, 2009).The same report highlights the need for a balance between improvement and accountability; the need to strengthen TE for improvement purposes, providing links between developmental evaluation and career-progression evaluation; the need to develop national common criteria with adaptation at the school context and to differentiate criteria according to the stage of the career and the type of education; and the importance of relying on three instruments: classroom observation, self-evaluation and teacher portfolio.The report concludes that there was a need to invest in training provided to teachers, to maintain quotas on ratings until the maturity of the system renders them unnecessary (i.e.only a small % of teachers may be rated as 'excellent' or 'very good' which is, normally, dependent upon the external evaluation of the school in which they work).Despite the importance of this report in identifying the shortcomings of new TE system in Portugal, further research is needed in order to better understand and improve its implementation in Portuguese schools.Not only is it essential to consider the ways in which both appraisers and appraises look at the appraisal process and the relationship between them (Chow et al., 2002), but it is also important to take into account the ways in which schools and head teachers put a given policy into practice as well as the nature and the purposes of the TE system itself and its implications for both teacher and school development (Flores, 2009(Flores, , 2010a)).
Data reported in this paper were collected at the end of the second cycle of TE implemented in Portugal (2010Portugal ( -2011)), and head teachers' perceptions were drawn from the system in place at that time (Flores, 2010b(Flores, , 2012)).At that time, between October 2010 and October 2012, National standards for TE were in place and included a five-level scale (excellent, very good, good, satisfactory and unsatisfactory) based upon the following dimensions: (a) professional, social and ethical; (b) development of teaching and learning; (c) participation in school activities and relationship with the community; (d) lifelong training and professional development.Self-evaluation was a key element in the appraisal process with classroom observation not mandatory.In each school or cluster of schools, a committee for coordinating TE conducted the process along with an appraiser designated by a teacher in the same department as the appraisee.The department coordinator was responsible for coordinating and supervising the appraisers' work in his/her department.The evaluation instruments included a self-evaluation report and the global form to be filled in during the appraisal process.It is important to note, however, the existence of a quota system.In order to fully understand the problems and the challenges of implementing a 'new' and 'changing' TE policy it is, therefore, important to look at the perceptions of head teachers as key players in mediating its implementation in context.

The purpose of the study
This paper draws upon data from a wider piece of research (see Flores, 2009Flores, , 2010bFlores, , 2012) ) aiming at looking at the implementation of a new teacher evaluation (TE) policy and its perceived effects on teacher and school improvement.In this paper, the following questions are addressed: (1) How do head teachers perceive the new TE policy?(2) How do head teachers rate different aspects of its implementation?(3) Do head teachers perceive that the policy has impacted teachers and the school?
If so, how?

Methods
Data were collected through a questionnaire which was emailed to school head teachers in Portugal.The questionnaire was designed based upon earlier work within the context of a broader research project (see Flores, 2009Flores, , 2012)).It included both closed and open-ended questions covering the following main dimensions: (a) motivation and job satisfaction; (b) purposes and focus of teacher appraisal; (c) key features of the appraisal system; (d) the implementation process of teacher appraisal (procedures, appraisers, criteria and instruments); and (e) perceived effects of the new policy in schools.
Of the 400 responses, 134 were received between October and November of 2011.The aim of the project was to shed additional light upon the complexities and dynamics of the process of implementation of a new TE policy in Portugal, as it embodied significant changes in terms of the nature and role of the appraisers and the criteria and procedures for TE in addition to the effects upon teacher career progression.The study reported in this paper analyses the head teachers' views of the ways in which their schools have been experimenting with and responding to the new TE policy.

Participants and data analysis
As for the participants, 54.9% of the head teachers are male.Most of them held a Licenciatura degree (57.5%), but 28.3% held a master's degree and 12.6% held other postgraduate degree or specialisation.In terms of experience, the participants served as head teachers in their schools between 1 and 30 years, although most of them had between 2 and 5 years and 6-19 years in their roles.Most of the head teachers worked in urban schools (42.3%), 29.2% worked in rural schools and 28.5% in suburban schools.
Quantitative data were analysed using SPSS (19.0).Open-ended questions were inductively analysed and substantive themes were identified.In the process of analysis of qualitative data, an inductive approach was used, and substantive themes were defined as they emerged from the data.The process of data analysis was undertaken according to two phases: a vertical analysis (Miles & Huberman, 1994) according to which each of the participants' responses was analysed separately.A second phase was then carried out according to a comparative or horizontal analysis (cross-case analysis) (Miles & Huberman, 1994) in order to look for similarities as well as differences.Validation by another researcher was also used as another way of ensuring the trustworthiness of the process of data analysis and coding.

Perceptions of head teachers about the purpose of new TE policy
In general, head teachers held a positive view of TE, but tended to value more formative rather than summative purposes.Table 1 summarises head teachers' views of the main purposes of TE.When asked about the main purposes for TE, they agreed that it should primarily aim at providing useful information for teachers to improve their performance (89.6%), to enhance teachers' reflection on their practice (88.1%) and to identify teachers' professional development needs (60.5%).Although they recognised that teachers are entitled to have their performance appraised (87.3%), they disagreed that TE should aim primarily at making managerial decisions (73.9%).This perspective is in line with teachers' views of the purpose of TE as they tend to value the formative dimension and to reject the idea of TE to be used for accountability and managerial decisions (see Flores, 2012).
The head teachers participating in this study agreed that TE is essential to raise the standards of teaching and learning (69.4%) and should be both for professional development and accountability purposes (76.9%).This view is in contrast to teachers' views who held more sceptical perspective in regard to these two items (see Flores, 2012).Nevertheless, the head teachers participating in this study disagree that the system for TE is reliable and valid (48.8%), and that it is fair (61.9%) but they find it useful for improving teachers' performance (55.3%), as it makes teachers more accountable for their work (50.4%).They also stated that it is useful for school improvement (57.2%).

TE and the role of appraisers
Peer evaluation has been one of the most critical issues of the Portuguese TE system.Interestingly, when asked about the appraisers, head teachers disagreed that the TE process should be done mainly by the teachers themselves (56.4%).Additionally, 45.1% of the head teachers participating in this study disagreed that TE should be conducted mainly by the head teacher.Some of them stated that TE should be done mainly by more experienced teachers (ex.Heads of department), whilst others agreed that it should be conducted by external elements/appraisers to the school (see Table 2).However, the majority of the participants agreed that TE should be conducted by both external and internal elements/ appraisers to the school (69.9%).This is actually the current situation in Portugal.Instead of internal appraisers only, the system in place includes an internal and an external dimension, which was introduced in 2012.

Critical issues in implementing a new TE policy at school
The head teachers held positive views in regard to different elements of the new system, which is in sharp contrast to the teachers' perspective in an earlier study who were more negative and sceptical (see Flores, 2012).In general, the head teachers agree that in their school existing instruments for TE are adequate (53.8%) that criteria and indicators are appropriate (53%) and they disagree that existing procedures are threatening for teachers (57.1%).They also agree that, in general, the appraisal process in their school allows teachers to explain decisions and actions (53.8%) and that it makes teachers more aware of their strengths and weaknesses as teachers (61.2%).The head teachers also felt that teachers' work was recognised in their school (75.2%).However, they are in disagreement in regard to the number of classroom observation which they find inadequate (67.1%) and with the existence of the necessary resources (including time) to undertake teacher performance evaluation (76.7%).
In addition, when asked about other issues related to the process of implementation of the TE policy in their school, the participants have shown mixed-feelings.They had different views in regard to the appraisers' knowledge and competencies to appraise teachers, to the role of TE in encouraging teachers to reflect on their teaching and to the consideration of the school context in implementing the evaluation criteria.Issues such as the mediating role of school leadership and of school culture explain these different views.Encouraging teacher participation, unpacking the implications and possibilities of the new TE policy through discussions and debates at school were some of the strategies developed by the head teachers: We have negotiated and reflected with the teachers in order to avoid the negative effects that happened in other schools in order to maintain the good school climate.
As the school administration, we tried to do our best to avoid conflicts at school.We tried to involved teachers in discussing issues of policy implementation at school.However, the participating head teachers hold different views about the existence of a culture of evaluation in their school (29.1% disagree, 34.3% do not agree nor disagree and 36.6% agree) and about the usefulness of feedback given by the appraisers (42.1% agree, 36.1% do not agree nor disagree and 21.8% disagree).Although 45.5% of the head teachers agree that there is a good relationship between appraisers and appraisees, 38.1% do not agree nor disagree and 16.4% disagree.Yet, they disagree that in their school the appraisers have adequate training to evaluate teachers (67.4%).However, and not surprisingly, they agree that as head teachers they have the required knowledge and competencies to appraise teachers (57.5%) (see Table 3).

Perceived effects of the new policy on teacher and school improvement
When asked about the perceived effects of the implementation of the new policy, the head teachers depicted a rather negative view.The vast majority of them agreed that TE has led to tensions amongst staff (85.7%) (see Table 4).The participants also agree that the implementation of the new TE has increased the bureaucratic work at school (85.8%).They are sceptical about the usefulness of evaluations to direct teachers' professional development and in general they were not happy with the quality of the support for teachers' professional development as a result of the TE process (see Table 4).
When asked about the effects of the implementation of the TE system on their school, the participants' responses were rather negative as they stated that they were not, in general, happy with the effects of TE in terms of rewards (72.9%) nor with the effects of TE as far as improving teaching quality is concerned (52.3%).Indeed, when asked about the perceived effects of the new TE on teachers' professional development, they spoke of 'no effect' (47%), although 30.3% stated that there were positive effects and 22.7% reported 'negative effects' (see Table 5).
The head teachers who had a negative view of the effects of the TE policy on teacher continuing professional development (CPD) spoke of a number of reasons.Issues such as lack of teachers' willingness to invest in their CPD, lack of motivation to work collaboratively from the part of the teachers, lack of reliability of the TE system itself, the ineffective way through which the new system was implemented and communicated to the teaching profession are at the forefront of their accounts: I don't see the majority of the teachers investing in their professional development and I don't notice any change in the working methodologies either.It hasn't been possible to implement a logic of working in which sharing best practice is a priority yet… I think that professional development has to do with teacher attitude and behaviour in the profession rather than with the TE system.It may be possible to identify teachers who have great ratings in their appraisal process but they don't care about their professional development.
I think that the most striking feature is that the TE system is not reliable, that's why there are no effects whatsoever.
The model has started wrong from the very beginning.It was poorly prepared, poorly presented and poorly developed.Thus, there were no effects at all.
Teachers do not believe in the system and they don't believe in its effects at all.
The more positive head teachers associate TE with a greater demand in regard to In-service training and other professional development opportunities from the part of the teachers.It is important to note that the participation in INSET was a mandatory item in the new TE system.They also spoke of greater teacher engagement in teaching and in school activities and greater commitment to ongoing professional learning: Although teachers feel the constraints and lack of coherence of the model, it is clear that they are now more engaged in their work, they care and they get involved in the teaching learning process and in the school activities.They are also more committed to their professional learning and they engage more in INSET and specialised training.
You notice greater concern with continuous learning and self-training.At the same time, the existence of the evaluation system makes teachers more involved in the school life.
At least teachers are more concerned with choosing training which is more adequate to their curriculum area.
Teachers are more realistic in terms of their training needs and they look for professional development opportunities that help them to overcome their needs.They are in general more concerned with their practice and they reflect and reorient their pedagogical strategies to meet students' learning difficulties.
There is a kind of normative pressure for teachers to engage more in their professional development.
The head teachers who held negative views about the contribution of TE to teacher CPD stress the focus on career progression and the credits to be obtained (25 credits of training per year) rather than on the real concern in regard to CPD.Some of them spoke of the move away from the essential element of TE, which relates to the improvement of teacher's practice and his/her professional growth, and they point to the emphasis on the summative side of the TE rather than on its formative dimension.
Merit is not recognised, so the evaluation process has become a bureaucratic process without implications for teacher professional development.
Teacher evaluation doesn't have any formative effects.It has been simplified over the years and it has also been subject to endless changes in legislation.The appraisers didn't get the best training.Appraisal was linked to career progression and this hasn't promoted an effective building of a culture of evaluation oriented towards teacher professional development and the improvement of teaching practice.
The TE system is linked to career progression and it is implemented by appraisers that, although they are the 'most experienced' teachers, they do not have the required competencies to carry out the appraisal process in its formative dimension.
Bureaucracy has undermined the professional development of teachers.The compliance with meeting the normative requirements has prevailed over professional interests of teachers.
Teacher professional development should be linked to organisational development of the school.Unfortunately, the endless changes in legislation has led to a very different situation.(…) On the top of that there is an increase in administrative work that prevents teachers to look for and to engage in adequate training in the light of their professional needs.Actually there isn't relevant training available for them.As far as the effects on the school are concerned, 42.2% referred to 'negative effects' , 30.5% to 'no effect' and 27.3% to 'positive effects' (see Table 5).The participants are clear when justifying the negative perceptions of the effects of the new TE system on their schools.They spoke of increased individualism amongst teachers, decrease in teacher motivation, the deterioration of the school climate and of professional relationships in general and the emergence of tensions and conflicts amongst teachers, in particular, the increase in bureaucratic work, the lack of direct link between TE and CPD and the issues of unfairness and inequity due to the competitive and summative nature of TE, which is very much associated with the existence of a quota system.The head teachers also reported that the new system has led to move the focus of concern of teachers from teaching to the evaluation system itself, the association between TE and career progression and the endless and contradictory legal texts which have contributed to make their life more difficult in dealing with all the changes related to the implementation of the new TE system.The following quotations illustrate this: TE has contributed to the deterioration of school climate and to the rejection of best practice from the part of teachers.And it has been a tough and tiring job to fight against this lack of motivation.
It has implied too much time and effort with no effects on teachers, I mean, their work in the classroom.(…) As a result of TE, and by contrast to what happened before the implementation of this new system, teachers are less willing to participate in projects and activities at school.TE has led to more individualistic behaviour from the part of the teachers.Teacher collaboration and sharing have been affected a lot in my school.
It has led to divisions amongst teachers, it has turned teachers against each other.
The quota system implies that well-qualified and competent teachers do not reach the highest ratings which leads, in turn, to tensions and lack of motivation and lack of investment in CPD.
If teacher evaluation is to lead to improvement, the endless changes in legislation do not allow any kind of improvement, as you have to start all over again all the time.TE has affected school dynamic as teachers have to focus on more bureaucratic and administrative work with no direct link to student learning.And this has affected school activities and school climate in a negative way.
As TE does not result in teacher professional development, the effects of the new system are negative.Teachers do not recognise themselves in a fragmented, confusing and bureaucratic evaluation system.
The school climate has deteriorated, especially as far as the relationship between appraisers and appraisees is concerned.Also, there has been a deterioration of the relationships amongst appraisees.Competitiveness amongst teachers has increased a lot.
The positive perceptions of some head teachers in regard to the effects of the implementation of the TE policy on their school are associated with greater involvement of teachers in schools' activities, greater concern with teaching practice, more reflection on teaching, tendency to value more joint planning in the light of a common project for the school, more willingness to work collaboratively, greater attention to what is going on at school, greater acceptance of the need to be held accountable and a decrease in teachers' absenteeism: In general, TE has contributed to change the ways in which the school operates, namely as far as Curricular Departments are concerned.You can start to see greater concern with pedagogical practice and student achievement.
Teachers reflect more on their practice and invest in improving it.
Teachers feel more responsible for students' outcomes and try to improve their performance.
Teachers started to recognise the importance of teamwork and some of them started to participate more in school activities.
The existence of a perspective centred on greater accountability has an impact on key elements of education as a public service namely on student achievement, on the quality of the activities undertaken at school and on a greater focus on the goals of the school educational project.
The collaborative work has increased and there is also greater monitoring and supervision from the part of the heads of department and greater reflection on teaching practice.
As far as the perceptions of 'no effect' are concerned, they are mainly related to the role of school leadership in managing tensions and challenges in the enactment of the new policy at school.The effects are not positive neither negative.The school climate has been the same as before the implementation of the evaluation system and teachers keep on working in collaboration.Our job as leadership team was to distinguish between central administration role and our own role in the implementation of the model.We have done what we could to avoid problems at school.We have negotiated and reflected with the teachers in order to avoid the negative effects that happened in other schools in order to maintain the good school climate.But this was a very tiring and tough job, especially taking into account the endless and confusing legal texts that have been published over the last years.
Over the last two cycles of evaluation, we have learned something and we have built an internal process of debates and discussion around teacher evaluation.Of course, there was resistance and conflicts sometimes but we have tried to build a more formative and reflexive process.
I guess some teachers have a wrong opinion about their own performance.They tend to give up if their work is not recognised.
Others spoke of a long process with no impact so far.They also highlight the resistance from the part of teachers who have rejected the model from the very beginning which, in their view, undermines any effect in practice.This is exacerbated by the bulk of endless legislation that has been issued since 2007 related to teacher evaluation: It is a long process and its effects are rather limited.
Teachers do not identify themselves with the model, so there are no effects so far.
Teachers do not see any kind of impact of the appraisal process in practice.
Due to continuous change in the legal framework, it is hard to see any real changes in practice.

Discussion
This paper examined head teachers' views of the implementation of a new policy of TE in Portugal.It adds to existing literature on TE policy and its mediating factors in implementing it in context.As Braun, Ball, Maguire, and Hoskins (2011, p. 585) suggest, 'policies are intimately shaped and influenced by school-specific factors, even though in much central policy-making and research, these sorts of constraints, pressures and enablers of policy enactments tend to be neglected' .A number of tensions, contradictions and dilemmas have been identified by the head teachers participating in this study, most of which are related to the key features of the TE system itself.Others are related to issues pertaining to the appraisers and to the implementation process with impact on the school climate.
The participants tend to value more formative rather than summative purposes, a finding consistent with earlier work (Flores, 2012;Tuytens & Devos, 2018).They recognise the importance of TE to provide useful information for teachers to improve their performance, to enhance teachers' reflection on their practice and to identify teachers' professional development needs.However, the head teachers are sceptical about the key features of the TE model which they associate with unclear procedures and features that proved to be problematic leading to various processes of 'adjustments' and 'simplification' (see Flores, 2009Flores, , 2010bFlores, , 2012)).As such, a number of challenges do remain, which are summarised in Figure 1.The problems that persist in the implementation of the TE policy give rise to a number of dilemmas that the head teachers have to face in their daily professional lives.

Persisting challenges of policy implementation
Despite the processes of adjustment and simplification, a number of critical issues remain.One problem relates to the key features of the model which entails excessive workload and bureaucracy, associated with its summative dimension and endless legislation, and also with the existence of a quota system (in order to differentiate teachers).Added to these is the lack of adequate training for both appraisers and appraises, which makes the process of implementation and sense making in context more difficult.Of particular complexity and controversy is peer evaluation which is still a critical feature identified by both teachers and head teachers.This has been soften with the introduction of an external appraiser (from another school teaching the same subject) in 2012.Yet, peer evaluation remains a critical aspect as this research and earlier empirical studies have shown (e.g.Flores, 2009Flores, , 2012;;Araújo, 2011;Lapo & Flores, 2016), which, to some extent, resonate with issues of professionalism and teachers' professional culture.Issues of competitiveness, individualism and deterioration of school climate have been identified, even though some head teachers identify some examples of greater acceptance of the need to foster collaboration at school.Within this context, dealing with unintended effects of the TE policy, building trust in the TE system and nurturing a culture of evaluation at school are also challenges that head teachers face in their schools.Another persisting challenge that remains is the articulation between teacher evaluation and school evaluation, i.e. looking at the evaluation process in a more comprehensive and systemic manner.Similarly, the need to link TE and the improvement of teaching and learning in schools and classrooms is also required in order for TE to be effective and to improve teachers' performance and, consequently, student learning.

Dilemmas in implementing the new policy
The implementation of a new policy is problematic (Timperley & Robinson, 1997) as it entails a rather dynamic, complex and sometimes controversial process (Midddlewood & Cardno, 2001;Avalos, 2004;Avalos & Assael, 2006;Asseal & Pavez, 2008;Flores, 2010a), related to a number of dilemmas head teachers have to face in context.As Braun et al. (2011, p. 596) state, 'The way that policy problems are solved in context is a multifaceted, iterative process.The rich 'underlife' and micropolitics of individual schools means that policies will be differently interpreted (or 'read'), and differently worked into and against current practices, sometimes simultaneously' .
This paper adds to a number of problems and dilemmas identified by Flores and Derrington (2017).This study provides evidence of the complexity of the dilemmas of the head teachers as they have to match the (competing) expectations of both Central Administration and teachers (which entail dealing with issues of performativity and summative evaluations and at the same time handling teachers' expectations in regard to support and formative purposes).For head teachers combining short and long-term goals is not always an easy job as they need to deal with summative evaluations for a given period of time (for one year contract teachers means every year).Fostering a culture of evaluation at school in the long run also remains a challenge.This requires a balance between the summative and bureaucratic requirements of the TE system and the needs of the teachers as well as monitoring the implementation of the procedures of the TE system and trying to enhance the confidence of the teachers in order to minimise the negative (and unintended) effects of the policy.Similarly, they need to be able to articulate issues of performativity and documentation of teachers' work and, at the same time, fostering CPD and school improvement.In other words, the implication is that head teachers must be creative in dealing with competiveness and at the same time creating a positive school climate.As Braun et al. (2011, p. 549) argue, 'policy enactment involves creative processes of interpretation and recontextualisation -that is, the translation through reading, writing and talking of text into action and the abstractions of policy ideas into contextualised practices' .Thus, strategies such as promoting teacher growth while making summative decisions, focusing on student learning while complying with directives related to summative assessments of teachers and maintaining a positive school climate while mediating negativity of the teacher policy have been identified (see, for instance, Flores & Derrington, 2017).
In Belgium, Tuytens and Devos (2010), drawing upon the characteristics identified by Fullan (2001) as affecting policy implementation, concluded that the structure a head teacher provides, along with the trust teachers have in their head teacher, is of key importance to teachers' perceptions of the policy's practicality.The same authors also identified vision as one of the most important dimensions influencing teachers' perceptions of the policy as it affects two essential policy characteristics: need and practicality.Empirical work has also shown that leadership directly influences feedback's usefulness and, thus, indirectly influences teachers' professional learning (Tuytens & Devos, 2011).

Perceived effects on teachers and schools
The effects of such a complex and controversial TE policy, which was subject to criticisms, including teachers' demonstrations and strikes, and whose original model has been (re) adapted over and over again, remain to be seen.However, the head teachers participating in this study described a rather negative view of the effects of the policy on teacher and school development so far.They spoke of tensions amongst staff and increased bureaucratic work at school.They were not happy with the quality of the support for teachers' professional development as a result of the TE.They were also critical of the effects of TE as far as improving teaching quality is concerned.Most of them identified 'no effects' and 'negative effects' of TE policy on teacher and school development.Issues such as increased individualism amongst teachers, decrease in teacher motivation, the deterioration of the school climate and of professional relationships in general and the emergence of tensions and conflicts amongst teachers, in particular, were recurrent elements in the participants' accounts.They also highlighted the lack of direct link between TE and CPD.Issues of unfairness and inequity due to the competitive and summative nature of TE, which is very much associated with the existence of a quota system, are at the forefront of their accounts.These findings lend support to earlier work carried out in Portugal (Araújo, 2011;Lapo & Flores, 2016).
Within a context of resistance and rejection of the model of TE from the very beginning, it is not surprising that the main perceived effects are rather negative.However, some head teachers did identify some positive features, such as greater involvement of teachers in schools' activities, greater concern with teaching practice, more reflection on teaching, tendency to value more joint planning in the light of a common project for the school, more willingness to work collaboratively, greater attention to what is going on at school, greater acceptance of the need to be held accountable and a decrease in teachers' absenteeism.These features have been associated with the role of school leadership in managing tensions and challenges in the enactment of the new policy at school and in minimising its negative effects as they (re)interpret the policy in context.

Ways forward
In this context, it is clear that more needs to be done in order to improve TE policy.Findings from this study highlight a number of issues that deserve further consideration if TE is to be successful.One critical aspect is the need to rethink the role of classroom observation in order to be considered the core element in TE.In addition, it is important to enhance the supervisory role of school leadership, mainly in what concerns its pedagogical dimension.In fact, existing literature has emphasised the need to reinforce supervision within the context of TE in systems in which head teachers are the sole or one of the appraisers.The study by Derrington and Campbell (2015, p. 323) has demonstrated the relevance of 'supervisory practices by providing supportive interventions, particularly with early attentiveness to relational and operational concerns' , which is even more crucial in contexts marked by mandated, high-accountability systems.
Consequently, there is a need to improve the quality of feedback provided to teachers within TE process as well as to provide them with relevant CPD opportunities.This is line with earlier work carried out in other contexts (e.g.Su et al., 2017).Tuytens and Devos (2017) also found that teachers considered the feedback they received from school leaders useful.The authors argue for the importance of school organisational characteristics and an integrated leadership approach for the feedback response of teachers.
Thus, it is essential to negotiate and clarify the roles of the TE stakeholders at school, mainly those of the appraisers, as well as the school leaders, in order to overcome resistance and to enhance acceptance and legitimacy of appraisers.In order for this to happen more needs to be done in regard to the recruitment of appraisers and to provision of relevant training in knowledge and skills related to TE as well as in providing useful and timely feedback.Another critical element that needs to be dealt with is associated with the importance of mediating the side-effects of TE at the school, department and teacher-level and, therefore, to develop creative and flexible strategies to implement such a complex (and controversial) model of TE.Training in TE is needed not only for appraisers and appraisees but also for school leaders, although the head teachers participating in this study claim that they have the necessary knowledge and skills to implement the new policy of TE at school (see also, Lavinge & Chamberlain, 2017).
Change is a complex process.In the case of TE policy, change is even more complex and challenging due to the wide array of dimensions involved, to the expectations of different stakeholders and to the intended (and unintended) outcomes.The complexity and controversy of TE is even more crucial when it entails deep changes in teacher professional cultures.Peer evaluation and the role of the head teacher in mediating the most critical features of TE are key to understand the development and adjustment of TE policy in the Portuguese context.
As Sachs (2016) stresses, teacher professionalism is shaped by external environments, and, in times of increased accountability and regulation, it is possible to identify various discourses, gaining legitimacy and impact on how professionalism is understood and enacted.In many parts of the world, it is possible to identify issues such as the rise of performative cultures through increased accountability and imposition of standards (Sachs, 2016) and the compliance of teachers with the demands of performance policies and the tyranny of audit (Lo, 2012).A national survey carried out in Portugal in 2014 (see Flores, 2014) shows that teachers seem to value more the individual and collective understandings of their professionalism in detriment to its managerial dimension in times marked by intensive school reform and austerity measures.Day and Sachs (2004, 7) analyse the characteristics of managerial and democratic professionalism, the former being described as 'system driven/ ends, external regulation, drives reform agenda, political ends, competitive market driven, and control/compliancy' , and the latter as 'profession driven/ends, professional regulation, complements and moves beyond reform agenda, professional development, collegial and profession driven, and activism' .Teachers may, therefore, be situated in the context of a constrained professionalism as their autonomy in classroom contexts may be influenced by a standardised curriculum and national exams and by decisions to be made under contextual pressures and time demands (Willis & Haymore Sandholtz, 2009).Teachers may also make use of their professional space even in contexts marked by growing standardisation (Oolbekkink-Marchand, Hadar, Smith, & Helleve, 2017).
This study entails a number of limitations.First, the number of participants is limited.Second, the study is based on self-reported questionnaire data.And third, it would be important to include other sources of data as well as other stakeholders, namely teacher appraisers, appraisees and those responsible for implementing the TE policy at school.Nevertheless, this study makes a contribution to the field as it sheds additional light on the ways in which head teachers perceive and experience the implementation of a new policy in their schools, particularly in a context marked by a centralised and bureaucratic education system.It also suggests avenues for further research.It would be important to further explore the ways in which TE is perceived by other stakeholders, namely students, parents and teachers.Also, it would be interesting to examine the articulation between TE and other policies, specifically in regard to CPD and school evaluation, both internal and external.If the interest of the child is at the centre of education, it is, therefore, essential to look at the ways in which the magnitude and complexity of policy implementation effects (both intended and unintended) influence the quality of teaching and learning in schools and classrooms.

Figure 1 .
Figure 1.The implementation of Te policy in context from the head teachers' views.

Table 1 .
purposes of teacher performance evaluation.

Table 3 .
aspects related to the implementation of the Tpa in schools.

Table 4 .
perceived effects of Te on various aspects of teachers and schools.

Table 5 .
perceived effects of Te on teacher and school development.