An emic perspective toward challenges and solutions of self- and peer-assessment in writing courses

Abstract

Self-assessment and peer-assessment have been appraised in a large number of studies, aiming at investigating their effectiveness in different contexts. However, the challenges of implementing self-assessment and peer-assessment have been investigated rather sporadically. Given that, the purpose of this study was to explore the challenges and probable solutions of implementing self-assessment and peer-assessment by delving into the perspectives of English language teaching (ELT) teachers and ELT students. In this study we used an emic methodology to examine the perspective of 11 ELT teachers and 56 undergraduate and graduate ELT students. We used MAXQDA to analyze the interviews and open-ended questions. The results reveled that on the part of ELT students, self and peer’s subjective judgment, learners’ lack of assessment literacy, and instructional problems were the most important challenges of implementing self-assessment and peer-assessment in writing courses. ELT teachers believed that these assessment types are not feasible in all cultures. They also admitted that ELT teachers’ lack of assessment literacy is a challenge in implementing these assessment types. As for the probable solutions, both ELT teachers and students posited that ELT teachers need to increase their assessment knowledge and to participate in teacher training courses. Whereas ELT teachers believed that learner autonomy could be a solution, ELT students were against it. The article’s conclusion bears some implications for ELT teachers while implementing self-assessment and peer-assessment in writing courses.

Introduction

Due to the utmost importance of assessment “to shape every part of the student learning experience” (Orsmond et al. 2000, p. 24), it has undergone a long way from the traditional assessment to the alternative one. The shifting and twisting aspects existing within the realm of language assessment are the result of considering “consequential validity” in language assessment (Gielen et al. 2003). Consequential validity is defined by Messick (1994) as one of validity types addressing the concept of validity in terms of the impact of tests on teaching and learning. One aspect of consequential validity is to involve learners in the process of learning and assessing. It is thought that the involvement of students in the process of assessment may enhance their learning opportunity (Black et al. 2003; Leahy et al. 2005). This involvement can be obtained through the outlets of alternative assessment: self-assessment and peer-assessment (De Grez et al. 2012).

Self-assessment and peer-assessment are proved as promising types of alternative assessment (Esfandiari and Myford 2013; Harris et al. 2014; Ross 2006). These assessment types are thought to motivate learners to be more responsible for their own learning, to integrate learning and assessing, and to prepare authentic activities to make the assessor’s judgment more accurate (Hargreaves et al. 2002). Moreover, Birjandi and Hadidi Tamjid (2012) emphasize that self-assessment and peer-assessment are the outcomes of changing in the curricula from teacher-centered to a logical interest in learner-centered curricula.

Self-assessment is, theoretically, in line with the model proposed by Zimmerman (2008) in which learners go through different phases of development to reach the final phase which is self-assessment. In the first phase of this process, learners will be obsessed with self-motivational beliefs, which is “where am I going?” Then, learners will obtain self-control over the task by practicing, which is “how am I going?” Finally, students will give self-reflection on their practiced tasks and self-evaluate their work. This is the phase in which the learners ask “where to next?” Schunk (1996) states that self-assessment triggers the ownership sensation of learning process in the learners and increases their motivation and self-efficacy. Topping (2003, p. 59) posits that self-assessment “can lead to interaction, such as when discussing assessment criteria or when the learner is called upon to justify their self-assessment to a peer or professional tutor.”

The theoretical underpinnings of peer-assessment imply extra sources of feedback on students’ work. Learners can provide feedback on their peers’ work and be provided with feedback by them. Hattie (2009) think of this type of feedback as more accessible in comparison with mere teacher provided feedback. On top of this, the process of feedback provision by the learners leads to their better understanding of their own work and improves collaboration among them (Topping 2013). Furthermore, peer-assessment is associated with peer assisted learning (Topping 2003). Researchers (eg Falchikov 2001; Topping and Ehly 1998) consider peer-assessment in line with the principles of peer assisted learning and argue for its effectiveness in learning within different contexts.

Although many studies showed the effectiveness of self-assessment and peer-assessment (eg Andrade et al. 2010; Chen 2008; Brown 2005; Topping 2010; Pope 2001), some questions remain about the validity, reliability, and practicality of these assessment types (Ross 2006).

Issues in implementing self-assessment

The reliability and validity of self-assessment are postulated as controversial (Topping 2003). Learners’ overestimation and/or underestimation of their work make their assessment open to doubt (Matsuno 2009). In respect of the reliability of self-assessment, researchers (eg Blatchford 1997; Ross 2006; Sung et al. 2005) conclude that consistency in self-assessment can be obtained across tasks and items over short time. However, this consistency is not applicable when longer time period is at focus. One reason for this may be the overestimation or underestimation which is done by the students not only about their ability but also about their self-assessment (Frye et al. 1992; Ward et al. 2002; Zoller and Ben-Chaim 1997).

Ross (2006, p. 3) states that “validity in self-assessment typically means agreement with teacher judgments … or peer rankings.” The studies done to address the validity of self-assessment yield mixed results. Researchers (eg, Boud and Falchikov 1989; Sung et al. 2005) showed while there exist discrepancies between self-assessment and peer-assessment, the agreement of these two assessment types are more than that of self-assessment and teacher-assessment. This can indicate that the students, either in self-assessment or peer-assessment, utilize different criteria from that of the teachers (Ross 2006).

Topping (2003) posits the reliability and validity of teacher-assessment is not higher but self-assessment is a little lower and more varied. Consequently, it seems that students tend to overestimate their abilities while doing self-assessment. The reviewed studies show that the “reliability and validity of self-assessment tends to be higher in relation to the ability of the learner, the amount of scaffolding, practice and feedback and the degree of advancement in the course, rather than chronological age” (Topping 2003, p. 65). There are also some other factors which researchers (eg, Fontana and Fernandes 1994; Ninness et al. 1998; Ward et al. 2002) consider to be effective in reliability and validity of self-assessment. In Topping (2003, p. 65) words these factors include “the nature of the subject area, the nature of the product or performance assessed, the nature and clarity of the assessment criteria, the nature of assessment instrumentation, and cultural and gender differences.”

Issues in implementing peer-assessment

Some studies address the reliability and validity of peer-assessment (eg, MacKenzie 2000; Matsuno 2009; Magin and Helmore 2001; Topping et al. 2000). Topping (2003) asserts that over 70 % of the studies referring to the validity and reliability of peer-assessment show an adequate degree of reliability and validity. However, in a meta-analysis done to compare peer-assessment and teacher-assessment marks, Falchikov and Goldfinch (2000) questioned the reliability and validity of peer-assessment. Falchikov and Goldfinch stated that the lower reliability and validity of peer-assessment is due to some factors such as assessing several different individual dimensions, doing peer-assessment on professional practice, and requiring beginners to do peer-assessment. Moreover, Falchikov and Goldfinch believed that experimental studies tend to show lower reliability and validity for peer-assessment in comparison to the research with qualitative design.

Furthermore, in the implementation of peer-assessment ethical challenges may appear (Esfandiari and Myford 2013). In some contexts in which the racial/ethical background of the students who are doing peer-assessment are different the classmate might claim that the peer-assessor’s ratings are biased, possibly creating significant tensions between those two students.” (Esfandiari and Myford 2013, p. 114). Vu and Dall’Alba (2007) reported that students may have an uneasy feeling about their peers assessing their work. Moreover, Freeman (1995) asserted that some peers may not do their responsibility seriously, causing challenges to the meaningfulness of peer-assessment. Orsmond et al. (1996) addressed the applicability of peer-assessment in a way that the criteria of assessing may not be clear for the students, leading them to unjustified assessment. Finally, as Mok (2011) declared some of the students may not be psychologically prepared for doing peer-assessment.

Self-assessment and peer-assessment in language pedagogy

Self-assessment and peer-assessment are implemented in language classrooms for the purpose of developing better understanding of assessment criteria and the assignments (Orsmond et al. 1997), giving students reasons for their obtained grades (Taras 2001), motivating learners to improve L2 learning process, benefiting from peer feedback (Topping 2003), and having real audiences caring about learners’ performance. Given the advantages of self-assessment and peer-assessment some studies were done to examine these advantages in different language skills. Through the lines mentioned below, we summarized self-assessment and peer-assessment in different L2 skills.

As it was stated earlier, there were some issues, among them were reliability and validity, associated with self-assessment and peer-assessment casting doubt on the applicability of these assessment types in all contexts. Brown and Hudson (2002, p. 84), nevertheless, believe that

Some of these problems can be overcome if the descriptions that students are referring to in rating themselves are stated in terms of clear and correct linguistic situations and in terms of exact and precise behaviors that the students are to rate

In the next section we summarize the key findings from research on self-assessment and peer-assessment done in different contexts and for different purposes.

In a study conducted by Patri (2002), the differences among self-, peer-, and teacher-assessment of oral skills were investigated. The participants of Patri (2002) study were undergraduate Chinese students. Training them to be familiar with the criteria, the researcher found more agreement between the teacher-assessment and peer-assessment than that of teacher-assessment and self-assessment. In like manner, Saito and Fujita (2009) investigated the assessment quality of essays assessed by selves, peers and teachers. They reported that the highest agreement was observed between peer-assessment and teacher-assessment. They mentioned some psychological factors such as self-esteem leading to less agreement between self-assessment and peer-assessment and self-assessment and teacher-assessment.

In a study carried out by Cheng and Warren (1999) the attitudes of the learners toward peer-assessment were investigated. By comparing the students’ attitudes toward peer-assessment they found that students did not show very positive attitudes toward peer-assessment. They reasoned their finding to learners’ uncertainty about language proficiency and linguistic competence. Moreover, Leach (2012) reported that students had negative attitudes toward doing self-assessment since they had a fear of being wrong in doing assessment, leading to a sense of reluctance to evaluate their work.

Some of the studies, moreover, have been conducted to address the severity that self-assessor, peer-assessor, and teacher-assessor are exercised while doing assessment. In 2009, Matsuno used Many-Facet Rasch model (MFRM) to examine raters’ severity/leniency in assessing writing. The findings of his study showed that self-assessors were so severe with regard to their performance. Peer-assessors showed very fewer bias interactions than that of self-assessors and teacher-assessors. In a similar vein, Esfandiari and Myford (2013) used MFRM to obtain the severity/ leniency differences among self-, peer-, and teacher-assessors rating EFL essays. After training assessors with a 6-point analytic scale providing 15 assessment criteria, they required the participants to assess essays. The results of their study indicated that teacher-assessors were the most severe while self-assessors were the most lenient. The findings of their study also showed that there was no statistical significant difference between the severity of teacher-assessors and peer-assessors.

Reviewing the related literature and pinpointing the discrepancies existing in the results of the studies conducted on the effectiveness of self-assessment and peer-assessment, we spotted a gap in the literature. We found out that the previous studies conducted on self-assessment and peer-assessment did not pay attention to the challenges existing in implementing self-assessment and peer-assessment. In this study, we wanted to explore the emic perspective, insider’s view, about the challenges and probable solutions of implementing self-assessment and peer-assessment in writing courses. It is worth mentioning that by emic perspective it is meant that we wanted to obtain the perspectives of ELT teachers and students. According to Freeman (1998) the emic perspective provides us with the information about “what insiders know”, the “classroom teachers’ and learners’ perception”, and “participant understandings.” By using an emic perspective, the researchers are able to address the deep beliefs and perspectives of participants about the subject of discussion (Gadsby and Cronin 2012). To pursue this purpose the following research questions were followed:

• 1. What are the challenges and probable solutions of implementing self-assessment and peer-assessment in L2 writing courses from the perspectives of ELT teachers?

• 2. What are the challenges and probable solutions of implementing self-assessment and peer-assessment in L2 writing courses from the perspectives of ELT students?

Method

Participants and sampling

The participants of this study were 11 ELT teachers and 56 ELT students. We used purposive sampling to select our participants. This type of sampling as Ary et al. (2014, p. 456) state is “sufficient to provide insight and understanding about that which they are studying.” Different strategies are used in qualitative research (Marshall and Rossman 2010) to fulfill the principles of purposive sampling. Hence, we used a criterion sampling. Ary et al. (2014, p. 458) posit that “in this type of sampling, the researcher set the criterion and includes all cases that meet that criterion.” Since we desired to follow the emic methodology to obtain the perspectives of ELT teachers and students about the challenges and probable solutions of implementing self-assessment and peer-assessment in the writing courses, we set two criterion. First, the data was gathered from the ELT teachers who have already implemented self-assessment and peer-assessment in their writing courses during their professional carrier. On the part of students, we selected the bachelor and master students in English teaching who have already passed advanced writing and language testing courses. In so doing, we assured that our participants had already realized self-assessment and peer-assessment and the necessary technical terms such as validity and reliability. The teachers participated in our study were all from Iran, having Persian as their first language. Table 1 shows the characteristics of the participants in this study.

Table 1 Characteristics of the participants

Instruments

We used two instruments to collect information from the participants. First, a semi-structured interview was conducted with ELT teachers by the lead researcher. After reviewing the literature on self-assessment and peer-assessment, we reached to four questions to be raised in the interview. These four questions were pinpointing the issues such as the challenges which ELT teachers faced in their writing courses, their perspectives about the reliability and validity of self- and peer-assessment, their students’ reactions toward implementing self- and peed-assessment in their writing courses, and their probable solutions for the challenges they mentioned. Each interview took around 15 min. Ary et al. (2014) believe that asking leading questions can overshadow the final results. Moreover, they require the researchers’ cautious about double-bared questions, which consist of two questions in one single stem. Since we have used MAXQDA to analyze the responses, posing open-ended questions in the interview could help us to tap upon the deep perspectives of our respondents. Considering these principle the interview questions were designed.

We also used a set of open-ended questions to obtain ELT students’ perspectives. Ary et al. (2014) believe that when researchers are probing to tap upon the deep-understandings of participants to find out about unpredictable possible responses they can use open-ended questions. With respect to that, we designed five open-ended questions. It should be noted that the first set of open-ended questions were given to eight undergraduate TEFL students to answer. These students had similar background as the participants of the study. The rationale behind this was twofold. First, as Malone (2011) declares if five graders do not interpret a question in the same way the researchers need to rewrite that question. Moreover, we wanted to know whether the pilot participants had any problems about the technical terms such as validity and reliability while answering the questions. Hence, the pilot participants were asked to put their words about the nature of the questions and what they thought to be changed. After collecting the information from the pilot participants we implemented the necessary changes in the questions for the purpose of the main study. We prepared four open-ended questions addressing different aspects such as the students’ knowledge about self- and peer-assessment, the challenges they faced and their probable solutions for those challenges, what they think about the reliability and validity of these assessment types, and how their teachers implemented self- and peed-assessment in their writing courses.

Data collection and analysis procedures

The interviews were conducted by visiting ELT teachers and posing the interview questions. The interviews were recorded and transcribed for the purpose of analysis. The open-ended questions aiming to address the perspectives of ELT students about the challenges and probable solutions of implementing self-assessment and peer-assessment were sent to the participants electronically via their email address. The participants answering the open-ended questions were asked to write as much as they thought it sufficed. In so doing, we wanted to reach data saturation (DS): the point when no new information is appearing (Dörnyei 2007). Since in this type of data collection many participants do not send back the responses we implemented the principles of DS. We sent the open-ended questions to 78 MA and BA English majored students; however, after three weeks and when 56 respondents sent back their responses we came to the conclusion that there is no new data in the responses; consequently, we started our data analysis.

The data analysis was done with the help of MAXQDA 10 software. This software is a package used for analyzing qualitative data. After transcribing the interviews to text format and entering the responses of ELT students to the open-ended questions to MAXQDA software we started data analysis by drawing the codes and themes. By creating codes and obtaining the main themes we examined the responses to see the main challenges and probable solutions which the respondents mentioned.

According to Ary et al. (2014) one main threat to the reliability of the content analysis research is the disagreement in coders’ codification. To address this issue we used inter-coder agreement which is provided by MAXQDA. Consequently, 50 % of the data was given to another coder to codify based on the introduced codes. The results of inter-coder agreement correlation was 92 %.

Results and Discussion

ELT Teachers’ perspectives about the challenges of self-assessment and peer-assessment in L2 writing courses

The responses of 11 ELT teachers regarding the challenges of implementing self-assessment and peer-assessment in writing courses were analyzed using MAXQDA 10. Ten codes were identified in the responses of the ELT teachers in that respect. Figure 1 illustrates the output of MAXQDA, addressing the perceptions of the ELT teachers.

Fig. 1

ELT Teachers’ perceptions about challenges and probable solutions of self/peer-assessment

The first theme we found in several ELT teachers’ responses about the challenges involving in self-assessment and peer assessment was the fact that self-assessment and peer assessment are culture-bound and they cannot be implemented in all cultures.

I think one of the foremost challenges of implementing self-assessment and peer-assessment is that it is not possible in all cultures to put the responsibility of on the shoulder of self-assessors and peer-assessors. This is so due to the fact in some cultures like ours learners try to overestimate their abilities in language learning in general and writing ability in particular. I think the origin of this overestimation should be searched in other aspects of our learners’ life. The culture in which our learners have grown up is the one which tries to teach them overestimation on things so this is not far from consideration to have L2 students who overestimate their writing ability…

Moreover, some of the teachers believed that there are some conflicts among students which affected their methods of assessing while doing peer-assessment.

… the thing is you may find conflicts among learners which affect their criteria for doing peer-assessment. Among those conflicts, as I saw sporadically, is the jealousy some learners have to others. I feel this will affect the quality of peer-assessment…

The responses of the ELT teachers showed that there is a challenge caused by the nature of culture in which L2 learners grew up. Brown (2005) and Matsuno (2009) reported that cultural background may affect the way L2 learners implement self-assessment and peer-assessment. For instance, the Japanese L2 learners seem to underestimate their ability while they are doing self-assessment (Matsuno 2009). Brown (2005, p. 186) direct us to the social limits as “modesty and ego.” It is believed that from the perspective of ELT teachers due to the cultural background and social constrains of EFL context in which English is taught as a foreign language, there will be some challenges toward implementing self-assessment and peer-assessment. Moreover, Casanave (2014) stated that when doing assessment, learners follow the culture of assessment which the educational system builds and learners are accustomed to.

ELT teachers were also asked about the reliability and validity of self-assessment and peer-assessment in writing courses. Although ELT teachers had different ideas in this regard, several codes were found that showed why they thought self-assessment and peer-assessment are not reliable and valid. Lack of assessment literacy upon the ELT teachers and ELT students is one of the themes we reached through Fig. 1.

… for the reliability and validity of self/peer-assessment I think since our L2 teachers are not that much familiar with language literacy assessment they are not able to implement these assessment types in appropriate way which lead to reliability and validity issues. Moreover, unaware L2 teachers of assessment literacy cannot instruct their learners how to implement self/peer-assessment so the learners will be illiterate in terms of self/peer-assessment, too.

Researchers (eg, Abell and Siegel 2011; Fulcher 2012; Inbar-Lourie 2013; Popham 2009; Vogt and Tsagari 2014) put emphasis on the ELT teachers’ and L2 learners’ familiarity with language assessment literacy. More often than not, it is thought that the role of the teachers is more important since they have to make their learners familiar with assessment literacy if they want to implement self-assessment and peer-assessment. Boyles (2005) believes that ELT teachers need to obtain some competencies in order to develop language assessment literacy. In words of Malone (2013, p. 331) “[t]hese competencies include the ability to understand appropriate testing practices, utilize various means of assessment, interpret and analyze assessment results, respond appropriately to the results and their meanings, and use the results in their teaching.”

Given the importance of ELT teachers’ familiarity with self-assessment and peer-assessment literacy, it seems that from the perspective of ELT teachers the lack of this literacy is a two-fold problem. On the part of ELT teachers, the lack of literacy in this domain will lead to teachers’ inability in implementing self-assessment and peer-assessment in their writing courses. This in its place, then, causes problems in implementing self-assessment and peer-assessment on the part of L2 learners. For instance, if L2 learners are not provided with the necessary literacy, let say how to use rating scale to self-assess and peer-assess, they are impotent in implementing self-assessment and peer-assessment.

Some of the ELT teachers believed that one of the challenges existing in implementing self-assessment and peer-assessment in writing courses is that the L2 learners are reluctant toward these assessment types. This unwillingness might be due to observing no progress in their writing performance even by using self-assessment and peer-assessment.

… I have seen that the students are reluctant and do not participate in this writing class. I think while I did my best to consider all the required factors involving in self-assessment and peer-assessment in my writing class the issue was something else. The students did not see any progress in their writing performance and this impeded them from doing the activities willingly.

Observing no progress in their writing ability makes L2 learners less self-confidence. Falout et al. (2009) stated that in this situation learners are less willing to use the techniques they use in the classrooms since they do not see the outcomes of those techniques in their learning progress.

The above mentioned challenges on the part of L2 learners lead to some issues regarding assessment fairness. It seems that lack of literacy in doing self-assessment and peer-assessment along with students’ overestimation and underestimation of themselves and their peers lead to the fairness problems. This means that in their assessment, either self-assessment or peer-assessment, they do not show a fair performance.

Using MAXQDA MAXMap we designed a Code-Subcode-Segment model of the challenges of self-assessment and peer-assessment stated by ELT teachers. Figure 2 illustrates the Code-Subcode-Segment model.

Fig. 2

Challenge codifications: ELT Teachers' perceptions (MAXMap)

ELT Teachers’ perceptions about the probable solutions to the challenges of self-assessment and peer-assessment in L2 writing courses

The ELT teachers participating in this study were also asked about the probable solutions they considered for tackling the challenges of self-assessment and peer-assessment in writing courses. We identified three main themes in the responses of the ELT teachers: Using the results of the research and action research addressing self-assessment and peer-assessment, participating in teacher training courses to foster their knowledge of implementing self-assessment and peer-assessment, and being more in line with the principles of post-method principles.

Some of the ELT teachers believed that there is a gap between the research done to show how to implement self-assessment and peer-assessment and the way ELT teachers implement these assessment types in writing courses.

One of the solutions to the challenges of implementing self- and peer-assessment is the negotiation of ELT teachers with the research done about these assessment types; suggesting that ELT teachers can first quench their theoretical issues in this regard and find a tune stance with respect to practical issues…

Another ELT teacher tapped upon the importance of using the research results in general and doing action research in particular:

Although the challenges are not easy to be removed, I think one way is the research resort. Our ELT teachers need to be more engaged in SLA research to resolve their problems. Action research is very much necessary for the ELT teachers, as well. If they go upon regular action research for the issues they face in the classrooms they may find solutions not only for implementing self-assessment and peer-assessment of their writing courses but also other classroom related problems.

Heilbronn (2008) referred to the relationship between research and practice as “evidence-based practice.” Nassaji (2012) stated that evidence-based practice “is based on systematic research evidence” (p. 338). Hubbard and Power (1999, p. 2) defined action research as the “research that is initiated and carried out by teachers in their classrooms and schools.” Action research, as posited by Price and Valli (2005), implies change in the reflection of teachers on their experiences. Doing action research helps ELT teachers to find the gaps in their instruction and take necessary measures to bridge those gaps. It is believed that ELT teachers are not merely responsible to teach English; however, they are responsible to do research to find and solve the existing problems in their classrooms (Davies 1999). Given that, one solution from the perspectives of ELT teachers is to do action research in writing courses in which they practice self-assessment and peer-assessment to find probable solutions for the existing problems.

Teacher training courses was a broad theme which the ELT teachers participating in this study called as another resource for responding to the challenges of self-assessment and peer-assessment. What we found from analyzing the teachers’ perspectives toward this theme was that ELT teachers need to take part in teacher training courses in terms of pre- and in-service to be more familiar with the principles of doing self-assessment and peer-assessment in writing courses. The pre- and in-service courses, as a kind of teacher training course, will help ELT teachers to gain knowledge on issues they encounter in L2 classrooms (Hedgcock 2002) and contribute to their ability in doing action research in their L2 classes (Nassaji 2012). By participating in pre- and in-service courses ELT teachers will be able to gain knowledge or what previously mentioned as assessment literacy in self-assessment and peer-assessment to tackle the challenges they may confront in their writing courses while implementing self-assessment and peer-assessment.

Finally, the ELT teachers referred to the principles of post-method pedagogy as a solution to be called upon against the challenges of self-assessment and peer-assessment. Some of them believed that by implementing the principles of post-method pedagogy such as learner-centeredness, learner autonomy and practicing upon those principles the ELT teachers can find solutions for their problems in self-assessment and peer-assessment writing courses.

Learner-centeredness and learner autonomy [as principles of post-method pedagogy] is a way to remove the challenges that ELT teachers may have for implementing self-assessment and peer-assessment. We need to teach learners in way that they will be responsible for their learning and in case of self- and peer-assessment we should do the same to make them responsible for their assessment.

The ELT teachers posited that the L2 teachers should go through these principles in an appropriate way; otherwise, they will be demotivated since they do not observe any progressive results in implementing self-assessment and peer-assessment and many challenges will be undeniable. The MAXMap of the probable solutions stated by the ELT teachers is shown in Fig. 3.

Fig. 3

Solution codifications: ELT Teachers' perceptions (MAXMap)

ELT students’ perspectives about the challenges of implementing self-assessment and peer-assessment in L2 writing courses

The perspectives of ELT students about the challenges of implementing self-assessment and peer-assessment were explored using a broad open-question tool. As it was stated earlier our objection of including broad questions was to tap upon the depth of ELT students’ understanding of the self-assessment and peer-assessment issues. Moreover, MAXQDA software is more helpful when analyzing general and broad information. Consequently, the main themes and codes produced by the ELT students in their responses to the open-ended questions were drawn. Figure 4 illustrates the code system of ELT students’ responses.

Fig. 4

ELT Students' perceptions about challenges and probable solutions of self/peer-assessment

As shown in Fig. 4, ELT students believed that the assessment literacy is a factor which is necessary for implementing self and peer-assessment in writing courses. This is in line with the perspectives of ELT teachers since they have referred to the importance of assessment literacy both on the part of ELT teachers and ELT students.

What we all know about language assessment of writing is about the traditional paper-and-pencil correction by the teacher, what we call product oriented assessment. However, doing self- and peer-assessment to assess myself and my friends is problematic since I do not think I am that much proficient in doing that types of assessment.

This leads to another sub-theme: learners are not responsible for assessing themselves and their peers. This is sort of belief which ELT students asserted. Some of them stated that since they are not proficient about doing self-assessment and peer-assessment “it is better for us not to take the risk and doing that.” This can be interpreted broadly as when students do not see themselves that much proficient in doing a part in their assessment they may feel there is no need for them to do something in that regard and put the responsibility, totally, on the shoulder of their instructors.

Similar to ELT teachers, the ELT students referred to practical issues of doing self-assessment and peer-assessment. “It is time consuming” and “it is difficult to implement them in the classrooms with large number of students.” Lopes (2015, p. 3) posited that “Time is another aspect to consider, as normally the alternative ways to evaluate students are time-consuming both for the teacher and for the students.” The issue of time is much more important in larger classes due to larger number of L2 learners. These factors along with the low assessment literacy can lead to daunting challenges in implementing self-assessment and peer-assessment.

ELT students also believed that they cannot accept other learners’ opinions about their work since they are subjective. On the part of peer-assessors, ELT students believed that due to some factors such as existing conflicts among peers they might not play fair while assessing their peers’ work. Moreover, n the part of self-assessors, they do not show a fair description of their work. Sometimes, self-assessors overestimate or underestimates themselves. This is in line with what we have already referred as the effect of cultural factors on doing self-assessment and peer-assessment (Chen 2008; Matsuno 2009). Esfandiari and Myford (2013) stated that

when Iranian students rate their own essays, they are not likely to assign ratings that are lower than those that they would assign to their peers’ essays. Because students very much appreciate higher ratings, they may be more likely to assign their own essays higher ratings than they actually deserve.

This is showing that the conflicts of comparing self with others may be a reason for the overestimation and underestimation of self-assessors. This is a herculean task for the ELT instructors to create a rather even atmosphere in the classroom. Moreover, as Butler and Lee (2010) asserted the subjectivity of self-assessors is a threat to the validity of self-assessment.

Finally, ELT students asserted that instructional problems with regard to doing self-assessment and peer-assessment are other reasons for the challenges of implementing self-assessment and peer-assessment. ELT learners also declared that learners with different language proficiency might assess their writing and their peers’ essays differently. Suzuki (2009) also found out that L2 learners with higher language proficiency have the tendency to assess themselves lower comparing to L2 learners with lower language proficiency.

… we are not instructed how to do self- and peer-assessment in our writing courses. Once our writing instructor required us to self-assess the essay we write for our writing course. He told us that read about self-assessment of writing essays in a specific book and then implemented self-assessment on your writing… but I think language proficiency level has a very important role in the final output of self- and peer-assessment. Some of my peers are very low in the level of proficiency which hampers us from a reliable self-assessment and peer-assessment…

As mentioned by the ELT learners the predetermined syllabus is a challenge to implement self-assessment and peer-assessment. This is so since in the predetermined syllabuses in writing courses there are some main objectives which the instructor and learners need to accomplish. This in its place leaves a very limited flexibility to the program for implementing self-assessment and peer-assessment.

In our writing classes we are just overloaded with the course itself and since we have to accomplish the syllabus of the course from A to Z there is no time to think of self- and peer-assessment in a way that they deserve. There are two things in this regard. First we do not do self- and peer-assessment and second we do but we do not pay sound attention to them.

By using MAXMap we draw the schematic map of challenges in implementing self-assessment and peer-assessment in writing courses from the perspectives of ELT learners. Figure 5 shows this map.

Fig. 5

Challenge codifications: ELT Students’ perceptions

ELT students’ perceptions about the probable solutions of implementing self-assessment and peer-assessment in L2 writing courses

In this study we have also investigated the probable solutions of implementing self-assessment and peer-assessment from the perspective of ELT students. ELT learners, in contrast to ELT teachers, believed that the responsibility of assessment in writing courses should be given to the ELT teachers. It is thought that the ELT students were not that much in favor of the principles of learner-centeredness which was once asserted by ELT teachers.

… teachers are not bias, nor are they judgmental. So our teacher is the best person who can assess our essays. My classmates are not that much proficient to do the assessment. They are sometimes bias and this leads to problem in the assessment criteria. All in all, learners should learn and teachers should teach. Each of them has his/her own responsibility.

Smith and Ushioda (2009) stated that the capacity of learners should be considered when learner autonomy (learner-centeredness) is focused. It is believed that care should be taken while putting the responsibility of assessment on learners’ shoulders since different learners belong to different personality characteristics and learning styles. Putting the perspectives of ELT teachers and ELT students together, we found a gap regarding ELT teachers’ and ELT students’ opinions about learner-centeredness and learner autonomy. While ELT teachers saw them as opportunities for learners and conducive to help them implement self-assessment and peer-assessment in writing courses, ELT students did not have such a perspective.

The other two solutions proposed by ELT students were more in line with the ones stated by ELT teachers. They believed that one probable solution to come up with the challenges of implementing self-assessment and peer-assessment is to instruct learners how to do self-assessment and peer-assessment. ELT students posited that the instructions should be at the level of theory to inform them why they should do self-assessment and peer-assessment and at the level of practice to inform them how to use the rating scales.

I think for the solutions the teachers need to teach us the why of doing self-assessment and peer-assessment. For the time being I think that teacher-assessment is the best type of assessment since I do not know about the tenets of self- and peer-assessment. Additionally, we should know how to do them. Can they be done the same as other types of assessment or not?

Finally, ELT learners stated that the ELT teachers need to increase their knowledge about self-assessment and peer-assessment. This helps them to introduce these assessment types better to their students. Moreover, they believed that if ELT teachers be knowledgeable enough they would find the capacity to implement self-assessment and peer-assessment in different situations.

The teacher who implemented self- and peer-assessment in our classroom was not that knowledgeable. I think ELT teachers should first study about self-assessment and peer-assessment and then introduce to their language classrooms.

We finally used MAXQDA to draw a MAXMap of the responses we got from the ELT students about the probable solutions to the challenges of implementing self-assessment and peer-assessment. Figure 6 shows this MAXMap.

Fig. 6

Solution codification: ELT Students’ perceptions

Conclusions and Implications

This study was an attempt to explore the perspectives of ELT teachers and ELT students about the challenges and probable solutions of implementing self-assessment and peer-assessment in writing courses. The results of the current study revealed that ELT teachers believed that implementing self-assessment and peer-assessment are challenging due to the practicality issues such as being time consuming and limitation in resources, thus supporting the results of the previous studies in this regard (eg Bullock 2011). Moreover, the findings of the study showed that one main concern of the ELT teachers was the capability of ELT students in implementing self-assessment and peer-assessment in writing courses (Bullock 2011). Additionally, from the perspectives of ELT teachers, culture was considered to possess a pivotal role in how ELT students implement self-assessment and peer-assessment. They believed that some cultural motives involving in the characteristics of ELT students are significant in leading them how to implement self-assessment and peer-assessment (Brown 2005; Matsuno 2009).

The perspectives of the ELT teachers were also investigated to observe their probable solutions to the challenges of implementing self-assessment and peer-assessment. The results showed that they mentioned action research, post-method principles, and teacher training courses as the important resorts for tackling against the challenges of implementing self-assessment and peer-assessment in writing courses. What we think about ELT teachers’ attitudes is that the ELT teachers participating in this study searched for a way to increase their knowledge about self-assessment and peer-assessment.

We were also interested in identifying the challenges and probable solutions of implementing self-assessment and peer-assessment in writing courses. Similar to ELT teachers, ELT learners considered ELT learners’ lack of assessment literacy as one of significant challenges of implementing self-assessment and peer-assessment. They believed that ELT learners need to be trained in different aspects such as self-rating and peer-rating to find the capacity of doing self-assessment and peer-assessment. The study by Patri (2002) also showed that some aspects of self-assessment need further training on the part of learners. Moreover, the subjectivity involved in self-assessment and peer-assessment make the assessment difficult for ELT learners “due to lack of assurance and confidence in their decisions” (Ibberson, 2012; p. 22).

Some probable solutions were given by the ELT students to deal with the challenges of implementing self-assessment and peer-assessment in writing courses. Instructing learners how to implement self-assessment and peer-assessment was a solution through which ELT learners thought that they could be trained to perform fair assessment. Blue (1988) believed in instructing learners to obtain ability to do self-assessment. In addition, similar to the ELT teachers, they posited that the ELT teachers need to increase their knowledge of implementing self-assessment and peer-assessment in writing courses. This fosters their ability to implement these assessment types and to train their learners.

Reviewing the challenges and probable solutions of implementing self-assessment and peer-assessment in writing courses, we consider some pedagogical implications. We believe that ELT teachers need to consider the context of implementing self-assessment and peer-assessment. The implementation process as Boud (1995) declared need to assure learners that they can rely on the results of their assessment and other students do the same without cheating. Moreover, as Kirby and Downs (2007) stated it is very much better to implement self-assessment while grades are not considered as the main criterion. We add to this the principle of instructing learners about the why of self-assessment and peer-assessment and the how of implementing these assessment types. Furthermore, according to Falchikov (2007) peer-assessment will be more effective if ELT learners practice it during several times. This means that by using other collaborative activities such as exchange of opinions about an issue learners can be prepared for implementing peer-assessment (Spiller 2012).

Previous studies on self-assessment and peer-assessment in writing courses showed the effectiveness of these assessment types on fostering learning. However, these assessment types are not without challenges. Different factors such as cultural background, educational system, learners’ language proficiency level, and teachers’ assessment literacy are pivotal in doing these assessment types. It is highly important for L2 teachers and practitioners to consider these factors in order not to encounter challenges in implementing self-assessment and peer-assessment in writing courses.