Longitudinal study of a procedure for training low-proficiency english language students

Abstract

In this research study, unlike previous studies that have examined the advantages of peer review for low proficiency students, uses a longitudinal action mixed methods design is used to explore the impact of peer review on the academic writing of students with low English proficiency. The format was a longitudinal action study, which analyzed the written peer review feedback, essays, and reflections of 40 learners. Results showed that at the beginning of the study, the students experienced challenges in identifying problems with rhetorical content of their peers and by the end of 11 weeks, they were able to provide succinct peer feedback that included examples and explanations of classmates’ grammatical and rhetorical errors. Results also showed improvement in their own writing skills. Implications for implementing peer review in foriegn language writing contexts are discussed.

Introduction

Peer review (PR) is considered important for supporting the writing development of English learners through process-based and genre-oriented sessions. PR has been found to assist in achieving learning objective (Taras, 2006), improving critical thinking skills (Bostock, 2000), and contributing to self-regulated learning (Becerra-Alonso et al., 2020). Although research has shown the benefit of peer review, there “is clearly a need for more longitudinal studies investigating whether such feedback has long-lasting efforts” (Storch, 2019, p. 156), especially in foreign language contexts (Lv et al., 2021). Based on a review of the literature, no studies have been conducted using mixed methods to explore the effect of training in the process of PR on the quality of academic writing skills for lower achieving students, especially in Saudi Arabia (SA), where the students’ ability to compose an essay with an emphasis on grammatical accuracy and rhetorical content is a critical issue (Al-Khairy, 2013). Low writing competency is a consequence of absence of opportunities for learners to perform away from the classroom, lack of enthusiasm to learn writing skills, and mismatch between syllabi of international curricula in writing courses and learners’ language capabilities (Al-Seghayer, 2014).

Therefore, the primary goal of this study is to explore the effect of process-oriented PR training courses on the quality of peer feedback over 11-week periods and the impact on students’ academic writing. The methodological design will be mixed methods in which essays will be analyzed quantitatively and perspectives of the students will be analyzed qualitatively. This study has the potential to contribute to the field of computer-mediated communication and instruction in foreign language contexts by examining the effectiveness of peer review as a pedagogical method for language teachers. The goal is to use computer-mediated communication for instruction in PR to improve the quality of academic writing, increase autonomy, and stimulate motivation to learn English among students.

Literature review

Benefits of peer review

PR in second language writing has been found to facilitate learning by providing learners with opportunities to highlight their own accomplishments and discuss feedback with co-learners. PR can improve self-directed learning, enhance problem-based learning, learner motivation, and independence (Marzano et al., 2001; Xiao & Yang, 2019), and develop critical thinking skills (Boud and Falchikov, 2006). During PR, the student’s opinions and understandings are strengthened through active engagement in the process of reflection and peer review, enabling them to internalize these ideas. Furthermore, students can gain experience in self-assessment by engaging with peers to correct their writing errors while developing their own judgmental skills (Reinholz, 2016).

Orsmond et al. (2000) found that the inclusion of PR methods in a tutoring plan could fortify the connection between teacher feedback and tutee learning by giving students the opportunity to describe tutee learning results and justify their method for accomplishing the task instead of only providing the finished product. In addition, according to Assessment et al. (2005), learners frequently interact more realistically with each other than with tutors, and the receivers of feedback tend to be more engaged when the feedback is given by a peer. Intrinsically, PR offers learners the opportunity to assess peer content from the viewpoint of an analytical reader (Faridah et al., 2020).

Need for forging language research

Low-accomplishing learners are frequently ineffective at assessing themselves and others. Smith’s (2017) qualitative study on learners indicated that coaching in PR increased learners’ understanding of various topics at different stages. PR training also resulted in increased competence in peer- and self-review, the expansion of students’ metacognitive knowledge, and an enhancement of their understanding of PR guidelines, which resulted in higher-value feedback (Hey-Cunningham & Miller, 2021). Hu and Lam (2010) conducted qualitative and quantitative action research (including pre- and post-writing tests and questionnaires) to test the effect of PR on the writing of low-proficiency students. Yu and Lee (2016) conducted quantitative studies to examine the effect of PR on lower-proficiency students using writing criteria such as content, grammar, and organization. These studies found a positive effect of PR on low-proficiency students. However, an examination of the literature shows that no study has incorporated a mixed methods design to investigate the effect of tracked these changes qualitatively and quantitatively and reported the process and results session by session during PR training, that supports the comprehension and application of learning criteria through the three stages of composition, comparison, and revision.

Conceptual framework

An analysis of current PR literature reveals that assisting learners with evaluative decisions of PR criteria involves the following characteristics that will serve as the conceptual framework of this study:

• 1. Integrating feedback that should connect to the learning curriculum, criteria, and students’ levels (Wylie & Lyon, 2019);

• 2. Training learners on the use of assessment criteria after clarifying the learning goals (Panadero et al., 2016);

• 3. Providing them with the scaffolding of a PR framework that includes clear feedback (Shute, 2008),

• 4. Involving them in the PR process (Panadero et al., 2016); and

• 5. Reporting the students’ progress (Wylie & Lyon, 2019).

It is found that teachers’ scaffolding is more effective when connected to students’ learning objectives, needs, and the criteria used to assess their progress (Andrade et al., 2008; Hattie, 2009; Wylie & Lyon, 2019;). As noted by Marzano (2001), this is important because “feedback should address the knowledge that students are supposed to learn and provide information that helps them know what needs to be done to improve their performance” (p. 16).

In order for PR to be effective, training on applying evaluation criteria is essential (Panadero et al., 2016). Rubric are considered particularly valuable in supporting learners in comprehending their goals, evaluating quality, and making decisions about revision, (Cahyono & Rosyida, 2016; Reddy and Andrade (2010). The literature suggests that after coaching students in evaluation criteria and providing feedback, tutors can involve learners in setting learning targets (Shepard et al., 2017; Wylie & Lyon, 2019). It is important to note that PR should not involve learners in grading each other and, instead, assist their peers in determining what might be missing in their work (Marzano et al., 2001). Reporting regularly on student progress is also pivotal for the integration of PR into writing instruction.

Another aspect of effective PR is scaffolding. Scaffolded feedback consists of signs, stimuli, replicas, clues, a part of the answers, and a straightforward direction (Hartman, 2002). It is believed that scaffolding facilitates explanation-centered learning, which facilitates the use of learning strategies and metacognition (Graesser et al., 2005). Such scaffolding is important during the early stages of learning, especially for lower-achieving students (Shute, 2008), as it facilitates goal-directed behavior through modeling a clearly defined activity and providing direction for the learner (Bransford et al., 2000). When used with computers, scaffolding enhances understanding through mixed-initiative dialogue (Shin et al., 2020).

In this study, I predicted that PR could be used successfully through direct modeling (Moreno, 2004) and elaboration feedback (Brookhart & McMillan, 2019; Shute, 2008), which are suitable for students with a low level of language proficiency. Elaboration feedback enables teachers or tutors to “(a) address the topic, (b) address the response, (c) discuss the particular error(s), (d) provide examples, or (e) give gentle guidance” (Shute, 2008, p. 158).

Hattie (2009) stated that “the most efficacious type of feedback… is (in) the form of audio, video, or computer supported guidance feedback, or associated feedback to the learning objective” (p. 174). Using and giving feedback via asynchronous techniques, such as video feedback, allows learners a real-time self-assessment of their work in accordance with evaluation criteria. This enables tutees’ learning regardless of their location, time of day, or access to their tutor (Park, 2011; Simpson et al., 2019).

Research questions

This study aims to answer the following questions:

Q1

To what extent do EFL students contribute to the quality of and changes in PR over the 11 weeks of the experiment?

Q2

What is the impact of PR training on students’ academic writing within the web-based Blackboard learning management system?

Methodology

Setting for the data collection

The researchers conducted the study at Jeddah University (JU) in Jeddah, Saudi Arabia. As a newly established university (2014), JU established a deanship of e-learning and distance education that is responsible for “supervis[ing] excellent e-services for e-learning and providing the necessary infrastructure for the high-quality educational environment” (Jeddah University, 2014). Through online learning management systems, such as Blackboard, students can access learning materials, submit their homework or projects, and participate in discussion threads.

Research design

The research design was integrated longitudinal action research. Action research is “simply a form of self-reflective enquiry undertaken by participants to improve the rationality and justice of their own practices, their understanding of these practices and the situations in which the practices are carried out” (Kemmis & McTaggart, 1988, p. 162). The objective of action research is to elevate the efficacy of the work in which students are individually involved and progress is monitored constantly (Stringer, 2014). This type of research involves investigating the specific dynamics in a local situation, taking a stipulated activity within that situation, and examining the outcomes (Stringer, 2014). This method of preparation and evaluation results in additional planning, action, and evaluation (Coghlan & Brannick, 2010).

In this study, a longitudinal action study was selected because it matched the research objectives to explore the effect of process-oriented PR training courses on the quality of peer feedback over 11-week periods and the impact on students’ academic writing. This can be done through action research because of the interactive nature of this approach in which the teacher-researchers recognize the problem, determine the data collection methodologies, examine and elucidate the data, and formulate an activity strategy based on the results. During each cycle, the researcher trained the students to conduct PR by analyzing the students’ drafts and providing feedback to improve their writing. A longitudinal method was used to detect changes over time for students with low English proficiency to engage in more advanced writing activities, problem solving, thinking, and constructing peer explanations.

Participants and setting

There were two groups of participants. First, 42 second-year female participants aged 23–24 were invited to participate as volunteers. Of these, 40 agreed to participate and signed consent forms in which it was acknowledged that their participation would not be reflected in their course grades. These students were enrolled in a Writing II course in the Department of Language and Translation (DLT) at Jeddah University. All were female because of gender segregation policies in the Saudi educational system. They used the textbook, Great Writing 4 (MacIntyre, 2012), which is at level B1, an intermediate level in the Common European Framework. According to the results of the entrance test of DLT, the participants’ level of English language learning was intermediate. Although these students studied English in their K-7 school programs and had completed four English courses before their admission to DLT, their English writing skills were low.

Second, there were two English essay raters and one expert rater from DLT. The first two raters each held a Master of Arts degree in TESOL and had six years of experience teaching English writing skills. The expert rater held Master of Arts and PhD degrees in TESOL and had 12 years of experience teaching English writing skills.

Data collection

Writing assessment

The teacher-researcher adapted an assessment rubric that was validated by DLT and used by their teachers, which was a holistic rubric that reflected the qualities of a formative assessment (Panadero & Jonsson, 2013). This rubric used a Likert-type scale to quantify four performance levels: excellent, proficient, satisfactory, and unsatisfactory (see Additional file 1: Appendix 1).

The researcher addressed two issues in adapting the assessment rubric. First, according to the literature, a holistic rubric should be used with advanced learners because it provides a score for students without breaking down feedback qualities (Alshakhi, 2019). Second, students with a low proficiency usually find it difficult to comprehend some rubric terms. To mediate this issue, the researcher reinterpreted the rubric criteria as questions to support the pedagogical process, enhance authentic performance and self-regulation, and help students determine whether they achieved their learning goals (see Additional file 1: Appendix 2).

The rubric included four criteria: task elements, grammatical components, lexical resources, and rhetorical content. The task elements criterion involves the assessment of two features of writing composition: (a) ability to master the three main parts of essay writing (introduction, body paragraphs, and conclusion) and (b) competence in differentiating between types of essay writing (narrative, cause–effect comparison, and argument). The grammatical components criterion focuses on ability to produce a variety of sentence structures and lengths, maintain consistency of punctuation, and avoid grammatical errors. The lexical resources criterion involves proficiency and adaptability in using extensive vocabulary to express exact meanings, generating few mistakes in spelling, and making appropriate word choices. The rhetorical content criterion focuses on conveying complicated concepts with distinct and succinct language and showing awareness of voice and audience. These criteria are intended to assist learners in concentrating on meaning more than form.

Student essays and reflective essays

To track writing improvement quantitively, the two raters assessed 40 student essays from the first and last sessions (see Table 1). This assessment supported the analysis of PR training’s influence on student writing skills. To track writing improvement qualitatively, students wrote two guided, reflective essays, one at the beginning and one at the end of the experiment. These reflections invited them to evaluate the impact of PR integration, as well as the training process, on their writing competence. The two types of data analysis enabled comparisons between student perceptions and actual performance at the beginning and end of the experiment.

Table 1 Teacher-researcher’s feedback based on students’ needs and learning objectives/outcomes

After the first session, the students wrote their first reflective essay in English, which required a response to the following question: What difficulties do students face when correcting their classmates’ drafts? The second reflective essay asked them to reflect on changes in their perceptions of the effect of PR and teacher feedback scaffolding on their writing skills after the 11 weeks. The data sources enabled triangulation of raters’ analyses and evidence for construct validity.

Procedure

After obtaining approval from DLT to conduct the study, the first week consisted of providing students with the purpose, procedure, and anticipated outcomes. Data collection began in the second week. Since an action research approach was used, data collection included the four stages of action research: planning, action, observation, and reflection (see Fig. 1). During the 11 weeks of the experiment, six chapters were covered, each of which involved four lectures and lasted two weeks.

Fig. 1

The process of action research

Planning stage

The planning stage occurred in the classroom. During the first lectures of each chapter, the teachers-researcher¹ asked participants to choose their own topics, outline the main points, and discuss the outlines with their peer assessors. This step was important for students to familiarize themselves with the essay content and to provide opportunities for correction (Shute, 2008). In the second and third weeks of each new chapter, the participants were asked to write their first and second drafts and in the fourth week, they submitted their final draft. The students were required to write these drafts outside the educational settings. During the process of writing the three drafts, the students were required to post their essays on the Blackboard discussion panel. This was an important step that allowed the teacher-researcher to follow the students’ progress in conducting PR and assessing their writing.

Action stage: PR training and procedure

The action stage consisted of two parts. The first part involved teacher-researcher-modeled PR, which covered the ongoing process for assessing classmates’ drafts. The feedback given to students was direct and sought to “describe what, how, and why of a given problem … [and] provide elaborated feedback in small enough pieces” (Shute, 2008, p. 177). This process was considered appropriate for low-achievement learners (Moreno, 2004; Shute, 2008).

The teacher-researcher selected four drafts of students’ work and elaborated on their weaknesses and strengths. She orally reflected on the English in their writing using audio–video recordings through the video feature of Blackboard Collaborate (Fig. 2). This tool allowed her to give feedback according to the four criteria discussed previously. In each unit, she provided two audio–video feedback sessions, each of which lasted 20 min.

Fig. 2

A screenshot of the recording session

Responding to the first and subsequent drafts, the teacher-researcher focused on a combination of the four criteria. To be more constructive, she analyzed the essays based on task elements and grammatical components in the first revision and rhetorical content and lexical resources in the last draft. This was to accommodate the difficulties of the last two criteria for students with low English proficiency. Additionally, Shute (2008) noted that “presenting too much information may not only result in superficial learning” due to their low proficiency levels (p. 177).

In the second part of the action stage, the students viewed the audio-visual feedback uploaded to Blackboard. The Excel function RANDBETWEEN randomly assigned the students to demonstrate peer correction. For the peer evaluation protocol, each learner worked as an assessor 33 times over the 11 weeks. The assessors reviewed all 40 students, and everyone was asked to log on to the discussion panel on Blackboard to correct their peers’ mistakes. Their identities were not anonymous. Therefore, the researcher could follow their progress. After the first week and at the end of the experiment, they also had to complete the reflective survey questionnaire.

Observation stage

The teacher-researcher conducted the observation in the third stage. She observed and kept a record of everything that happened during PR and then reevaluated the students’ PR. The observation stage also involved linking the feedback to students’ needs, learning objectives, and outcomes related to their curriculum and elaborating on what they needed to do next. This step was important in encouraging continuous learning.

Reflection stage

The investigator used the outcomes of the third phase to deliver the fourth phase, which focused on reflection. She analyzed the data to determine if anything was lacking in order to offer targeted feedback for the next phase.

Data analysis

The students’ PR was analyzed through deductive thematic approaches that “would tend to be driven by the researcher’s theoretical or analytic interest in the area and is thus more explicitly analyst-driven” (Braun & Clarke, 2006, p. 84). These criteria were derived from the assessment rubric (i.e., task elements, grammatical components, lexical resources, and rhetorical content).

The researcher analysed and coded students’ PR qualitatively and quantitatively based on the four criteria. The researcher used NVivo to organize the data into categories, track changes in the quality of feedback, and calculate the frequency of each criterion. However, given the importance of determining changes in sentence-level (i.e., grammatical errors) and rhetorical features, these two criteria were dominant in the analysis.

For the essays, an expert judgment model was used to determine inter-rater reliability (Johnson et al., 2005). First, the two ESL raters scored the essays from the first and last sessions using the assessment rubric of the Writing II course. Then, the expert judge engaged a third arbitrator to perform a blind evaluation of each. One characteristic of the expert judgment prototype is that the arbitrator must have greater accuracy and dependability than the other raters, a fact that is reflected in their increased scoring power (Johnson et al., 2005). The arbitrator’s score incorporated both initial scores before becoming the active rating. Therefore, if the arbitrator provided the maximum scores possible, the initial raters’ scores did not affect the outcome. In addition, the expert rater could evaluate the initial scores and alter one or both scores before finalizing them. This design assumes that the judgment of the expert rater offers a more precise estimation of ability than the collective judgment of the initial raters (Johnson et al., 2005).

The researcher analyzed the students’ reflective essays using inductive thematic analysis while engaging with the entire dataset and categorizing recurring patterns or topics. The aim of the inductive method was to shift from focusing on explicit data to looking at usual conceptions in order “to develop a model … about the underlying structure of experiences or processes that are evident in the text data” (Thomas, 2006, p. 238). For both instruments, assigning an independent coder who held a master’s degree in applied linguistics was important to ensure inter-rater reliability and consistency checks. Differences in coding outcomes were determined through conversation, which improved initial codes.

Results

This section summarized the findings of the research aims. The first part addressed the first question and delineated the changes in the PR of students with low English proficiency over an extensive period. The second section quantitatively investigated PR training’s impact on students’ academic writing, supporting the results based on the students’ perspectives.

The first research question explored the extent to which EFL students contribute to the quality of and changes in PR over the 11 weeks of the experiment. Results showed that the students showed marked improvement over the course of the experiment. It is of note that no spelling errors were identified. As it seemed that this finding was a result of the word processor’s auto-correction function, spelling was not included as a factor in the findings.

Qualitative thematic analysis of peer feedback

For the first session, the students only provided peer feedback on punctuation and grammatical errors (35 times). After the first reflective essay, they reported that it was difficult to find their classmates’ mistakes in relation to rhetorical and lexical content as this was the first time, they had reviewed work with these criteria in mind. Moreover, the feedback they provided was short and in bullet-point form. Some of the students found it difficult to understand the corrections, as demonstrated in their discussion panel responses. They reported feeling that they needed explanations for suggested changes. As a result, the teacher-researcher encouraged the students to explain their reasoning for the corrections in the next cycle.

For the second session, the students found even more grammatical mistakes (50 times) and many fewer rhetorical problems (19 instances). In addition, five of the peer reviewers corrected whole sentence instead of using bullet points, as shown in Excerpt 1, which appeared to reflect the scaffolded training they had received.). Regarding the task elements criterion, the assessors tended to advise that some parts of the essay needed modification, such as separating the introduction from the body paragraphs (see Table 1).

Excerpt 1

Student C:

RE: Narrative essay.

The stress full of snow.

Correction: The streets are full of snow.

I will never forget that day in paris. Last summer, I plan to reavel with my family to paris because we haven’t ever been to it and saty for seve days. On summer 2018, we have prepard and to the airport enthusiastically.

Correction: *Try to start the sentence differently rather than just using “summer” at the start to avoid being repetitive.

• 1. You should use a capital letter for cities.

• 2. Use the past tense (gave, asked).

• 3. The airport was big (huge). i (I) woke up (around) seven in the morning.

In the third session, grammatical mistakes (40 times)² still outnumbered errors in rhetorical content and tended to be more accurate than in the first two sessions. The students also provided explanations for these corrections (see Excerpt 2).

Excerpt 2

Student D:

RE: My essay.

And (should not be used in academic essay) instead write also or start the sentence without it.

There is important = there is an important.

Must to know = must is strong enough to be use without “to”.

You cannot write “whatever” in academic writing. Instead, replace it with “however”.

However, at the end of week six and for the first time, the students’ advice about enhancing the quality of the writing and addressing rhetorical content improved (25 times; see Excerpt 3). Regarding the task elements, the assessors observed that some students did not incorporate the basic elements of essay writing. For the lexical resources, they also moved from providing the correct words to using academic words, which added detail and depth to the writing. The assessors often offered a clear word to convey a precise meaning.

Excerpt 3

Student G:

RE: comparison essay.

What is the difference between the star and the planet? And how can we differentiate between them with the naked eye or the telescope? We can notice the first and most noticeable differences between the two (by seeing the body glistening or not.) Stars constantly (shine and shine while planets don’t).

Correction.

What is the difference between the star and the planet? And how can we differentiate between them with the naked eye or the telescope? We can notice the first most noticeable differences between the two by seeing whether the body glistening. Firstly, stars constantly shine while. Planets, on the contrary, don’t glint.

I think this is clear to understand.

In the fourth session, slight changes were noted and the dominant focus of PR was on organization and clarifying meaning (39 instances), rather than solely focusing on correcting grammatical mistakes (35 times). The students’ skill level in providing corrections and offering examples improved. In cases in which the meaning was unclear, the students stated explicitly that it was necessary to clarify the meaning (see Excerpt 4).

Excerpt 4

Student Y:

RE: Tourism in Saudi arabia.

It helps to meet new people from cities or regions that we haven’t visited before and this brings a positive change in the society ^X (the meaning is not clear so I added some words that helped to make it easier to understand + the before society is not necessary).

It helps (us) to meet (different) people from (different) cities and regions that we haven’t visited before. This brings a positive change in society .

Some sentences are not clear. Try not to translate the sentence from arabic to english because sometimes it doesn’t the same and clear meaning.

Furthermore, the students emphasized academic writing more than in the previous sessions (12 instances). The analysis of peer corrections shows that they expressed their ideas clearly (6 times). Regarding the essay elements, 16 of the assessors corrected and rewrote some parts of the essay instead of simply addressing problematic parts. Some differentiated between weak and well-written parts, explicitly stating which parts were incoherent. They also gave suggestions for strengthening the ideas presented and increasing paragraph length (see Excerpt 5). In addition, at the end of week eight, it was mentioned five times that the writer needed to get straight to the point, unlike in the essays written in the Arabic language, in which the main points could be mentioned at the end of the essay.

Excerpt 5

Student Y:

RE: the effect of colour.

*The essay is too short in all the paragraphs, you should explain more in both paragraph that light colors and dark colors. An idea to make your paragraphs longer is maybe write about: give more examples about colors affecting mood. e.g.: Some colors increased blood pressure and metabolism therefore, most restaurants uses red in their logos to open people’s taste.

*The summary should restate the intro an opinion of how it is important to know how to choose the right color.

Generally, in the fifth session, the students continued to focus on organization and clarifying meaning (44 occurrences) rather than solely focusing on correcting grammatical mistakes (39 times). However, they explored their peers’ work even further and utilized their critical thinking skills to discover errors. They also justified the corrections logically. They instructed that the body should relate to the paragraph’s main idea. The students also mentioned that the essays should have cohesion, coherence, and a smooth flow. It is notable that the students were able to organize their ideas and identify missing elements. The correction methods were divided into three forms: 25 provided examples, 11 rewrote paragraphs, and eight rewrote the entire essay as a model for their classmate. Additionally, content analysis was also present in this session, with the reviewers assessing the quality of writing from an academic perspective, identifying unclear structural features or lack of content knowledge, and highlighting inconsistencies in arguments (e.g., contradictions or weak points of view). For the first time, some assessors used a phrase of necessity, such as “you must change” or “you have to,” instead of advising words, such as “you should” and “I prefer to change” (8 occurrences; see Excerpt 6).

Excerpt 6

Student B:

RE: Argument essay.

(Some people cannot enter the university of collage, maybe because of they have minimum marks then cannot move to university, whatever, the students of university should know that they had a good chance which is going to the university. So, it is very important to know the ethics which had related to the university. This essay will discuss about the ethical on university in general, like while studying, on the other hand, when student spend his time her time with friend you need to respect them.)

Correction.

There is a topic sentence but it is not well written. The first sentence must corrected like ( some people cannot get a place in university because their grades are not met by the univerisities requirements. Students should take going to university seriously as it is one of the best chance to lead successful career. As a student of a certain university you should learn and follow the ethics of the university to ensure a well pleasant stay at it) The thesis statement is written wrongly, it would be like ( This essay is going to discuss some of these ethics that should follow such as studying as a university student and spending quality time with your friends).

For task elements, the students analyzed the essay to identify any deficiencies. For lexical resources, in cases of word repetition, the students suggested replacing words with synonyms. They also affirmed the need to use academic words instead of slang vocabulary such as “awesome,” “things,” and “very much.” They further identified synonyms and expressed praise for peers who utilized a diverse vocabulary. For example, Student B said, “There is a wide range of vocabulary that has the same meaning: protection = prevention, lack = shortage.”

Quantitative statistical analysis of student essays

The second research question examined the impact of PR training on students' academic writing with the web-based Blackboard learning management system.

Overall, the initial paired samples t test suggested that in all four cases, scores rose statistically (see Table 2). The mean for the initial task element measurement was 4.35, with a standard deviation of 0.975. The mean for the post-task elements measurement was 5.90, with a standard deviation of 0.871. The initial and post-measure figures for the other three measures were also given for grammatical components (initial mean of 3.95 and standard deviation of 0.876; post mean of 5.588 and standard deviation of 0.7917), lexical resources (initial mean of 3.58 and standard deviation of 0.675; post mean of 4.70 and standard deviation of 0.8305), and rhetorical content (initial mean of 4.725 and standard deviation of 0.8767; post mean of 6.38 and standard deviation of 0.740).

Table 2 Means and standard deviations of scores for each criterion before and after the experiment

Table 3 shows the statistical difference between the initial measures and the post-treatment measures. Overall, the mean increase for task elements was 1.550, with a 95% confidence interval of 1.261 and 1.839. The mean increase for grammatical components was 1.6375, with a 95% confidence interval of 1.3973 and 1.8777. The mean increase for lexical resources was 1.125, with a 95 percent confidence interval of 0.8801 and 1.3699. The mean increase for rhetorical contents was 1.650, with a 95 percent confidence interval of 1.3466 and 1.9534.

Table 3 Paired samples test for each criterion

Qualitative thematic analysis of reflective essays

After the first session, the students generally reflected that they (a) had difficulty differentiating between academic and informal essay writing, (b) could not construct viable and solid arguments, (c) lacked an understanding of the essential components of essay writing and an adequate grasp of English, and (d) had difficulty generating a variety of sentence structures and vocabulary. Results of the second reflective essay showed that the training sessions of PR and asynchronous technology positively influenced the students’ writing and learning processes. Specifically, the students expressed the following insights about the use of asynchronous technology and peer review.

The students noted that communicating with classmates via Blackboard inspired their thoughts and encouraged collaboration. They also reported that contacting their teacher outside the learning environment helped them negotiate meaning, gave them space to reflect on the review process, and facilitated the exchange and evaluation of different perspectives and PR-related skills. Overall, their reflections indicated that PR and asynchronous technology helped them to feel more confident about their academic writing and encouraged autonomous learning, as shown in the following excerpt:

Student S: “Watching the video at my home was useful, so we can study at our own pace and search for information”.

They also felt that noticing the mistakes of others in a collaborative environment enabled them to write clearer, concise, complex, and more effective sentences within paragraphs that contained coherent ideas, as illustrated by the following excerpt:

Student C: “When I correct the articles of my colleagues in blackboard, I am indirectly learning³ how to write an entire academic style without gaps.”

The students further reported that by the end of the PR experience, they were able to think and write in English, as opposed to thinking in Arabic and then translating into English. They described how the need to correct their classmates’ essays led them to look up errors via Google instead of depending on a grammar program such as Grammarly. Many of them also described feeling that they had the right to make changes based on feedback instead of simply copying the assessor’s correction. At the beginning of the experiment, some of the students mentioned that they were shy and apprehensive about correcting their peers’ essays, which changed with PR experience as show in the following excerpt:

Student B: At the beginning, I feel shy when the teacher told us to correct our essay. However, when I and my friend started to correct the essays, I did not feel shy because she told us and showed us the way of correction.

All of the students stated that by the end of the training sessions, they were able to differentiate between academic writing and informal language and to develop persuasive and critical skills. They also mentioned that the teacher’s scaffolding helped their writing and assisted them in developing academic writing techniques, such as paraphrasing, summarizing, simplifying meaning, reducing the overarching ideas in the essay, and searching for information to support an argument. They also noted that implementing PR by formatting the criteria as questions helped them understand how best to correct their classmates’ work, identify learning goals, and evaluate their work critically.

Discussion and implications

This study aimed to explore the effect of process-oriented PR training courses on the quality of peer feedback over 11-week periods and the impact on students’ academic writing. Analysis of the last three sessions of PR drafts showed changes in feedback quality that focused on delivering high-quality responses about language problems such as clear argument and structure, rather than sentence-level correctness.

Some students even demonstrated the ability to restructure and assess entire essays. These results differ from those of previous studies (e.g., Leeser, 2004), indicating the importance of pairing lower- with higher-level students to benefit from collaborative work. Findings from the current study showed that low-proficiency students could contribute to peer feedback when engaged in collaborative interaction. These findings are strengthened by the statistically significant results for increases in the students’ writing skills.

This study demonstrates four important aspects of the PR process through computer-mediated communication. First, asynchronous communication provided the students with low writing skills the time needed to express their thoughts, apply higher-order thinking skills through explicit correction, and foster self-paced learning (Chang & Windeatt, 2016; Kamhi-Stein, 2000). Second, the students believed that the integration of PR with assessment criteria and scaffolding training over an 11-week period had a positive influence on their academic writing (Hu et al., 2010) in terms of writing better essays, using clearer and more concise language, improving planning processes, and creating relevant arguments.

Third, results indicated that providing understandable assessment guidelines about writing performance and offering training were the most effective strategies. Furthermore, the feedback was corrective and explicit, which suited students’ low- proficiency levels and related to the learning objectives and students’ needs (Hattie, 2009). This result is consistent with previous findings (Narciss et al., 2007; Panadero et al., 2016). This is an interesting avenue to consider for language teachers. Instead of relying on generic international market curricula, as is often the case in SA, teachers could upcycle the curriculum by integrating PR in writing classes, connecting to students’ levels and needs. This helps to clarify the assignment criteria, assisting them to understand the desired performance and involving them in the learning process. This is an important step in adapting curricula and providing meaningful experience.

Fourth, although the proficiency level of the students was low, they appeared to be more confident in their feedback and more enthusiastic about the writing process by the end of the experiment, similar to Covert’s (2014) result that activating learners’ agency positively impacted their language competencies. Being responsible for their own learning appeared to have three benefits. First, as shown in the second reflective essays, some students stated they appreciated having the option to choose whether to make suggested corrections, search on Google for the right answer, or negotiate a correction with their classmates through the discussion panel. Next, at the beginning of the program, their corrections were in the form of advice. However, by the end of the experiment, feedback was more explicit, using statements such as “you must” and “you need to change.” Finally, they took on the teacher’s role by providing illustrations about how to improve writing quality, indicating improvement in writing confidence and learning agency. Although the PR identity was not anonymous, the students appeared confident in critically evaluating peers and expressing their opinions freely. These results were in contrast to findings from previous studies, such as Lu and Bol (2007) and Panadero and Alqassab (2019), who found that using automated feedback seems to provide advantages for students’ perceptions about the learning value of peer assessment and delivering more critical PR. On a pedagogical basis, allowing students to solve writing problem at their pace, promoting positive talk, and offering a sense of belonging are important to enhance feelings of competence when integrating PR.

Conclusion

Results indicated that PR training and collaborative writing contributed to enhancing writing methodology, supporting students’ sense-making, and improving understanding of their learning processes. A limitation of the study is the narrow age range of the participants in the sample and that male students were not included in the study due to gender segregation. Future research could focus on a greater number of students of different ages and genders.