An empirical study of claims and qualifiers in ESL students’ argumentative writing based on Toulmin model

Abstract

English argumentative writing is difficult in second language writing for ESL students. Although there has been a lot of research on English argumentative writings, empirical research focusing on claims in argumentative writing is very scarce. Based on the Toulmin model, this paper studied the relationship between the claims and English argumentative writing performance. 117 time-limited writings were collected at a top university in Northeast China. The qualifiers were in the form of words, participle and sentence (QW, QP, QS), based on which claims were classified. Through the text analysis of each argumentative writing, 23 types of claims were extracted. Types of claims and 117 argumentative writings were scored respectively. Correlation analysis showed that claims based on qualifier divisions did not predict holistic writing performance. Claims in the higher-mark writings were also discussed. The findings led us to conclude corresponding pedagogy implications in promoting argumentative writings.

Introduction

Writing, as an important language output, can reflect learners’ ability to use language flexibly. Argumentative writing is the most challenging type for writing teachers. English argumentative writing plays an important role in all kinds of tests and daily learning. Argumentative writing is also a style which can best reflect the process of argumentation. The researches on English argumentation in China mainly focus on lexical level (Jin & Zhai, 2013), syntactic level (Lu & Xu, 2016; Xu et al., 2013) and discourse level (Liu & Chen, 2015; Wang & Sun, 2006). Students at the university stage not only need to learn how to critically judge, evaluate and respond to the opinions put forward by others, they are also expected to express their own opinions in an appropriate way (Bridgeman & Carlson, 1983; Feak & Dobson, 1996; Liu, 2020; Varghese & Abraham, 1998). However, college students generally lack critical and logical thinking ability (Hao & Wang, 2013), so English writing has become the biggest challenge in ESL college students’ English learning. In recent years, there are increasing researches using Toulmin argumentation model as a framework to study English argumentative writing, but most of them focus on the main body of argumentation such as claims and counterarguments (Liu, 2020; Liu & Stapleton, 2020; Nussbaum, 2008; Qin & Karabacak, 2010). Very few studies focus on the openings of the argumentation. Thus, the formation process of writers’ standpoints and the type of claims are overlooked. Based on Toulmin argumentation model, this paper mainly focuses on claim and qualifier part. While identifying the types of claims, an attempt was made to identify the relationship between the claims and the overall quality of the argumentative writings.

Literature review

Toulmin’s claim, thesis statement and topic sentence

Concepts that are similar to claims in the Toulmin model have been sorted. Thesis statement and topic sentence are chosen to be compared for Chinese translation are almost the same for the three terms. Definitions of the three terms and corresponding examples are listed (Table 1).

Table 1 Definitions and examples of Toulmin’s claim, thesis statement and topic sentence

The concept of claim originates from Toulmin’s argument model (Toulmin, 2003), which is characterized by controversial truth value, so it is easy to cause controversy (Crammond, 1998). Usually, a claim is put forward to answer a problem or to solve a perceived problem. A claim can take the form of a proposal or policy statement, an assessment statement, a definition statement, or a causal relationship, depending on the question or the type of question raised (Ehninger & Brockriede, 2008; Fahnestock & Secor, 1983). Many scholars choose to take claims as subclaims in each paragraph in their studies (Abdollahzadeh et al., 2017; el Majidi et al., 2021; Liu, 2020), but according to the Toulmin model and the feature of argumentative essays, the claim is an asserted answer in response to a contentious topic or problem (Qin & Karabacak, 2010) that should have uniqueness and be the central sentence of the whole essay.

Moore and Cassel (2010) believe that thesis statements have dynamic feature that changes and evolves throughout the whole writing process, because “the exact words for the thesis statement are not finalized until the paper is nearly complete.” (p. 8) They hold the opinion that thesis statement is made up by context, subject and claim. Zou (2012) provides a more detailed analysis of the thesis statement. She classifies thesis statements into three basic forms of presentation, the open-ended form, the phenomenon + claim form, and the others’ views + the author view form. We can see the researchers tend to take the thesis statement as the holistic part of the writing, including various kinds of background information (Moore & Cassel, 2010; Zou, 2012). Good thesis statement is short and concise, focused on one main idea about the topic, and in a declarative sentence that contains no qualifiers (McClain & Roth, 1999).A topic sentence is “the surface manifestation of an element at the top or macro level in the semantic hierarchy of a paragraph.” (Popken, 1988, p77.)

In summary, claim in Toulmin model is characterized by its semantic independence and it exists independently of context. The claim is the core of the entire argumentative essay and is about the writer’s choice of position. Thesis statements are more macro in nature, presenting both attitudes towards the subject and statements of fact (Chesla, 2018). Scholars often divide the thesis statements into three parts: context, subject and claim (Moore & Cassel, 2010). A topic sentence defines what a paragraph is about and is beyond the scope of this study. Miller and Pessoa (2016) analyzed the thesis statements and topic sentence at the same time, trying to find the lack of college students’ writing in the two aspects. It is found that students tend to overgeneralize the thesis statements and confuse the thesis statements with the topic sentences. This underscores the importance of focusing on the claim, as this can be a starting point for students to distinguish these concepts and write qualified first paragraphs.

In the writing task designed for this study, students were required to express their positions and opinions in the first paragraph according to the topic requirements, so the claims in this study were all taken from the first paragraph of the 117 essays. The claim adopted in this paper is the one in Toulmin model, which treats the central sentence of the first paragraph as a complete semantic whole. In this way, background information is easy to be peeled off and students’ viewpoints are easy to identify.

Qualifiers

According to Toulmin’s definition, qualifier refers to adverbs such as “probably, possibly, perhaps” to soften the claim (Toulmin, 2003). Modified by qualifiers, the claim can be made neither too absolute nor too general.

Claims in this study are divided into core claims, qualifiers in word form (QW), qualifiers in participle form (QP) and qualifiers in sentence form (QS) according to Berger’s (2014) analysis of claims (Table 2).

Table 2 Classification and examples of defined components (Berger, 2014)

Qualifiers in word form (QW) are in the form of prepositions in this study. They are followed by nouns, pronouns and gerunds to form prepositional phrases, indicating the relationship between people, things and events. Three forms of non-finite verbs as adverbials compose qualifiers in participle: infinitive, present participle (- ing) and past participle (- ed). After manual coding, all the sentences that can be classified as QS are adverbial clauses. The modified adverbial clause accounts for the largest proportion in this study. Many grammarians believe that this kind of clause can be located before the subject sentence or after the subject sentence, with its function and meaning remaining unchanged (Shi, 1999). The modified adverbial clause modifies the predicate in the main sentence or the whole sentence, playing a role of modifying and limiting in the sentence. Berger’s division effectively divides the qualifiers in terms of phrases, participles and sentences, which can help make a clear boundary between qualifiers and claims for researchers.

Toulmin model

Before 1958, the independent development of logic and rhetoric made it difficult for scholars to seek inspiration for improving argumentation from these two disciplines, In 2003, Toulmin proposed the limitations of formal logic to science and other disciplines. He then put forward the Toulmin argument model (Toulmin, 2003). Since then, the model has been widely used in the teaching and research of argumentative writings (Aziz & Said, 2020; Crammond, 1998; Stapleton & Wu, 2015). Specifically, this framework is widely used to explain all stages of English argumentative writings. According to Toulmin (1958/2003), each argument consists of three main elements (data, warrant, claim) and three secondary elements (qualifier, backing and rebuttal). In argumentation, these six elements do not have to appear at the same time, the author can choose according to the context (Fig. 1; Table 3).

Fig. 1

Toulmin argumentation model (Toulmin, 1958, 2003)

Table 3 Elements and definitions in the Toulmin model

In Toulmin’s view, syllogism in traditional formal logic attempts to sum up argumentation in all fields into a universal formal system, only recognizing the difference between premise and conclusion, ignoring the complexity of argumentation, which is not conducive to our correct understanding of the actual argumentation in real life, and confuses the use of language as well (Yang, 2010). Toulmin clearly distinguishes the 6 functional elements, making the model more widely applicable in practical argumentation and thus has greater practical significance. Toulmin’s argumentation model is widely used in many fields (informal logic, legal studies, verbal communication, education, etc.), especially in second language argumentative writing (el Majidi et al., 2021; Hirvela, 2017).

Toulmin model is widely used in a large number of writing textbooks, scaffolding teaching and evaluation of argumentative writings. The most popular one is Writing arguments: a rhetoric with Readings (Ramage et al., 2018), each chapter of the book focuses on a detailed explanation and example of an element in the Toulmin model. A guide to argumentative research writing and thinking focuses on teaching academic writing skills based on Toulmin model (Wentzel, 2017). Wentzel integrates the idea of argumentation into the teaching of academic English, which provides teachers with new perspectives and methods for academic English writing.

Toulmin’s model is fit for analyzing concrete structures in argumentative writing because it is accepted broadly and applied widely. Firstly of all, his model dovetails nicely with L2 argumentative writings at ESL/EFL levels because it is generally enough to cover the meta structures of argumentative discourse (Cheng & Chen, 2009). Despite the fact that other theories of argumentation have been proposed, such as van Eemeren and Grootendorst’s (2016) pragma-dialectics and Walton’s (1996) argumentation schemes, these counterparts to Toulmin’s model are believed to be sophisticated and not practical for L2 college students. As a result, they are not considered to be the ideal model in pedagogical and research context (Cheng & Chen, 2009). In addition, the textual quality of students’ argumentative/persuasive writing can be explained and predicted by the elements in Toulmin model (Crammond, 1998; Knudson, 1992; Qin & Karabacak, 2010; McCann, 1989).

The relationship between claims and argumentative writings

Claim, data and warrant form the basic structure of Toulmin argumentative model, also known as the complete structure (Budke et al., 2020). Numerous empirical studies have demonstrated that argumentative essays with a complete structure are of higher quality, i.e. the more often these main argumentative elements appear, the better the quality of the argumentation (Gronostay, 2017; Lam et al., 2018; Von Aufschnaiter et al., 2008).

“Claims and data are considered to be the backbone of every argument” (Cheng & Chen, 2009, p. 30), so formally, before and during the writing process, claims play the role of setting up a thinking framework for writing. Claim is tested and discussed as the essential element in argumentative writings based on Toulmin model (Abdollahzadeh et al., 2017; el Majidi et al., 2021). In terms of the content, claims are the soul of English argumentative writing and represent the essence of the whole argumentation.

A systematic and detailed analysis of the stylistic features of claims can help students master the writing skills of argumentative sentences (Liu & Cui, 2017). Many scholars believe that claims are closely related to the quality of argumentative writing (Christie & Derewianka, 2010; Ravelli, 2011). Christie and Derewianka (2010) believe that in order to write claims successfully, students need to extract information from lengthy and complex topic texts and predict how the text will unfold. Similarly, Ravelli (2011) believes that students who can predict, position and restate their own discourse are more likely to persuade others in writing.

However, the relationship between claim and overall quality of argumentative essays is not clear in studies applying the Toulmin model as an analytical framework in the field of second language writing. An empirical study conducted by Abdollahzadeh et al. (2017) proved that Iranian postgraduates’ quality of English argumentative writing was significantly and positively correlated with claim, with a correlation coefficient of 0.29, p < 0.01. While in Qin and Karabacak’s (2010) study, in which the same adapted Toulmin model was used, the two parts are not correlated. This illustrates the instability of the adapted model. Therefore, it is more necessary to apply the original Toulmin model as an analytical framework to measure claims.

Research questions

In a sentence, prepositional phrases, participles and subordinate clauses can all be used as modifying components of the sentence (Berger, 2014; Kirk, 1997; Yazdani & Younesi, 2008). Both numbers and types of qualifiers are taken into account to classify claims in this study. According to the text analysis, there are claims with zero qualifier (core claims) and claims with one type of qualifier. A claim with two or three types of qualifiers is defined as the claim with more complex qualifiers in this study.

To recapitulate, in spite that Toulmin model has been applied in many researches on second language writing, few studies take qualifiers into consideration. The current study endeavors to fill part of this research gap. The following research questions are addressed in this study.

• 1. What types of claims are in Chinese college students’ English argumentative writings? Does the claim with more complex qualifiers correlate with argumentative writing performance?

• 2. What characteristics the types of claims show in a high-scoring writing?

Theoretical framework of this paper

Based on Toulmin’s argumentation model, qualifier and claim parts are chosen as the focus of this study. After qualifiers are classified into QW, QP and QS respectively, they are combined with claims as a whole to represent the beginning of an argumentative writing. The main expected outcome for this survey is to find out the possible types of claims that can influence the quality of argumentative writing. The features of the claim type in high-score writing is also the focus of this study. Based on the literature review, the author has designed the theoretical framework diagram for this study (Fig. 2).

Fig. 2

The theoretical framework of this paper

As shown in the figure, based on the Toulmin framework, this paper focuses on the claim together with its qualifying elements and their relationship with argumentative writing.

Research methods

Design

117 English-language argumentative essays by Chinese university students with limited time were selected as samples that can be analyzed. In this study, the overall grade for the argumentative essay is treated as a separate factor representing the overall level of students' writing. In contrast, claims and qualifiers are separated out as elements of the argumentation and analyzed in relation to the overall grade. The author attempts to identify the relationship between the argumentative elements and the overall level of writing.

Participants

The subjects were freshmen of a top university in Northeast China. They were all science and engineering students. There were 88 boys and 29 girls with an average age of 18.3 years. The writing test was conducted in December 2020. The students were asked to write a 150–200 word essay on “whether it is right for marine parks to stay open”. Students needed to write their own titles. This task was completed in 30 min in class. A total of 117 writings were collected for this study.

Procedures

The flow chart in Fig. 3 illustrates the process of this paper. After the scoring of the argumentative writing was completed, the essays were divided into three groups: high-score group, medium-score group and low-score group. Correlation was then calculated according to the complexity of the claims (taking into account the number and type of qualifiers) and the total score of the essay. Finally, the author analyzed the characteristics presented in the claims of the high-scoring essays.

Fig. 3

Procedures of this study

Data analysis

Composition review

CET4 writing standard was adopted to assess the students’ essays in terms of relevance, clarity, thoroughness and coherence of their writings. CET 4/6 is a national English test in China, which aims to provide an objective and accurate measure of the actual English language ability of university students and to provide an assessment service for the teaching of English at university. It is administered by the Department of Higher Education of the Chinese Ministry of Education.

Two experienced raters were invited to score the writings. Rater A, 38 years old, has been a university English teacher at an ordinary university in Northeast China for 14 years. Having participated in CET 4 and CET 6 rating sessions for 11 times, she has extensive teaching and rating experience. Rater B, 36 years old, has been a university English teacher at a top university in Northeast China for 10 years. She has been a rater of CET 4 and CET 6 writings for 9 times. She has a good ability to respond to students with varying levels of writing.

Claims marking

Firstly, the researcher extracted claims from the first paragraph of 117 argumentative essays. The main processing information is the background information of the first paragraph. If the background information and the claim formed a complete causal chain, then the background information sentence was regarded as qualifying elements. If the background information sentence existed independently of the core claim, the background information sentence was removed during the marking process. Then, according to Berger’s (2014) marking method for restrictive elements, the author annotated the claims in the first paragraph of 117 point compositions.

In order to ensure the reliability of the manual coding process, the author conducted two rounds of claim marking. The author first screened and marked the claims in March 2021. In order to try to avoid the influence of the first marking, the author conducted a second round of screening and marking of the claims in July 2021 and found that the following seven claims were marked inconsistently (Table 4).

Table 4 Examples of claims with inconsistent markings

After careful analysis and comparison, the results of the second marking were chosen for this study.

Interrater reliability

The scores from the two raters were recorded in an excel document and analyzed using SPSS 22.0. The correlation coefficient was 0.771, showing a high positive correlation. The final score of the students adopted in this research was the average score of the two raters. The correlation coefficient between the author’s two rounds of the claim marking was 0.886, also showing a high positive correlation (Table 5).

Table 5 Correlation coefficient of two review results

Results

After scoring the composition, the author analyzed statistically according to the scores of the composition, the highest score was 15 points, and the lowest score was 6 points. The author divided the composition into high-score group (12–15 points), medium-score group (9–11 points) and low-score group (6–8 points) (Figs. 4, 5; Table 6).

Fig. 4

Distribution of claim types

Fig. 5

Proportion of claim types

Table 6 Composition score and proportion of high, medium and low scores

As shown in the figure, there were 23 kinds of claims in the 117 compositions. In order to answer the research question (1) 1. What types of claims are in Chinese college students’ English argumentative writings? Does the claim with more complex qualifiers correlate with argumentative writing performance? According to Berger (2014), qualifying elements in sentence and participle forms (QS, QP) as major qualifiers were scored 2 points each, while qualifying elements in word forms (QW) as minor qualifiers were scored 1 point. Each claim was scored 1 point. For example, the claim of QS + QS + QW + O type was scored as 2 + 2 + 1 + 1 = 6. Next, correlation between the complexity of claims and writing scores were calculated.

As shown in Table 7 (P = 0.321, greater than 0.05). A conclusion was drawn that the diversity of claim types had no correlation with argumentative writings (Tables 8, 9).

Table 7 Descriptive analysis of writing scores and claim types

Table 8 Correlation analysis of argumentative writings and claim types

Table 9 Types of claims with high frequency

Based on the above results, the corresponding claims of compositions with more than 10 scores were QW + C (11.09), QW + C1 + C2 (10.42) and QW + C + QS (10.17) (Table 10).

Table 10 Examples of argument types corresponding to higher score compositions

Types of claims were concluded corresponding to higher score compositions began with QW. Writers began their claims with phrases expressing personal positions (for example, as far as I’m concerned, in my opinion, from my perspective,), which represented that authors themselves had a clear understanding of argumentative writings. They began their argumentative writing with the qualifying elements of their positions, setting a boundary from the previous background information. It also made the core claim more prominent.

Discussion

Some papers that applied Toulmin's argumentative model focused on the relationship between the number of claims and the overall quality of argumentative essays. Cheng and Chen (2009) found that the average number of claims in English argumentative essays of Taiwanese and American university students was 1. i.e., students used the same number of claims in both L1 and L2 writing. The author of this paper argues that claim is an element that necessarily occurs in an argumentative essay, therefore only correlating the number of claims and the quality of an argumentative essay is clearly one-sided when the number of claims does not change with grade and writing level (Crammond, 1998). This paper is an innovative study standing in the perspective of synthesis (an amalgam of claims and qualifiers). It was confirmed that there was no correlation between the claims classified by qualifiers and the quality of argumentative essays in this study. Based on this finding, the author further discussed the reasons why frequent use of some qualifiers didn’t correlate positively with the overall quality of the essay.

Since the classification of claims is based on the type and location of the qualifiers, numerous forms of qualifiers lead to a variety of types of claims. This paper summarizes 117 claims and finds that there are 23 types of claims in total. Each claim is assigned a score based on the type and number of qualifiers. The correlation analysis was conducted between the claims classified by qualifiers and holistic score. Ineffective qualifiers or overly lengthy claims can account for the disassociation between the two.

• (1) Claims with ineffective qualifiers

For example: As we know, many whales and dolphins suffer a lot and even die there, so l think it is wrong for marine parks to stay open. QS + C (N 16).

The qualifier used by this student was semantically duplicative of the reason 1 the writer used in the body paragraph. (many movies, books expose the cruel life that the whales and dolphins in marine parks. This even make them live much shorter than they should.), and because the writer used reason 2 (Human and other animals are given birth by nature, we all have the right to have freedom and live.), this led to a mismatch between the reason and the claim, because the core claim was only qualified for the cruelty of humans and sufferings of the animals. He got 9 for this writing. Although this student used a qualifier in sentence form, he neglected the role of the claim in the overall context of the essay and failed to write a clear and strong claim, which is one of the reasons why he failed to achieve a high mark in his essay.

• (2) Overly lengthy claims

For example: In my opinion, it’s not right for marine parks to say open. Despite the fact that marine parks do appeal to people especially children to a great extent, we should never turn a blind eye to the injuries animals are suffering. QW + C1 + QS + C2 (Number 24) The writer used two core claims in the first paragraph. However, he did not discuss C2 in the body paragraph and this writing was scored 9. Overly complex claims can lead to unfocused presentation of ideas and the reasons in the middle paragraph do not correspond to the claims, resulting in a poor holistic mark for this essay.

It is found that claims based on qualifier divisions were not correlated with holistic writing grades, but it does not show that all types of arguments are not correlated with writing grades. The authors sought to explore whether there was a correlation between simplified claims and overall writing grades. The relationship between high frequency types of claims and their corresponding mean scores of writing was explored. The most claims in high score compositions begin with QW + C.

The claims with QW + C type has the largest number (22) and is featured as the simplest type in this study. Writings begin with this type of claims tend to be scored higher (11.09), which is contrary to the result of Lan’s (2012) corpus study that low-level learners overuse in my opinion chunks, much more than high-level learners. The other two types of claims follow up are QW + C1 + C2 and QW + C + QS types.

QW + C + QS claim type has the number of 12. Writings begin with this type of claims tend to be scored higher (11.09). This finding confirms previous research. On the one hand, qualifiers can help soften the directness or abruptness of the claims. On the other hand, they may make the core claims obscure. Hinds (1990) proposed that the writing style of Oriental language is different from that of western language. He believes that in Oriental writing, the author expects readers to “read between the lines” and interpret the author’s intention. Because collective spirit and solidarity are more preferable in Chinese culture, Chinese people often hide their own views in their expression, which leads to indirectness in writing (Scollon & Scollon, 1997, 2001).

In the students’ claims, the number of QS before the core claim is more than that of QS after the core argumentative sentence. This is consistent with Lin’s (2019) finding that prepositional adverbial clauses are significantly more than postpositive ones. Students often use modifying elements of sentence form in writing (65.0%), which is contrary to Zou’s (2012) “Chinese ESL students use simple sentences in argument sentences”.

In general, simplified claims were more likely to get high scores, which further explains research question (1), i.e., there is no relationship between claims with many qualifiers and overall writing quality, but the ones with concise claims are more likely to achieve high scores. Besides, simplified claims are the ones with only a QW, a short and effective marker to set a boundary to make the core claim stand out. In this way, readers can get straight to the point. In other words, writers of these argumentative essays convey clear position information to the readers in the beginning, which also confirms that good essays require the writers to have reader awareness (Holliday et al., 1994; Kirsch & Roen, 1990).

Conclusion

As an exploratory study in the field of second language argumentative writing, there are some limitations to this study to be mentioned. First, subjects of this study come from one university in Northeast China, which is not sufficiently representative of the level of Chinese university students as a whole. Second, the sample size of 117 is not large enough, and the two classes selected are the university’s experimental English classes, in which the students were best at English for they were selected through an entrance English test. Therefore, this study is not sufficient to explain the whole issue of argumentative essays in English among Chinese university students. In a follow-up study, greater sample size and more extensive data are preferable as to further explore essential issues in argumentative writings among university students. In addition, this study is an innovative study that addresses the limitation of previous studies in which the number of argumentative elements alone is used to judge the quality of argumentation. There are few researches that have studied from this perspective to refer to. Therefore, subsequent text analysis or experiments are needed to further validate the results of this paper.

The study based on the text found that the overall performance of argumentative writings was not related to the complexity of the qualifiers of students’ claims. The pedagogical insight we can draw is that writing teachers can focus more on teaching claims to express more precise ideas in concise sentences. There is no definitive answer to the correlation between syntactic complexity and writing performance, but this study suggests that it is possible to focus less on the qualifiers and complexity of the claims in the first paragraph and more on whether the claims clearly present the writer's position. Writing teachers can improve their teaching of the claims in three ways.

• (1) Teachers should help raise students’ awareness of the writing topic and train them write clear and strong claims after examining it quickly. A good beginning helps students to better develop their argumentation in the body paragraph.

• (2) Writing teachers need to cultivate students’ awareness of time allocation before writing. First paragraphs are not supposed to be written as long as possible. Some students only repeat the topic information in the first paragraph or describe the picture provided in the topic, which may cause deviation from the main topic. Or they may write too many initiative-oriented statements in the first paragraph, resulting in a haphazard ending paragraph with nothing to say. At the same time, writing teachers should cultivate students’ consciousness of the time allocation in their writing.

• (3) It is also found that claims are mostly in obvious places in sentences, i.e. after qualifiers in word form and before qualifiers in sentence form. Writing teachers could provide relevant guidance on overly complex claims in students’ writing.

This study examines claim and qualifier in the Toulmin model in depth and draws out practices that can be applied to practical teaching. It also helps further expand the measurement dimension of the original argumentative model and enrich the empirical study of the relationship between the argumentative element and the overall quality of the argumentative essay. In the following study, the relationship between the other elements of the Toulmin model and the argumentative essay will be considered in an integrated manner to further enrich the theory of teaching the argumentative writing.

Appendix

Slides of the writing instructions shown in class.