From reading style to cognitive style and its possible application: an eye-tracking and CFT approach

Abstract

Reading strategies and cognitive styles have been the objective of many researchers. However, the relation between these two concepts remains unclear. This paper is focusing on three goals. Firstly, we verify the reliability of the eye-tracking indicators of risky and conservative reading style. Secondly, we aim to explore an additional eye-tracking pattern that may reflect the reading style on the global level. Thirdly, we explore the relationship between reading style and the holistic/analytic cognitive style. The study is based on a combination of Compound Figure Test (CFT) as a main tool for analyzing cognitive style and eye-tracking study consisting of several text stimuli related to verify reading style patterns. Results showed stability across the reading tasks, which validates for the usefulness and reliability of original Rayner’s as well as the new additional eye-tracking metrics. The stability of the eye-tracking metrics allows us to treat them as a behavioral profile in information processing that may be viewed as a stable personality trait known as reading style. However, the eye-tracking results and the CFT global preference score did not show a detectably close relation between cognitive style and reading style. In conclusion, we also discuss the possibility of applying these eye-tracking patterns to foreign language material design.

Introduction

In the application of technology to language education, human interaction to the information received is an important area of research. From day to day, humans use an wide variety of methods to obtain different sorts of information, with reading being a main incoming channel of information mediated in written linguistic form in language education. In literature, reading, as an integral component of foreign language education, has been studied from a wide variety of perspectives using different methodological approaches, among which eye-tracking being one of the most promising.

The eye-tracking method has been extensively applied to the research of reading patterns. For instance, it has been found that existing knowledge may impact reading patterns (Gwizdka et al., 2019) and that fixation time, reading speed and length are all good indicators of domain knowledge levels (Cole et al., 2013). Lu et al. (2018) similarly investigate the domain knowledge effect using eye-tracking, reporting that in a navigational table of contents system, the total fixation duration correlates negatively with domain knowledge.

So far, eye tracking has been extensively used in investigating the reading pattern when a group of subjects are exposed to a certain type of reading tasks. However, it remains a question yet to be explored whether the reading pattern is stable across different reading tasks performed by the same individual and can thus be called an individual trait. Nevertheless, students in higher education, especially foreign language education, are a population that receives information mainly through reading learning materials. Several concepts try to present and explain the individual differences between students from various points of view, e.g., learning styles (Pashler et al., 2009). Recent research (Vishnyakova et al., 2020) also focuses on the relation of cognitive styles and learning performance, which we will return to in the next sections.

Reading strategies and risky/conservative reading

Over the past 40 years, much has been done to investigate the reading style of skilled readers (Rayner, 1978, 1998). However, more remains to be done on the factors that influence an individual’s style of reading. In the field of eye movement research in reading strategies, it has been found that older readers generally read more slowly and make more fixations than younger readers (Kemper et al., 2004) and that older readers tend to have a symmetric perceptual span, to have a higher skipping rate of certain selected words, appearing to tend to guess those target words, and to make more regressions (Rayner et al., 2006). The older population also returns to fixate the skipped words more often, showing that their guess can go wrong and a regression and fixation needs to be made to correct the mistake. The study shows that the older population tends to exhibit a more risky reading strategy than young readers, in order to compensate for the slower lexical processing. Rayner et al. (2009) argued that the more risky reading strategy of the older population can be a consequence of the less efficient processing of non-foveal information that leads to a smaller and symmetric perceptual span. Koornneef & Mulders (2017) look at how different types of readers react to verb-based implicit causality in different ways. The authors find that risky/proactive readers slow down significantly upon fixating a bias-inconsistent pronoun, whereas conservative readers slow down much later in the sentence, and confirm that the framework of risky reading, originally designed for older readers, may be applied to younger populations as well. In particular, the authors find that proactive young readers exhibit eye-movement patterns similar to old readers and relating to an earlier eye-movement study on garden-path sentences (von der Malsburg & Vasishth, 2013), the authors postulate the potential influence of working memory capacity on reading strategies based on the fact that higher capacity readers re-read more often than their lower capacity counterparts.

However, in literature, it remains unknown whether there may exist any other factors that are potentially associated with the reading style of an individual.

Cognitive style (analytic/holistic) and its possible connection to reading patterns

Cognitive styles have received extensive scholarly attention, with the early attempts such as Witkin (1949, 1950, 1952) addressing how people perceive upright space in relation to the environment. In the rod-and-frame and the chair-and-room experiments, the subject determined the perception of the item with respect to the tilted surrounding field. It was found that the subjects showed individual differences in how they construed the item (or the focal object) against its surroundings (that is, the field) and showed consistency of such tendency across sense modalities (Axelrod & Cohen, 1961; Witkin et al., 1968). A subject can be said to have a more analytic (field-independent) cognitive style if the influence is weak and to have a more holistic (field-dependent) cognitive style if the influence of the field is strong. At a more general level, cognitive style can be seen as an individual’s adaptation to its immediate physical and socio-cultural settings (Kozhevnikov et al., 2014).

Following on from those earlier work on cognitive style (especially those in the 1960s and 70s), analytic and holistic thinking continued to attract considerable scholarly attention in the recent decades in cross-cultural psychology (e.g., Ji et al., 2004; Nisbett & Masuda, 2003; Nisbett et al., 2001). In this connection, we may return to the possible connection between reading style and cognitive style—if cognitive style is an individual trait of how an individual adapts to its environment that may manifest across various underlying domains of human perceptual and cognitive processes, then does cognitive style similarly play a role in accounting for the reading pattern of an individual? In particular, the relevant bridging question that can be raised is: If a person is more holistic in reasoning, would such personality trait drive him to try to acquire more contextual information as he encounters a sentence or a passage in discourse? At a different level, if the individual analytic/holistic distinction has been found to hold across sense modalities (Axelrod & Cohen, 1961; Witkin et al., 1968), and can be argued to be a common cognitive determiner that operates across different cognitive domains (Kozhevnikov et al., 2014), then is reading, as a central human daily activity of both lower-order and higher-order cognitive processing, similarly associated with the cognitive style of the individual learners during studying from text materials? Such discussion is still lacking and theoretically relevant.

Based on the above reasoning, we hypothesize that if a person is more holistic and field-dependent, it may be more likely for him to try to acquire more contextual information in reading, exhibiting the characteristics of a risky or proactive reader discussed in Rayner et al. (2006, 2009), which can be evidenced by long saccades and many regressions. On the other hand, if a person is more analytic and field-independent, he may tend to pay less information to the context in reading, exhibiting the conservative reading pattern, with relatively shorter saccades and less regressions. In the conclusion of the paper, we return to a discussion of possible application of the reading style to foreign language education, especially in the area of material design.

Methods

Participants

There are 21 Czech participants in our analysis—out of the 24 who participated, 3 had to be excluded because of a low gaze data quality. The age of participants varied between 19 to 28 (Me = 22, IQR = 4). The vast majority of them were female (N = 17, 81%). All participants were college students of social studies and humanities at Masaryk University. The whole procedure took around 15 min and participants were financially rewarded for their participation. The study was approved by the Research Ethics Committee of Masaryk University and all volunteers signed an informed consent.

Instruments

CFT: A diagnosis of holistic/analytic cognitive style

Compound Figure Test (hereafter “CFT”) is a method based on the Navon test (Navon, 1977) and has been extensively applied to psychological research to study the distribution of attention as an indicator of an individual’s global and local level of processing (i.e., holistic and analytic cognitive style; e.g., Čeněk et al., 2020; Kubíček et al., 2016; Lacko et al., 2020; Opach et al., 2018; Šašinka et al., 2018). Prior research showed that CFT is a valid and reliable measure with satisfactory empirical reliability of maximum a posteriori estimates, test–retest reliability, and discriminant validity with intelligence and personality (see Lacko et al., 2021). In a CFT, the participant is shown a Navon figure, which is a recognizable large number composed of many smaller numbers, and the participant is asked to identify as fast as possible the large or the small number.

The reaction times (hereafter “RTs”) for identifying the large (global) number and the small (local) one are measured and these RTs might be subsequently modeled within shifted-Wald distribution which represents a simplified process model that does not utilize accuracy since CFT tasks are very easy to solve (see Anders et al., 2016; Faulkenberry, 2017; Steingroever et al., 2021). Within the shifted-Wald distribution, three parameters are usually estimated (drift rate, threshold and non-decision time; Anders et al., 2016). However, the fourth parameter which additionally models drift rates across trial variability was described recently (Steingroever et al., 2021). For an estimation of the level of analytic and holistic cognitive style, Lacko et al. (2021) suggested applying the latter mentioned, i.e., the Bayesian four-parameter shifted Wald distribution process. The main indicator of cognitive style is the drift parameter (higher value means higher level of analytic/holistic cognitive style).

It has been found that in general, people exhibit a holistic precedence, evidenced by their faster identification of the global figures (Navon, 1977) and that there exist cross-cultural differences in the sense that East Asians as well as participants from remote cultures are faster and more accurate in the holistic processing (McKone et al., 2010) than their Western counterparts and that Himba (in Namibia, Africa) people are faster than their Western counterparts (Davidoff et al., 2008) in identifying the small figures, showing an analytic preference.

The method has already been extensively applied to investigating the connection between the perceptual-cognitive style and map reading behavior (Kubíček et al., 2016; Lacko et al., 2020; Opach et al., 2018; Šašinka et al., 2018; Stachoň et al., 2018). However, the method is yet to be applied to studying the connection between cognitive style and reading, which is the theme of this paper.

The CFT was constructed to be administered on the PCs through a software called Hypothesis (see Popelka et al. [2016] and Šašinka et al. [2017] for a description of that). In the experiment, we used four digits (2, 4, 5, 8) for the global and local numbers. The participant was asked to identify the global or the local stimulus from four options, out of which only one was correct. The choice was made with a mouse-click and the subject was instructed to react as soon as possible. Before the real experiment, the participants had been trained with three trial tasks to ensure they had understood the instructions well. The body of the experiment contained three global and three local trial tasks and 16 tasks of global processing and 16 tasks of local processing. The local tasks came before the global ones throughout the experiment. Calibration was conducted with a fixation cross displayed for 0.5 s before each trial (Fig. 1).

Fig. 1

CFT stimulus example and task procedure

Reading task

Reading strategies are an area of interest for contributive research in recent years (see Koornneef & Mulders, 2017; Rayner et al., 2006, 2009). According to the literature, there are two types of readers: risky readers, which are characterized by frequent word skipping, long saccades and a high number of regressions, and conservative readers, which produce less word skipping, short saccades and a lower number of regressions compared to the risky readers (Rayner et al., 2006, 2009).

To investigate the reading strategy of the subject, a reading task was designed. For this purpose, we prepared the stimuli in the form of two thematically similar texts on the culture of a fictional island. In the experiment, both texts had the same structure, consisting of a title, the author's name and three paragraphs with the same number of lines (which is 6) and similar numbers of sentences and words (see Fig. 2). Gaps between the paragraphs were intentionally made wider in order for precision in the measurement of eye movement.

Fig. 2

Text A with description of the paragraphs

Each paragraph’s (P1-P3) topic was adjusted in this structure—P1 (consonant), P2 (contradictory), P3 (different topics). Thus, P1 (consonant) is about the origin of the culture in both texts, P2 (contradictory) deals with the social principles of the fictional island society. However, the information written in P2 of Text A is in conflict with P2 of Text B. P3 (different topics) is about the fictional island’s agriculture in Text A, while in Text B, P3 is about its art. The purpose of such a design of the texts was to control for the thematic stability in P1 across the texts, to present a semantic contradiction (within the same topic) in P2, and to create a minimal degree of topic relevance in P3, allowing us to observe the eye movements in a controlled manner when readers were exposed to pairs of written passages that differ in their information content in different ways. The reading task was constructed to be administered on the PC via eye-tracking software SMI Experiment Center 3.7 (Fig. 3).

Fig. 3

Schematic description of Text A and Text B structure

Before the beginning of the reading experiment itself, participants had been briefly orally instructed about the eye-tracking measurement process. The eye-movement system was calibrated using 9 calibrating points with the maximum deviation of 0.5 degrees. The instructions were repeated at the beginning of the experiment. If participants understood the instructions, the experiment proceeded. The participant’s task was to read both texts carefully and remember the important information from them. To centre the participant’s gaze, before each text was presented the fixation cross for 0.5 s. The fixation cross was located at the same position as the title of the subsequent text to eliminate false saccadic movements. After reading Text A, participants proceeded to Text B by pressing the spacebar. Participants were not limited in time during the reading of both texts. The eye movements of the participants were measured during the reading of Text A and Text B. Subsequently, participants were asked to answer 7 control comprehension questions printed on paper (see Fig. 4).

Fig. 4

Reading task procedure

Eye-tracking metrics

Rayner et al., and and’s (2006, 2009) recent research of differences in reading style employ metrics that can capture an individual’s reading strategy, namely frequency of word skipping, length of saccades, frequency of regressions and position of word fixation. Based on the Rayner et al., and and’s (2006, 2009) and Koornneef and Mulders’s (2017) descriptions of proactive (risky) and conservative readers, the metrics used in our analysis are the mean length of saccades (i.e. the amplitude of each saccade in visual degrees) and frequency of regressions (number of saccades and regressions ratio). Due to the structure and design of the stimuli (size of words and spaces between lines), we decided to exclude the Rayners’ suggested metric, the word skipping due to the stimuli setting. In addition to the above metrics, we have included another one, which is the number of transitions among paragraphs (Chanceaux et al., 2014), showing the long saccades (i.e. skipping) among designed paragraphs. The rationale behind such addition is as follows: Paragraphing is considered meaningful partitioning that breaks down a text into chunks. When a reader processes the information of a particular chunk, s/he needs contextual information to help make sense of the exact information conveyed, as no paragraph exists in isolation. Thus, the reader’s transition, or with other words the number of direct saccades from one paragraph to another (e.g., from P1 to P2, see Fig. 2) provides valuable information that allows us to understand how readers process a text at a more global/discourse level, going beyond the local/sentential level that was the focus of the previous literature. Namely, participants identified to adopt a more risky reading strategy may show a higher number of transitions between paragraphs while reading the text in comparison to conservative readers.

Procedure and apparatus

The data were gathered in the HUME lab (Experimental Humanities Laboratory) at Masaryk University. The test battery was administered in the following order: 1) Demographic questionnaire, 2) Compound Figure Test, and 3) Reading task. The eye-movements were captured via SMI REDm eye tracker with 60 Hz sample rate and with the software SMI Experiment Center 3.7. In order to control for the experimental conditions, each participant fulfilled the reading task with a chinrest fixing their head placed 65 cm in front of the 22" LCD monitor. CFT was administered individually via the software Hypothesis.

Data Analysis

The initial analysis of the eye-tracking data was performed in SMI BeGaze. We have determined certain criteria for the data cleaning. Firstly, a quantitative criterion was applied: all eye-tracking records with data loss higher than 5% were excluded. In addition, we screened all eye-tracking data by excluding those that did not fit the criterion for requested gaze data quality (e.g., skewness of an eye movement record). Regarding the eye-tracking metric “number of regressions”, we have excluded from the dataset all saccades shorter than 0.3 visual degree and with the inclination from the horizontal axis (text lines) higher than ± 40 degree.

The main analysis was performed in R (v4.0.3; R Core Team, 2020), in package psych (v2.0.12; Revelle, 2020) and in JAGS module for R (see Steingroever et al., 2021). We replicated exactly the same procedure of estimation Bayesian 4-parameter shifted-Wald parameters as Lacko et al. (2021) in their CFT validation study. We used the Gibbs sampling method with three Markov Chain Monte Carlo chains and 40,000 iterations. We also used the same weakly informative priors (taken from https://osf.io/7ezax/). The dataset as well as the R syntax are available online (see https://osf.io/m9j5d/?view_only=2f4c4dd0a47b4f739b0f97e1f877c78e).

Since our sample was rather small, we used Kendall’s Tau-b rank correlation coefficients in the main analysis, because it is less biased and does not overestimate significant associations in small samples (Arndt et al., 1999) and its confidence intervals are more reliable (Kendall & Gibbons, 1990). This procedure is able to identify the lower bound of significance, which is the more rigorous, confidential and sound statistical approach to identify relationships in sample sizes similar to ours. Furthermore, all p-values were corrected by the Holm-Bonferroni method for multiple comparisons in order to reduce potential type I. error rate (Curtin & Schulz, 1998). We also used ten thousands bootstraps in order to calculate 95% confidence intervals and p-values.

Results

Since one of the aims of this study is to analyze potential relationships between cognitive style and reading style (i.e., to analyze the degree to which those variables and their changes are related), we decided to perform the correlation analysis. In the first step, we report the descriptive statistics of the main eye-tracking metrics as well as the drift parameter which represents an indicator of cognitive style (see Table 1).

Table 1 Descriptive statistics of relevant variables

Firstly, correlations among eye-tracking metrics obtained from Text A and Text B were assessed in order to provide evidence about stability of reading patterns during the reading tasks with different contents (see Table 2). All Kendall's τ_b were statistically significant and showed high or medium effect size, suggesting strong associations between the variables. These results suggest that the reading style is stable in terms of eye-tracking metrics during different reading tasks and that the reading task itself yields satisfactory test–retest stability.

Table 2 Correlations between different content of reading tasks

Regarding the correlations among different eye-tracking metrics, we did not find any statistically significant associations on the 5% of level significance (see Table 3). This finding can be attributed to the low statistical power of our study due to the small sample size, which makes it not completely advisable to interpret the null findings as evidence for the null hypothesis.

Table 3 Correlations between eye-tracking metrics

Furthermore, low to medium correlations were detected between the mean length of saccades and the frequency of regressions in both texts, although the results were not statistically significant.

Finally, the correlations between eye-tracking metrics and drift parameters for holistic and analytic cognitive style were verified. The people with higher level of analytic cognitive style tend to do less regressions (τ_b- = -0.448, p = 0.023 for a text A and τ_b- = -0.348, p = 0.080 for a text B). The rest of associations was, however, statistically insignificant (see Table 4).

Table 4 Correlations between reading and cognitive style

Discussion and conclusion

The main goal of the study was to verify whether the proposed eye-tracking metrics (mean length of saccades and frequency of regressions), which we hypothesized to manifest in the designed reading tasks, could be used as reliable tools for measurement of the reading style. Another goal of the study was to explore whether an additional eye-tracking metric number of transitions between paragraphs, might be used to detect differences in the reading strategies at a more global level. Furthermore, the possible connection between reading style and the concept of cognitive style was explored using CFT.

Our results based on the eye-tracking metrics of reading style showed strong stability over the two presented texts, which may serve as valid evidence for the usefulness of the metrics. Kendall’s Tau-b rank correlations were strong and statistically significant in all cases. These results support the assumption that considered eye-tracking metrics are manifestations of stable reading strategies, i.e. a behavioral profile in information processing that may be viewed as a relatively stable personality trait known as reading style. These results are supported with previous studies focusing on reading strategies patterns (e.g. Hyönä & Nurminen, 2006; Rayner et al., 2006, 2009). In our experiment, the frequency of regressions and the level of analytic cognitive style showed a statistically significant relation which corresponds to our expectations. However, other eye-tracking metrics and the drift parameters for holistic and analytic cognitive style did not show a detectably close relation between cognitive style and reading strategy. Kendall’s Tau-b rank correlations were weak and insignificant between the eye-tracking metrics and the CFT global preference score in most of the cases. These results cannot prove our hypothesis of the relevance between these two concepts.. Therefore, we consider further study much needed in order to investigate the relation between cognitive style and reading style. Future research could also include other eye-tracking patterns such as look-back, re-reading or progressive first pass fixation time suggested by Hyönä and Nurminen (2006).

One of the main limits of the study is its limited sample size, which could lead to a slight bias in the results. We tried to alleviate this deficiency by increasing the accuracy of eye-tracking records included into the analysis and the quality of the statistical processing, incorporating strict criteria. Another limit stemming from the sample size lies in the small statistical power, determining that we were able identify only large effect sizes. Another limit lies in the small number of CFT items (16 items per subtest). Since at least 50 items per subtest are recommended, the estimation of drift parameters might be unreliable (Anders et al., 2016).

The results described above confirm that selected eye-tracking metrics may indeed serve as a suitable indicator of reading style following Rayner’s proposal (see Rayner et al., 2006, 2009). In addition to the confirmation of Rayner’s selection of behavioral metrics, the added eye-tracking metric, i.e. the number of transitions among paragraphs, showed stability in the reading of both texts. We, therefore, consider that a useful tool showing the relatively stable features of the reading strategy on a more global level and propose a possible implication for foreign language education in this regard. As for the possible connection between cognitive style and reading strategy, no significant relation was found. Our results support the existence of stable individual differences in the art of reading and these findings raise the practical question of how to adjust the form of information presentation and design in learning materials in foreign language education for different types of learners in order to offer optimal ways of receiving information and learning. Namely, our study suggests that there exist individual differences in reading style that can be manifested by our selected eye-tracking metrics and that such eye-tracking metrics may be useful in improving the design of language teaching materials.

Our results have shown that reading strategies are stable across similar tasks. We have demonstrated a stable feature—the way of reading the same structured task in an individual. Nowadays, a huge amount of information is perceived through reading and especially during primary and secondary education. Based on our findings, is it possible to distinguish types of reading strategies that would be more convenient for various learning tasks in the field of foreign language education (i.e. preparation for written exams)? Do the readers differ in their level of flexibility to adapt their reading strategy? If so, is it possible to teach the readers with a rigid reading strategy to adapt this strategy to different types of tasks on purpose?

Based on our findings, we are able to detect the reading strategy of an individual when reading a specific task. This finding could be used for screening of reading strategies in young learners and can also serve for future development of sophisticated diagnostics of reading.