Thinking Skills and Creativity 37 (2020) 100674

Contents lists available at ScienceDirect

Thinking Skills and Creativity

journal homepage: www.elsevier.com/locate/tsc

Integrating critical thinking into the classroom: A teacher’s

T
perspective
Martín Cáceres*, Miguel Nussbaum, Jorge Ortiz
Computer Science Department, School of Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile

ARTICLE INFO ABSTRACT

Keywords: Studies on critical thinking often overlook the work done by teachers in the classroom to develop
Critical thinking this skill. However, studying critical thinking from the teacher’s perspective is key to closing the
Teaching practices gap between theory and practice. This study looks to characterize the work done by Spanish-
Teachers' perspectives speaking teachers in Latin America to integrate critical thinking into their practice. To do so, an
Inservice teachers
online survey was sent to the participants of a Massive Open Online Course (MOOC), asking them
Curriculum enaction
to describe their work with critical thinking in their classroom. Responses from 278 participantes
were then analysed using an exploratory sequential design. During the qualitative phase, a series
of categories emerged based on the topics chosen by the teachers, the activities they proposed,
and how they linked these to the development of critical thinking. The frequency with which
these categories appeared in the teachers’ responses was then measured during a subsequent
quantitative phase. Variations in these frequencies were also analysed based on the subjects
taught by the teachers and the age of the students. The results reveal that teachers primarily try
to develop their students’ critical thinking skills by integrating them into their subjects; not
teaching them separately. Furthermore, they do so by choosing topics that help the students
understand the world better from different subject-specific practices. Therefore, when im-
plemented by teachers, critical thinking is highly dependent on the subject. These results suggest
that there is a mismatch between educational practice and existing research, which tends to
advocate the specific and explicit teaching of critical thinking, whether as a separate subject or
through a cross-curriculum approach.

1. Introduction

Different education systems have highlighted the importance of citizens acquiring the necessary skills to participate in an ever-
changing and increasingly-complex world (Pellegrino & Hilton, 2012). This includes critical thinking, which is thought to strengthen
democracies and allow citizens to actively participate in an economy that requires increasing levels of preparation (Behar-Horenstein
& Niu, 2011). Furthermore, mastering critical thinking is a better predictor of successful life decisions than other factors, such as
intelligence (Butler, Pentoney, & Bong, 2017; Kuhn, 2018).
Several issues have been addressed by the research into critical thinking over the last few decades. This includes how to define,
measure and develop critical thinking, as well as analysing the effectiveness of different teaching strategies for developing said skill
(Larsson, 2017). Despite the wealth of research, there is still little evidence regarding what teachers are effectively doing in the
classroom in order to teach critical thinking (Abrami et al., 2014; Larsson, 2017). This is especially true in school settings, even

⁎
Corresponding author.
E-mail address: [email protected] (M. Cáceres).

https://doi.org/10.1016/j.tsc.2020.100674
Received 29 October 2018; Received in revised form 1 May 2020; Accepted 28 May 2020
Available online 04 June 2020
1871-1871/ © 2020 Elsevier Ltd. All rights reserved.
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

though it has been proven that critical thinking can be developed from a young age (Kuhn, 1999).
Understanding educational phenomena from everyday teaching practice can help bridge the gaps between research, public policy
and continuing professional development (Farley-Ripple, May, Karpyn, Tilley, & McDonough, 2018; McIntyre, 2005; Vanderlinde &
van Braak, 2010). Given its importance, this study aims to reduce the gap between research and teaching practice when it comes to
critical thinking. This is done by exploring the work that teachers are doing to develop critical thinking in the classroom. Our research
has a practical focus and is guided by ongoing discussions in the literature. We look at the topics teachers choose to teach critical
thinking, the activities they propose for their students and how these activities relate to the development of this skill. Furthermore,
we also look at how these practices vary according to the subject that is being taught and the age of the students.

2. Literature review

2.1. Definitions of critical thinking

Given the variety of definitions for critical thinking and the lack of consensus among experts, previous authors have proposed
different ways of classifying these definitions. Lai (2011) suggests that the definitions should be grouped according to the author’s
field. Furthermore, she identifies three main approaches when trying to define and integrate critical thinking into education: the
psychological approach, the philosophical approach, and the educational approach.
According to Lai (2011), definitions from the field of psychology tend to refer to cognitive skills. In this sense, critical thinking is
understood to be a process. As such, the definitions therefore focus on the mental activities required when using this skill. One
example of this is the definition proposed by Dwyer, Hogan, & Stewart (2014), who define critical thinking as “a metacognitive process
that, through purposeful, reflective judgment, increases the chances of producing a logical conclusion to an argument or solution to a problem”
(p. 43).
Definitions from the field of philosophy have their roots in ancient Greece and Socratic philosophy, both of which are still valid to
this day. This approach focuses more on the result of critical thinking, rather than on the process itself. Two of the most frequently
cited definitions from this category include Ennis (1985), who considers critical thinking to be “reflective and reasonable thinking that is
focused on deciding what to believe or do”, and de Paul (2009), who views it as “disciplined, self-directed thinking that exemplifies the
perfections of thinking appropriate to a particular mode or domain of thought”.
Finally, in the field of education critical thinking is used interchangeably with the concept of higher-order thinking, from Bloom’s
taxonomy (Anderson et al., 2001). According to Bloom’s taxonomy, cognitive skills can be classified according to their level of
complexity. In this sense, higher-order thinking skills therefore correspond to the levels of analysis, synthesis, and evaluation.
Barnett (1997) acknowledges the difficulty of defining the concept of critical thinking, arguing that it depends on how the skill is
used. He therefore identifies at least four different ways of using critical thinking: as disciplinary competence, as practical knowledge,
as political engagement, and as a form of strategic thinking. Together with the multidimensional nature of the proposed constructs,
this range of definitions ensures that critical thinking is a difficult concept to operationalize (Bensley et al., 2016). As a result, instead
of discussing what critical thinking means in theory it is perhaps more important to understand what it translates to in practice (Davies
& Barnett, 2015; Moore, 2013). Moore (2013) suggests that research into critical thinking has tended to define the skill in abstract
terms, separating it from its practical uses. Davies and Barnett (2015) add that critical thinking is often reduced to a series of mental
processes that take place on an individual level, rather than looking at it within the context of social interactions. In the following
section, we will therefore look at how critical thinking can be integrated into the classroom by in-service teachers.

2.2. Integrating critical thinking into classroom instruction

Regardless of its definition, several different strategies have been proposed for teaching critical thinking in the classroom. Ennis
(1989) provided a classification that is still applicable today (Abrami et al., 2014), dividing these teaching strategies into three
groups: general, infusion and immersion.
The general approach suggests that critical thinking is a cross-curricular skill that requires specific knowledge of how it works. The
teaching of critical thinking must therefore focus on explicitly teaching its guiding principles, as well as putting the skill into practice
through exercises that promote its use. Pioneering studies into critical thinking focused on this type of intervention (Paul & Elder,
2009).
Infusion strategies also suggest explicitly teaching the principles and practices of critical thinking, albeit integrated into another
subject. Robert Swartz (Swartz & Parks, 1994) has produced plenty of material for integrating critical thinking into key areas of the
curriculum, as well as suggesting how to select the most suitable topics for doing so. Alan Bensley and Spero (2014) worked with a
class of undergraduate psychology students to prove the effectiveness of explicitly teaching different methods for developing me-
tacognition and analysing arguments, both of which they consider to be essential components of critical thinking. Finally,
McLaughlin and McGill (2017) proposed teaching a school-level history class how to identify pseudoscientific information, under-
standing critical thinking to be the ability to question proposals based on evidence.
The third group of strategies, referred to as immersion, suggest that critical thinking can be developed by teaching subjects
rigorously and inviting students to question and engage in deep thought. However, they do not believe in the need to outline the rules
for critical thinking explicitly. Authors who adhere to this approach suggest involving students in rigorous thinking, an exchange of
ideas, or civic engagement (Huber & Kuncel, 2016).
In their meta-analysis, Abrami et al. (2008) conclude that the most effective strategies are those that explicitly teach critical

2
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

thinking while tying them to a specific subject. Despite the wealth of literature recommending that critical thinking be taught
explicitly, there is little evidence on the transferability of such knowledge and skills to different contexts or domains (Pellegrino &
Hilton, 2012). There has therefore been growing interest over recent years in designing strategies for specific subjects, such as
psychology (Stupple et al., 2017), healthcare (Carvalho et al., 2017), and the humanities (McLaughlin & McGill, 2017). Furthermore,
there are even fewer examples of how to integrate the teaching of critical thinking into different subjects in school settings. In
addition to the study by Swartz and Parks (1994), one exception to this is the work by Cargas, Williams, and Rosenberg (2017)), who
propose designing a rubric to define achievement criteria based on critical thinking standards that can be used in different subjects.
Given this research gap, the present study will focus on teaching practices in school settings; specifically real-world classrooms led by
in-service school teachers.
Several strategies have been developed by critical thinking experts for integrating and developing critical thinking into the
classroom. These can be classified into four categories (Abrami et al., 2014). The first of these, individual study, includes strategies
where students must work on their own on the proposed activities. The second, dialogue, is characterized by an emphasis on dis-
cussion. The third, authentic or anchored instruction, refers to the selection of authentic problems or situations that are of interest to
the students. Finally, mentoring, refers to strategies that bring together a subject matter expert with novices. Abrami et al. (2014)
suggest that the best results are obtained by combining dialogue with authentic or anchored instruction. However, there are few
studies on how this is performed by in-service teachers in actual classrooms.

2.3. Critical thinking in practice

The definitions, classifications and strategies described above have been used to design a series of interventions, as well as to
study their impact (Abrami et al., 2014). This has provided an evidence base for how to effectively develop this skill. However, there
has been little research into how in-service teachers integrate critical thinking into the classroom (Davies & Barnett, 2015), other than
the interventions designed by academics (Abrami et al., 2014; Moore, 2013). In order for research to be useful to educators it must
focus on the teachers’ practices (Farley-Ripple et al., 2018), as well as on understanding the teaching and learning practices involved
in critical thinking (McIntyre, 2005; Vanderlinde & van Braak, 2010).
Of the studies that do focus on critical thinking from a teacher’s perspective, some are based on higher education, while others
focus on school settings. The studies in higher education conclude that what is understood by critical thinking largely depends on the
cultural context (Chen, 2017). Similarly, Howe (2004) compares the beliefs held by Canadian and Japanese school teachers regarding
critical thinking, concluding that the conceptualization of this skill varies between cultures. Within the field of healthcare, where
research into critical thinking has a proud tradition (Carvalho et al., 2017), one outstanding study is Huang, Lindell, Jaffe, and
Sullivan (2016)). In this study, the authors look at the what, how and why of teaching critical thinking in health-related degree
programs in higher education. The results of this study suggest that the lecturers who were interviewed focus on teaching “mental
habits”, such as the use of higher-order thinking skills or metacognition. They also teach critical thinking by rigorously implementing
the principles of clinical practice, doing so to ensure that their graduates go on to become better healthcare professionals. Finally, at
school level, one study reveals that teachers in Jordan do not know what critical thinking is, despite the stated need for students to
develop this skill (Bataineh, 2009). Teachers in the USA, on the other hand, highlight the need to develop content for teaching critical
thinking skills (Reynolds, 2016).
This wide range of definitions, classifications, and strategies for teaching critical thinking suggests that many of the debates in the
literature are still ongoing. Despite there being some level of agreement on how important and beneficial it would be to address these
issues from the perspective of teaching practices, there are very few studies which do so. It is therefore hugely important to study the
development of these skills among school-age students (Reimers & Chung, 2016) and from the perspective of teaching practice. In this
sense, our main research question asks “What are teachers doing to integrate critical thinking into the classroom?”. More specifically,
our study aims to answer this question by looking at which topics teachers use with their students, the activities they expect them to
complete, and how these activities relate to the development of critical thinking. Furthermore, given the importance of the subject
that is being taught (Abrami, 2014) and the age of the students (Kuhn, 1999), the study will also look at how the integration of critical
thinking varies depending on these two variables.

3. Methodology

3.1. Participants

Convenience sampling was used to select the participants from a database of teachers enrolled on the online course “Towards a
constructivist approach in the classroom”1 . This course was delivered on the Coursera platform and was aimed at Spanish-speaking
teachers. Furthermore, the course was designed for teachers who are interested in developing constructivist learning environments.
The course covers topics such as skill development, classroom culture and integrating technology into the classroom. An invitation to
participate in this study was sent to all 29,216 participants enrolled on the course. Table A1 in Appendix shows the demographics of
the database to which the survey was sent. The invitation included an explanation of the objectives of the study, while the procedure
was approved by the ethics committee at the lead researcher’s university.

1
https://es.coursera.org/learn/aulaconstructivista

3
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

Table 1
Participants’ demographics.
Age of the students N (Percentage) Subject areaa N (Percentage)

5 to 13 years 104 (37.4 %) Mathematics 38 (13.4 %)

14 to 18 years 174 (62.58 %) Natural Sciences 85 (30.6 %)
Social Sciences 57 (20.5 %)
Age of the participants N (Percentage) Humanities 85 (30.57 %)
18−24 17 (6.1 %) Other 13 (4.67 %)
25−34 46 (16.5 %)
35−44 73 (25.2 %) Gender N (Percentage)
45−54 94 (32.8 %) Male 176 (63.3 %)
55−64 41 (14.7 %) Female 95 (34.2 %)
> 65 7 (2.5 %) Other 7 (2.5 %)

Participant’s country of residence N (Percentage)

Chile 58 (20.9 %)
Mexico 57 (20.5 %)
Colombia 27 (9.7 %)
Ecuador 27 (9.7 %)
Peru 24 (9.6 %)
Guatemala 6 (2.2 %)
Honduras 6 (2.2 %)
El Salvador 6 (2.2 %)
Dominican Republic 6 (2.2 %)
Venezuela 6 (2.2 %)
Other 34 (12.2 %)
a
Table A2 in Appendix includes a description of the subjects comprising each area.

A total of 380 people answered the survey in full, over a period of four months. Responses from participants who taught in higher
education were not considered (88). Nor were responses from participants whose answers did not provide the information that was
requested (14). Therefore, a total of 278 responses were analysed. Table 1 shows the demographics of the participants in this study.

3.2. Procedure

The present study uses a mixed method approach called exploratory sequential design (Creswell & Plano Clark, 2018), that
includes both a qualitative and a quantitative phase. During an initial qualitative phase, a series of categories were constructed from
the data that would then be applied during a subsequent quantitative phase. This method is used when there is little empirical
evidence available on the research topic, as well as an aim to develop and test a new instrument, variable, framework, taxonomy,
intervention, or technology (Creswell & Plano Clark, 2018). As described in the introduction, there is currently a lack of research into
teaching practice when integrating critical thinking into the classroom. The first phase of the study will therefore be a qualitative
exploration of the descriptions of how teachers integrate critical thinking into the classroom. From this initial exploration, the
qualitative findings will be used to build a taxonomy in response to the main research question: “What are teachers doing to integrate
critical thinking into the classroom?”. More specifically, our study aims to answer this question by looking at the topics teachers use
with their students, the activities they expect them to complete, and the reasons teachers give when explaining how these activities
relate to the development of critical thinking. An analysis of these three components (topics, activities, and reasons) will be used to
develop a response to our main research question. In a subsequent quantitative phase, the taxonomy will then be used to look at how
the responses vary depending on the subjects being taught and the age of the students.
When studying previously unexplored phenomena that require an interpretative and inductive focus, Twining, Rachelle,
Nussbaum, and Tsai (2016)) recommend adopting a qualitative approach. Therefore, following this approach, an online survey was
designed to include both closed and open-ended questions. The open-ended questions were designed to gather information on the
teachers’ practices, while the closed-ended questions were designed to gather information on the participants’ demographics
(country, age, the subject they teach, age of their students). The survey was sent to the participants enrolled on a MOOC (Massive
Online Open Course), with an e-mail invite sent using the MOOC’s internal messaging system. The invitation was sent twice, while the
survey remained open for a period of three months. The majority of responses were submitted during the first few days after each
invite was sent. A description of the MOOC can be found in section 3.1.
The responses to each of the open-ended questions were analysed using an inductive methodology based on emerging categories
(Glaser & Strauss, 2017) and a technique for analysing eclectic content (Saldaña, 2015). Emerging categories allow different phe-
nomena to be understood through situated analysis (Glaser & Strauss, 2017), which is the aim of the present study. In general terms,
the categorization used corresponds to a process of structural analysis (Guest, MacQueen, & Namey, 2012), where the answers to
analytical questions are sought from within a corpus. The first, exploratory phase of analysis involved a process of open coding
(Corbin & Strauss, 2008), where the responses were broken down into units of interest in order to be compared and contrasted. The
approach that was adopted during this process is known as in vivo coding. This approach looks to respect the terms used by the

4
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

respondents and understand the practices of particular groups of interest (Charmaz, 2006). Following this, a process of subcoding
(Gibbs, 2007) was then used to conceptually group together the codes that were generated during the first phase. For the second
round of analysis, a process of focused coding was carried out (Charmaz, 2006). This technique consists of grouping together different
concepts based on semantics in order to define the categories for a taxonomy.
Pair coding was used to ensure the rigorous nature and reflexivity of the coding process (Twining et al., 2016). In this case, the
pair consisted of the lead researcher, whose research interests focus on strategies for developing critical thinking, and a research
assistant, who had no knowledge of the literature on critical thinking but did have previous experience in qualitative research.
Meetings were held during each phase of the coding process, during which the coding process was discussed, the results were
compared, and decisions were made on how to proceed. These decisions were made by reviewing the researchers’ notes from their
individual work, including their doubts, reflections, and clarifications on the coding process. Each phase was then iterated until there
was 100 % agreement on the codes and classifications. Furthermore, the categories were then validated by another researcher, who is
an expert in the design and implementation of critical thinking research in schools. Given that the present study aims to develop
categories from the participants' perspective, the role of the expert was not to guide the analysis. Instead, it was to validate whether
the questions on the survey could successfully prompt responses from the participants and, following the analysis, to validate the
meaning of the categories.
The closed-ended questions were used to gather information on the participants’ demographics, such as their country of residence
and gender, as well as the subject they teach and the age of their students.
During the quantitative phase, a descriptive analysis was carried out based on the categories that were developed in the quali-
tative phase and the participants’ demographics. In order to do so, the frequency with which each category appeared in the parti-
cipants’ responses was calculated. Finally, the relationships between the responses to the three open-ended questions were also
analysed. In order to explore the relationships between the different categories, these were visualized using the R package ‘ggalluvial’,
while the frequency was counted using a program in Python. A chi-squared test (McHugh, 2013) was used to see whether the
differences in the frequency of responses for each category was significant when separating by subject or student age.

3.3. Instrument design

An online survey was designed with the aim of understanding what teachers do to develop critical thinking among their students.
This survey included both closed and open-ended questions. Online data collection has the advantage of being able to reach a large
number of participants in a short space of time, considerably reducing the costs of field work (Lefever, Dal, & Matthíasdóttir, 2007).
However, there are some concerns when it comes to using this approach for academic research, such as samples being biased towards
participants who have access to internet, high non-completion rates or fraudulent responses (Lefever et al., 2007). Nevertheless,
because of its exploratory approach, an online survey was used for the present study. This is because it allowed the researchers to
reach a considerable audience and gave the participants the opportunity to describe their perspectives.
Open-ended questions are considered appropriate for studies that look to produce a wide range of information. This is because
they allow the topics of interest to come from the participants themselves (Reja, Manfreda, Hlebec, & Vehovar, 2003). The in-
formation on the participants’ demographics was obtained through a series of closed-ended questions, which are more appropriate for
gathering accurate information.
Given that the main aim of the present study is to understand how critical thinking is used in the classroom, rather than to focus
on theoretical definitions, the teachers were not explicitly asked for a definition of critical thinking (Moore, 2013). To understand the
teachers’ practices, they were asked three open-ended questions (from now on Survey Questions) about an activity that they had used
in the classroom to develop critical thinking. These questions were developed and validated by the research team, following the
literature review. A pilot study was conducted with a small group of people with classroom experience (N = 15) so as to ensure that
the survey provided the information that was needed to answer the research questions. The participants in this pilot study were also
selected through convenience sampling. Once the results of the pilot had been analysed, confirming that the responses would allow
for the kind of analysis proposed in this study, the survey was then sent out to the participants enrolled on the MOOC “Towards a
constructivist approach in the classroom”.
Firstly, to encourage the participants to describe how they integrate critical thinking into their teaching practice, the following
question was presented: “Which specific question or problem have you found to be useful for developing your students’ critical
thinking skills?” (Survey Question 1).
While the responses to Survey Question 1 provide a general overview of the activity itself, they do not reveal what the students
actually do during the activity. Simply being exposed to a question or problem will not lead to the development of critical thinking.
Instead, the students must carry out an activity in order to answer the question or solve the problem (Moore, 2013). To explore this
idea in more detail, the teachers were then asked the following question: “What did the students do in order to answer the question or
solve the problem?” (Survey Question 2). This question allows us to go beyond mere topics and instead helps us understand which
actions the teachers feel develop their students’ critical thinking.
To understand the relationship between critical thinking and the activity described in Survey Questions 1 and 2, a third question
was then proposed. This question asked the teachers to explicitly link the activity that they had described to the development of
critical thinking. The question therefore asked the following: “Why do you think that the activity described in the previous question is
useful for developing your students’ critical thinking?” (Survey Question 3).
The combination of the responses to these three questions provides an overview of what teachers are doing to develop their
students’ critical thinking and therefore allows us to answer our main research question.

5
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

4. Results and analysis

4.1. Qualitative analysis: creating categories

According to the qualitative methodology described above, the results must emerge from the analysis. The following section
therefore describes how the analysis led to the main results.

4.1.1. What are the questions or problems about?

Based on the responses to the question “Which specific question or problem have you found to be useful when developing your students’
critical thinking skills?”, our aim was to discover the topics teachers use to develop their students’ critical thinking. It is therefore
important to first analyse the topics that teachers choose as the setting for developing their students’ critical thinking. As described in
the introduction, there are different views on how to develop critical thinking. It can be regarded as a cognitive process, a disciplinary
approach, or a way of understanding everyday problems, among others. Digging a little deeper into the topics chosen by teachers can
therefore provide information on which approach they follow. To obtain this information, we asked the following analytical question
and applied it to the responses to Survey Question 1: “What are the questions or problems about?” During the first phase of coding, the
fragment of the response where the teacher refers to the topic of the question or problem was rewritten in the same terms used by the
participant. To demonstrate the coding process, we will use the response given by a high-school natural sciences teacher. This
teacher’s response to the first question was “Talking about the consumption of psychoactive substances by producing a short film”. The
result from the first phase of coding was therefore “consumption of psychoactive substances”.
Having analysed all of the responses, the second phase of coding then took place. This consisted of developing a list of concepts. In
order to develop this list, a meeting was held to analyse the researchers’ notes and propose a list of possible concepts. The concepts
which covered the largest number of responses were then chosen. For the example response given above, the corresponding topic was
“issues that concern the students”. Following this, in the second phase of analysis, these concepts were then grouped into categories. In
this case, we realized that there were a lot of questions that covered a range of issues. This included local issues, global issues and also
hypothetical issues. These were all grouped together in the category labelled as “Issues”. Table 2 shows the categories that were
developed during the coding process.

4.1.2. What do the students have to do?

Based on responses to the question “What did the students do to answer the question or solve the problem?”, we analysed the
responses describing the type of activities teachers designed to develop their students’ critical thinking. This analysis focused on the
main verbs that were used in these responses, as the verbs state the actions that were carried out by the students. This analysis
focused on the main verbs that were used in these responses. This is because such verbs tend to describe the actions that were carried
out by the students. To guide the in vivo coding process, the following question was answered: “What do the students have to do?” For
the first phase of the analysis, the section of the response containing the main verb was extracted or paraphrased. It is worth noting

Table 2
Topics chosen by the teachers in order to develop their students’ critical thinking.
Category Description Examples N (278)

Core concepts The topics in this category are inherent to the subject that is 1 Can everything we observe be measured? 115 (41.4
being taught. This category includes the core topics of a 2 What is art? %)
subject, as well as any activities that require students to 3 Could life have started through spontaneous
master a subject-specific skill. generation?
Issues This category groups together topics that refer to real-world 1 What are the chances of our country 83 (29.9
issues, whether realistic or hypothetical. This includes local, implementing clean technologies? %)
national and global issues, as well as personal and 2 What arguments do you have in favor of
hypothetical issues. decriminalizing the consumption of drugs?
3 Do social networks decide for us?
Metacognition These topics relate to the students’ mental processes. They 1 What do I know? What do I want to learn? What 36 (12.9
include learning itself, as well as the students’ thoughts and did I learn? %)
behaviour, both in and outside of school. 2 How could you change the process in order to
get a different result?
3 What makes you say that?
Usefulness of the acquired The questions in this category refer explicitly to how the 1 What’s the point of me learning this? 30 (10.8
knowledge topics that are being covered relate to everyday life, as well 2 How does learning about biology help with the %)
as linking theory with practice. economic and environmental development of
this country?
3 How can you relate what you have learned to
your own life?
Personal interests These topics focus on issues that are relevant to the students’ 1 What’s your goal in life and what are you doing 13 (5%)
long-term futures, as well as to their own interests, opinions to achieve it?
or beliefs. 2 What more would you like to learn about what
we studied?
3 What do I most enjoy doing?

6
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

Table 3
Activities that the students had to do in order to answer the questions or solve the problems posed by the teachers.
Category Description Examples N (311)

Analyse Detailed examination of a topic. Includes in-depth reading and 1 They observed and looked for everyday outdoor objects 84 (27 %)
critical analysis of cultural objects, such as works of art or throughout the city and started to describe old buildings, such
audio-visual materials. as the church and the municipal library.
2 They engaged in critical/analytical reading of a text.
3 3. They analysed different works of art.
Inquire Carrying out a full-scale or mini research project. This 1 They defined a hypothesis based on their experience and then 58 (18.6
category groups together different ways of applying the tested this hypothesis with an experiment. %)
scientific method and using different means to search for 2 They made a list of what they knew and didn’t know about the
information. topic. They then researched what they didn’t know using
different sources, analysed the information and gave an
opinion.
3 3. They researched in books and on the internet and then asked
people who were involved.
Create Creating a product, whether building something physical or 1 They wrote a script and then put together a series of scenes and 49 (15.8
developing an original conclusion at the end of a process. reflections. %)
2 They wrote a short story or piece of creative writing.
3 3. They developed their own algorithm in order to solve a new
problem.
Reason Developing or exchanging arguments. Includes debates, 1 They had a debate on a topic. 45 (14.5
discussions and analysing arguments. 2 They had a group discussion. %)
3 3. They looked at possible arguments and the strength of the
reasoning behind them. This starts with a brainstorm and
becomes more and more complex as the arguments start to run
out.
Express Oral or written expression. Usually involves answering a 1 Several students gave their opinion on the story, making 30 (9.6
question orally or in writing. suggestions as to what they would have done in the character’s %)
position.
2 In a group class, the students gave their opinions in response to
the questions that were asked.
3 3. They listened to the statement and then raised their hand to
give an opinion.
Reflect Introspective process of deep thought. Thinking carefully 1 When reading the text, the students questioned it based on 29 (9.3
about a topic and considering different points of view. their own reality. %)
2 The students reflect at the start of the class using a question
prompt.
3 3. They reflected on how they might solve a problem from
different angles or in different situations.
Collaborate The focus is on pair or group work, where priority is given to 1 They worked in groups of 5, talking about their families’ 12 (3.9
the exchange of ideas or working together on a solution. current economic difficulties and how they hoped to help. %)
2 They worked in groups of 4, with each student giving an
answer, then they had to develop an outline using the 4
answers.
3 3. They talked in small groups.
Apply Putting into practice a procedure or concept in a different 1 They read several times, collected data and then turned the 4 (1.3 %)
context or in a different way than was explained by the data into equations.
teacher. 2 Based on the concept of fractions, they looked for
equivalencies, applying these concepts to their own lives.
3 3. They used an abacus to practice additions.

that in several cases there was more than one main verb. In these cases, the responses were assigned two categories through a process
of simultaneous or concurrent coding (Miles & Huberman, 1994). In the case of the example participant mentioned previously, their
response to this question was the following: “The students met each week in order to find out where people consumed [drugs] at school and
spoke with some of their classmates in order to understand their perception of psychoactive drugs. Using this information, they wrote a script
and put together a series of scenes and reflections.” The result of the in vivo coding for this response was “Following some inquiries, the
students made a short film.” Having compared this and other codes obtained during this phase of the study, the concepts “Create an
audio-visual product” and “Make inquiries” were suggested as umbrella terms. Following the second round of coding, the categories
that emerged from this and similar responses were “Create” and “Inquire”. Table 3 shows the categories that were generated by
analysing the responses to the question “What did the students do to answer the question or solve the problem?”

4.1.3. How does the activity develop the students’ critical thinking?
Responses to the question “How do you think that the activity described in your response to the previous question helps your students
develop their critical thinking?” were used to identify how the teachers justify that the activities address building critical thinking.
During this analysis, we noticed that the teachers tended respond this questions by describing the effect that the activities had on
their students. We therefore took the main verb used in the teachers’ responses as their justification for using the activity to develop

7
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

Table 4
Reasons given by the teachers to explain how their activities develop the students’ critical thinking.
Category Description Examples N (321)

Understand Refers to how, at the end of the process, the students have 1 Because when they start the experiment they realize 86 (26.8
improved their understanding of a phenomenon or that there are several, interrelated variables. %)
concept, enriched their view by incorporating other 2 Because, although it’s an everyday activity, they are not
perspectives, or become aware of an aspect of reality that aware of how much water is consumed in their homes,
they had not previously considered. nor whether or not it is wasted. By analysing the
situation, they become more aware of this situation.
3 3. Because they listen to and analyse several points of
view and possible explanations for the cause of the
problem.
Develop cognitive skills Refers to the students putting into practice one or more 1 It’s useful because, by analysing the situation, they have 69 (21.5
cognitive skills, i.e. higher-order thinking skills according to comprehend, assess and judge it. %)
to Bloom’s taxonomy. 2 Because it undoubtedly requires them to apply different
levels [of thought], from basic knowledge through to
analysis.
3 3. Because they don’t just memorize. Instead, they have
to relate what they’ve learnt to real life. In this sense,
they take the time to read, analyse, draw conclusions,
ask questions, clarify doubts and exchange points of
view, all of which helps them to be more creative.
Reflect Refers to the students having engaged in a process of 1 Because it encourages them to think about things in 64 (19.9
deep, introspective and meditative thought. more detail and analyse all of the issues surrounding %)
them.
2 Because any activity that makes the students reflect
helps them develop critical thinking.
3 3. Because they start to question whether what they
believe or hear is correct.
Work autonomously Focuses on the importance of the students playing a 1 Because it makes the students reflect on and search for 54 (16.8
leading role in the process. something themselves, without there being a right %)
answer. The aim is for each student to give a justified
answer, based on what they found.
2 Because they can build their own knowledge.
3 3. Because it allows them to give an explanation based
on their own criteria.
Develop social skills Reveals the importance of putting into practice the skills 1 As feelings are subjective, when working on this topic 20 (6.2
that are required to work with others or contribute more the students have to be respectful and empathetic, as %)
to society. well as being mindful when giving their opinion. They
also have to learn to contribute at the right time and not
just to say any old thing. The students are therefore
forced to think about what they say and to reflect. They
learn to listen and to develop their opinion, based on the
context.
2 Because that way they participate or contribute to
society as humans and agents of change.
3 3. Because it teaches them to respect each other’s
opinion and opens their minds by exposing them to
several different responses.
Reason Suggests that the process of logical reasoning is key to 1 Because it allows the students to debate, analyse and put 16 (5%)
developing critical thinking. forward different points of view
2 Because it leads them to back up their thoughts or
opinions with evidence.
3 3. Because it develops their reasoning skills.
Transfer Focuses on the importance of applying knowledge to real 1 Because they transfer their knowledge to everyday life. 12 (3.7
life. 2 Because it allows them to link the topics covered in class %)
to their own context and/or issues.
3 3. Because they don’t just acquire information; they
learn how to apply it, too.

critical thinking. These verbs were also used to generate the codes, concepts and categories for subsequent analysis. Continuing with
the example described above, in response to this third question, the participant in question answered, “Because they answered their
own questions about the use of psychoactive substances while at the same time setting themselves the task of producing something that would
allow their classmates to reflect on the issue”. The first round of coding produced the following: “Because the activity led the students to
answer their own questions, and they produced something for others to reflect upon”. The category to come from this type of response was
“Work autonomously”. Table 4 shows the categories that were defined based on responses to the question “How do you think that the
activity described in your response to the previous question helps your students develop their critical thinking?”

8
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

Table 5
Comparing topics by subject.
Mathematics (N = 38) Natural Sciences (N = 83) Social Sciences (N = 39) Humanities (N = 85)

Categories N % N % N % N %
Core concepts 16 42.1 36 43.4 21 42.9 35 44.3
Issues 6 15.8 28 33.7 12 24.5 28 35.4
Metacognition 10 26.3 7 8.4 4 8.2 10 12.7
Usefulness of the acquired knowledge 4 10.5 11 13.3 8 16.3 6 7.6
Personal interests 2 5.3 1 1.2 4 8.2 6 7.6

4.2. Quantitative analysis: comparing between subjects and student age

There is an ongoing discussion in the literature regarding the importance of incorporating critical thinking into others subjects, as
opposed to teaching it as a separate subject (Abrami et al., 2008; Ennis, 1989). We therefore analysed whether the topics, activities
and teachers’ reasons varied depending on the subject and age of the student.

4.2.1. Comparing topics by subject

Table 5 shows the topics used to teach critical thinking in each subject. A chi-squared test was used to see whether there were any
significant differences in the distribution of the responses across the different categories for each subject. The results of the test were
2
= 18.387, df = 12, p-value = 0.1044. This shows that the differences in the distribution of the responses are not significant
(p > 0.05), therefore suggesting that the topics chosen by the teachers do not depend on the subject they teach. However, the results
do show that the most common category for each subject (with more than 40 % of responses) is “Core concepts”. The four remaining
categories each have a much lower share. Although there are no significant differences, it is interesting to note that there is a greater
number of responses for metacognition in mathematics than in other subjects.

4.2.2. Comparing student activities by subject

Table 6 shows the kinds of activities done by the students in each subject. A chi-squared test in this case revealed significant
differences in the distribution of the responses ( 2 = 76.388, df = 21, p-value = 3.212e-08, (p > 0.05)). This means that the student
activities reported by the teachers vary depending on the subject that they teach. The most notable difference is in mathematics,
where the majority of responses fall within the category ‘Create’. In natural sciences, however, the most common category is ‘Inquire’.
The distribution in social sciences is much more balanced, while in humanities there is more of an emphasis on activities where the
students have to Analyse.

4.2.3. Comparing how the activities develop critical thinking by subject

Table 7 shows the reasons given to explain how an activity develops critical thinking across different subjects. In this case, the
results of the chi-squared test reveal that there are no statistically significant differences between subjects ( 2 = 24.167, df = 15, p-
value = 0.0623, (p > 0.05)). Although there are some differences in the distribution of the responses, especially when comparing
mathematics with the other subjects, these are not statistically significant.

4.3. Comparing by student age

4.3.1. Comparing topics by student age

Table 8 shows the topics used with students of different ages. Although Metacognition is more prevalent among primary school
students and Usefulness of the acquired knowledge is more frequent among high-school students, the chi-squared test reveals that these
differences are not significant ( 2 = 8.377, df = 4, p-value = 0.0787, (p > 0.05)). This result suggests that the kinds of topics used to
teach critical thinking do not vary based on the age of the students.

Table 6
Comparing student activities by subject.
Mathematics (N = 59) Natural Sciences (N = 93) Social Sciences (N = 60) Humanities (N = 89)

Categories N % N % N % N %
Analyse 14 25 13 14 15 25 34 38.2
Inquire 6 10.7 34 36.6 9 15 7 7.9
Create 26 46.4 11 11.8 17 28.3 15 16.9
Reason 1 1.8 12 12.9 10 16.7 14 15.7
Express 4 7.1 6 6.5 4 6.7 11 12.4
Reflect 4 7.1 11 11.8 4 6.7 5 5.6
Collaborate 1 1.8 5 5.4 1 1.7 3 3.4
Apply 3 5.4 1 1.1 0 0 0 0

9
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

Table 7
Comparing teacher reasons by subject.
Mathematics (N = 38) Natural Sciences (N = 99) Social Sciences (N = 55) Humanities (N = 96)

Categories N % N % N % N %
Understand 7 18.4 29 29.3 15 27.3 28 29.2
Work autonomously 6 15.8 22 22.2 9 16.4 12 12.5
Reason 0 0 1 1 5 9.1 6 6.3
Reflect 7 18.4 16 16.2 9 16.4 21 21.9
Develop cognitive skills 12 31.6 26 26.3 14 25.5 14 14.6
Develop social skills 4 10.5 3 3 2 3.6 9 9.4
Transfer 2 5.3 2 2 1 1.8 6 6.3

Table 8
Comparing topics by student age.
7−13 years 14−18 years

Category N % N %
Metacognition 19 18.3 17 9.8
Usefulness of the acquired knowledge 6 5.8 24 13.8
Core concepts 38 36.5 71 40.8
Issues 34 32.7 49 28.2
Personal interests 6 5.8 8 4.6

4.3.2. Comparing student activities by age

Table 9 shows the kinds of activities done by students of different ages. In this case, the results of the chi-squared test reveal that
the differences are statistically significant ( 2 = 15.938, df = 7, p-value = 0.02569 (p < 0.05)). The most noticeable differences are
the categories Inquire and Reason, which are both more frequent with older students, while Analyse is more common with younger
students.

4.3.3. Comparing how the activities develop critical thinking by student age
Finally, Table 10 shows the reasons given to explain how an activity develops critical thinking for students of different ages. The
results of the chi-squared test reveal that the differences are also significant ( 2 = 18.677, df = 6, p-value < 0.01)). The most
noticeable differences are the categories Understand and Work autonomously, which are more frequent with older students, and
Develop cognitive and social skills, which are more common with younger students.

4.4. Relationships between categories

Descriptive analysis was used to give an overview of how the topics chosen by the teachers relate to the activities and the reasons
they gave to explain why these activities develop their students' critical thinking. This was done by counting the frequency with
which the categories were connected to one another by the respondents. These relationships are shown in the following graphs.
Figs. 1–5 reveal the relationships between the categories developed for this study. The links between the categories (Topics,
Activities, and Reasons) connect the categorized responses from each of the participants. The links are coloured in different shades so
as to make it easier to follow the path between the different categories for each participant. As an example, we can see in Fig. 1 that
the first line connects the Topic Metacognition with the Activity Analyse, and from there to the Reason Understand. From this, we can
therefore see that one of the participants proposes a question about metacognition. In order to solve the question, his students must
analyse. Finally, the participant then explains that his students develop critical thinking in this case because they have developed a
new understanding of a particular phenomenon or concept.

Table 9
Comparing activities by student age.
7−13 years 14−18 years

Categories N % N %
Analyse 35 32.1 49 24.7
Inquire 17 15.6 41 20.7
Reason 8 7.3 37 18.7
Express 13 11.9 17 8.6
Create 16 14.7 33 16.7
Reflect 13 11.9 16 8.1
Collaborate 7 6.4 5 2.5
Apply 3 2.8 1 0.5

10
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

Table 10
Comparing teacher reasons by student age.
7−13 years 14−18 years

Categories N % N %
Understand 20 19.8 50 29.2
Work autonomously 14 13.9 35 20.5
Reason 3 3 9 5.3
Reflect 17 16.8 32 18.7
Develop cognitive skills 30 29.7 33 19.3
Develop social skills 14 13.9 5 2.9
Transfer 3 3 7 4.1

Fig. 1. Relationships between topics, activities and reasons for each participant.

The graphs therefore represent a network between each participant’s responses and can be used to visually represent potential
patterns among the responses, as well as revealing different relationships between the categories. In these graphs, the nodes are the
categories for each response, and the links connect each participant’s response. In a random graph distribution, most nodes have, on
average, the same number of links. Therefore, if we can find some relationships that appear more than others, the distribution is not
random, and suggesting that there are underlying connections (Barabási, 2002).
The responses to the questions were labelled as Topics (i.e. what the question/problem is about), Activities (i.e. what the students
have to do), and Reasons (i.e. how the activity develops critical thinking). Overall, we can see that the number of relationships
between all the categories are low. This means that there are no strong patterns of relationships. In other words, teachers engage their
students in different topics using a diverse range of activities and give different reasons to explain how these activities help their
students develop critical thinking.
Nevertheless, there are more noticeable relationships between the Activities and Reasons, with strong links between Analyse and
Understand (N = 27), as well as between Inquire and Understand (N = 21), Work autonomously (N = 17), Reflect (N = 13) and Develop
cognitive skills (N = 15). Similarly, there is also a strong link between Create and Develop cognitive skills (N = 15).
When breaking the information down by subject (Figs. 2–5), we can also see some differences. In mathematics (Fig. 2), there is a
strong link between Metacognition and Analysis (N = 6), which is less prevalent in social sciences (Fig. 2) and absent from the other
subjects. In natural sciences (Fig. 5), there is a noticeable relationship between Core concepts and Inquire (N = 27), as well as between
Inquire and Understand (N = 11), Work autonomously (N = 10), Reflect (N = 10) and Develop cognitive skills (N = 12). In both
humanities and social sciences there is a strong link between Analyse and Understand (N = 12 and N = 5, respectively), though the
link between Analyse and Develop cognitive skills is stronger in social sciences (N = 10). In the case of humanities, there are also
noticeable links between Analyse/Create and Reflect (N = 8 and N = 6, respectively).

11
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

Fig. 2. Relationships between topics, activities and reasons for each participant that teaches Social Sciences.

Fig. 3. Relationships between topics, activities and reasons for each participant that teaches mathematics.

5. Discussion

Based on these results, we will now look at how the teachers’ responses relate to the literature.
Firstly, we will look at the topics used by the teachers to develop critical thinking. The categories that emerged from the responses
to the question “What are the problems or activities about?” were Metacognition, Usefulness of the acquired knowledge, Core concepts,
Issues, and Personal interests. The most common category from the participants’ responses was Core concepts, which accounted for

12
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

Fig. 4. Relationships between topics, activities and reasons for each participant that teaches subjects related to the humanities.

Fig. 5. Relationships between topics, activities and reasons for each participant that teaches subjects related to natural sciences.

more than 40 % of responses. It is interesting to note that this differs from the recommendations made by researchers in the area
critical thinking, who recommend teaching critical thinking explicitly (Abrami et al., 2008; Marin & Halpern, 2011), even when
integrating it into other subjects. The responses classified within this category do not address critical thinking itself. Instead, they
address core concepts within each subject. The second most common category was Issues. This category is similar to what Abrami
et al. (2014) refer to as authentic or anchored instruction.
Given that Core concepts and Issues account for 70 % of responses, we can conclude that teachers mainly opt for the kind of

13
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

strategies that Ennis (1989) refers to as immersion. These strategies have been promoted by frameworks that are frequently used
within initial teacher training and continuing professional development, such as Understanding by Design (Wiggins & McTighe,
2005). Furthermore, it is also consistent with the approach adopted by the national curriculum in two of the countries involved in this
study, accounting for 41.4 % of the responses that were analysed (see Bellei and Morawietz (2016) for the case of Chile and Cárdenas
(2016) for the case of Mexico). Although critical thinking is not included as a topic on the national curriculum in Chile, it does appear
as a cross-curricular skill in all programs and recommendations are made to integrate it into every subject (MINEDUC, 2013B;
MINEDUC, 2013C; MINEDUC, 2013A). Some studies even suggest that the development of critical thinking depends more on the
school culture and interactions between the relevant actors than on instructional design itself (Huber & Kuncel, 2016; Tsui (2001).
The third most common category is Metacognition, which has been widely acknowledged in the literature as being important for
developing critical thinking (Dwyer et al., 2014; Facione, 1990). Finally, Usefulness of the acquired knowledge may look to link the
topic to the students’ interests, which is also the case with Personal interests. Taking students’ interests into account, as well as their
need for the content to be meaningful, are both necessary conditions for learning and is in line with recent studies on motivation
(Priniski, Hecht, & Harackiewicz, 2018). Furthermore, teachers of different subjects and with students of different ages all follow
similar trends when it comes to the topics they use to integrate critical thinking (Tables 5 & 8).
In terms of student activities, the categories that emerged were Analyse, Inquire, Reason, Express, Create, Reflect, Collaborate and
Apply. The most common category is Analyse (27 % of responses). Analysis is considered a key sub-skill within critical thinking by
many of the definitions (Facione, 1990; Dwyer, 2014; Lai, 2011). The second most common category is Inquire (18.6 %). This
highlights the importance that teachers attach to having their students engage in research and knowledge construction. The re-
lationship between critical thinking and scientific thinking has been addressed by the literature (Forawi, 2016), evidencing strong
links between the two. The main elements of critical thinking coincide with the standards of scientific thinking, such as establishing
relationships between evidence and explanations, conducting research and thinking logically in order to make inferences from
unstructured problems (Forawi, 2016; Kuhn, Iordanou, Pease, & Wirkala, 2008). Inquire is followed by Create (15.8 %), which is the
highest-order skill according to the hierarchy of skills developed by Anderson et al. (2001), based on Bloom’s taxonomy. Although
closely linked to critical thinking and essential for solving problems, inquiry is considered different to critical thinking (Wechsler
et al., 2018). Inquire is, in turn, followed by Reason (14.5 %), which, like analysis, is an important sub-skill in different oper-
ationalizations of critical thinking (Shehab & Nussbaum, 2015; Kuhn, Hemberger, & Khait, 2017). The next category is Reflect (9.3
%), which coincides with definitions that emphasize reflective thinking, i.e. questioning one’s own points of view and assumptions
when querying any evidence that is presented (King & Kitchener, 2004). The penultimate category is Collaborate (3.9 %), which,
although suitable for developing critical thinking (Espey, 2018), does not appear to be so important for teachers. Finally, Apply (1.3
%) is even less frequent among the responses, which is in line with the thinking that application is not usually considered a higher-
order skill (Anderson et al., 2001). The majority of categories to emerge from the teachers’ responses are higher-order thinking skills
that appear in Bloom’s taxonomy (Anderson et al., 2001), which is widely used in the world of education (Lai, 2001).
Significant differences can be found when comparing student activities across different subjects (Table 6). This finding is in line
with the analysis of the topics chosen by teachers, as they focus on Core concepts within each subject, adding that teachers also focus
on practices that are inherent to each discipline. In this sense, natural science teachers refer mainly to activities where the students
are required to Inquire (36 %). Inquiry is key to scientific practice (Bailin, 2002; Forawi, 2016), while the emphasis in mathematics is
on activities where the students must Work autonomously (46 %). This category includes activities where students have to develop
their own response, which is common in problem-solving in mathematics. Furthermore, in social sciences, Create and Analyse are the
most important categories. Within this subject, the category Create refers to suggesting solutions to real-world problems. The pre-
dominant category in humanities is Analyse, which may be explained by the fact that this subject often involves analysing texts and
other cultural objects. From this analysis, we can see that the way in which teachers understand and integrate critical thinking into
the classroom depends heavily on the subjects they teach. There are also significant differences when comparing the activities done
by students of different ages (Table 9). In this sense, we can see that younger students are required to analyse more, while older
students have to inquire and reason. These differences may be due to the students’ level of cognitive development (Kuhn, 1999).
When analysing how the teachers link these activities to the development of critical thinking, the following categories emerged:
Understand, Work autonomously, Reason, Reflect, Develop cognitive skills, Develop social skills and Apply (Table 4). These verbs reveal the
actions that teachers regard as central to the development of critical thinking. The main category, in this case, is Understand (26.8 %
of responses). This category groups together responses that refer to the students developing an improved understanding of a given
topic. This is in line with frameworks such as Understanding by Design (Wiggins & McTighe, 2005), which refer to the concept of
mastery (Belenky & Nokes-Malach, 2013), i.e. mastering a concept to allow for deep thought. The second most common category,
Develop cognitive skills (21.5 %), is closer to the definitions found in psychology (Lai, 2011). The third category, Reflect (19.9 %), is
similar to King & Kichtener’s (2007) position, who refer to reflective thinking as a way of understanding reality by analysing the
epistemology itself. This is followed by Work autonomously (16.8 %), which has often been the focus of the new maker movement
(Holbert & Wilensky, 2018), with an emphasis on students engaging in authentic work by producing artefacts. The next category is
Develop social skills (6.2 %), which groups together more attitudinal aspects (Facione, 1990) and highlights the relationships between
students, as well as their relationship with society. Reimers and Chung (2016) suggest that this element (which they refer to as
Interpersonal skills) is generally underdeveloped on a curricular level and has been poorly implemented across the world. The ca-
tegory Reason (5%) is less commonly cited by teachers as justification for how their activities develop critical thinking. This differs
from those who suggest that there is an equivalence between critical thinking and reasoning (Shehab & Nussbaum, 2015). The
category Apply (3.2 %), where the aim is set on applying knowledge to real life, is referenced less by the teachers than it is in the
literature (Halpern, 1998). When comparing by subject, we can see that there are no significant differences in terms of the reasons

14
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

given by the teachers (Table 7). However, when comparing by age, there are significant differences (Table 10). Again, this may be
due to the students’ level of development (Kuhn, 1999).
Finally, when looking at the links between the teachers’ responses, we can see that they use a range of activities to cover the same
topics and also provide a range of reasons when explaining how these activities develop critical thinking (Figs. 1–5). Nevertheless,
when analysing these links by subject, certain interesting trends do start to emerge. For example, there are stronger links between
Metacognition and Analyse/Understand in mathematics, while in natural sciences there are strong links between Core concepts and
Inquire. Furthermore, Analysis is linked to the concept of Understand in social sciences, while in humanities it is linked to Reflect.
Teachers, therefore, tend to select activities that strengthen everyday practices within their subject (Jones, 2015). In this sense,
metacognition has been closely linked to mathematical thinking (Schoenfeld, 2016), while inquiry is central to science (Forawi,
2016). In humanities, on the other hand, it is the students’ ability to analyse and reflect that plays a leading role (MINEDUC, 2013B),
while in social sciences it is more about trying to understand social reality (MINEDUC, 2013C). This finding is in line with Jones
(2015), who suggests that the way of looking at critical thinking depends heavily on the traditions, conventions and knowledge of
each subject.

6. Conclusions

As highlighted in the introduction, studies on critical thinking often overlook the work done by teachers to develop this skill.
However, studying critical thinking from the teacher’s perspective is key to bridging the gap between theory and practice. Initial
training and continuing professional development both require a knowledge of what teachers are doing in the classroom if they are to
contribute towards improving teaching processes (Farley-Ripple et al., 2018). This study therefore looked to characterize the work
done by Spanish-speaking teachers to integrate this skill into the classroom. This was done by analysing the topics chosen by the
teachers, the activities their students had to complete, and how they linked these activities to the development of critical thinking.
In terms of the topics that were chosen, the majority of teachers preferred topics that relate to the core concepts within their
subject, as well as to real-world issues. The most common activities required the students to analyse, inquire and come up with
solutions. The main reasons given by the teachers to justify their activities were that they encouraged the students to have a better
understanding of reality or subject-specific concepts, to develop cognitive skills, and to reflect on their thinking. When comparing
subjects, we can see that there are differences in the kind of activities the students are expected to complete. This is also the case when
comparing student ages.
Teachers mainly try to develop their students’ critical thinking by integrating it into their own subject. They do so by focusing on
core concepts of each discipline, using topics that help the students understand the world from the perspective of the subject, as well
as familiarizing them with subject-specific practices, making students develop solutions by putting into practice core skills of each
discipline. This perspective is in line with pedagogical frameworks such as Understanding by Design (Wiggins & McTighe, 2005) and
the gold standard for Project-Based Learning (Larmer, Mergendoller, & Boss, 2015). This finding shows that the teachers’ perspective
is in line with suggestions made by several authors, who believe that critical thinking must be situated within a particular subject
(Jones, 2015; Willingham, 2008; Moore, 2015), as it is built on the conventions, methodologies and knowledge that are specific to
each subject. This approach differs from the recommendations made by the approach that advocates for the specific and explicit
teaching of critical thinking (Abrami et al., 2008), whether as a separate subject or integrated into other disciplines.
Overall, the results revealed in this study can provide an approximation of what teachers understand by critical thinking. More
than a cognitive process or a set of rules, the teachers in this study seemed to agree that critical thinking is about applying subject-
specific knowledge and practices to solve real-world problems. This view is different from other constructs, such as creativity,
problem-solving, or reflective thinking, which are proposed as cross-curricular skills (van Laar et al., 2017).
The results of this study go some way to bridging the gap between research and practice as they take into account the teachers’
perspective. This is highly relevant considering how little influenceacademic research has on innovation in teaching (Crook & Gu,
2019), as well as the need for researchers to convey the teachers’ demands (Mills, Lawlor, Watling Neal, Neal, & McAlindon, 2019).
Overall, the results suggest that there is a mismatch between research and practice in education while posing challenges for both
(Farley-Ripple et al., 2018). In this sense, researchers must position their studies within the same context in which learning takes
place. They must also ask questions that are relevant to teachers and be capable of answering these in a way that makes sense to
educators. Teachers, on the other hand, must find the time and space to implement continuing professional development initiatives
that are in line with findings from the research.
There is a series of limitations to this study. Firstly, the results cannot be generalised to all teachers. The sample only included
Spanish-speaking teachers from across Latin America. However, it is not representative of this population as the survey was only sent
to participants on a particular MOOC. The participants therefore had to have access to the internet, a set of certain digital skills and
the time needed to take part in the study, which is not necessarily the case for most teachers across Latin America. Furthermore, the
study was based on a system of self-reporting and, therefore, may not be a true reflection of what the teachers do in their classrooms.
In order to address this issue, future studies should look to include in-depth interviews, as well as classroom observations.
Observations or more detailed interviews could help us understand the relationship between the definitions of critical thinking given
by teachers and the way they integrate it into the classroom. Replicating this study in other parts of the world would also allow us to
see whether the results are applicable to other contexts. As such, a more in-depth study including classroom observations may help
expand on the conclusions provided by the current study.

15
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

CRediT authorship contribution statement

Martín Cáceres: Conceptualization, Methodology, Formal analysis, Writing - original draft. Miguel Nussbaum:
Conceptualization, Validation, Writing - review & editing. Jorge Ortiz: Conceptualization, Visualization, Formal analysis.

Acknowledgement

This paper had the support of FONDECYT / CONICYT1180024

Appendix

Table A1
Demographic information of the participants in the study.
Number of Participants 29,216

Gender Percentage
Male 0.43
Female 0.56

Ages Percentage
18−24 9.6
25−34 43
35−44 24
45−54 7.4
55−64 14.6
> 65 0.4

Country Percentage
Chile 42.39 %
Mexico 24.94 %
Colombia 9.97 %
Spain 5.98 %
Peru 6.23 %
Ecuador 4.23 %
Argentina 3.74 %
Other 2.49 %

Table A2
Classification of subjects in subject areas.
Subject selected by the participant Subject area

Language Arts, Foreign Languages, Philosophy, Ethics, Art, Music and Religion. Humanities
Physics, Chemistry, Biology, Natural Sciences, Technology, ICT Natural Sciences
History, Social Sciences, Economics, Citizenship Social Sciences
Mathematics Mathematics
Guidance, Cross-Curricular Skills Other

References

Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2014). Strategies for teaching students to think critically: A meta-analysis.
Review of Educational Research, 1–40. https://doi.org/10.3102/0034654314551063.
Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and
dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102–1134. https://doi.org/10.3102/0034654308326084.
Alan Bensley, D., & Spero, R. A. (2014). Improving critical thinking skills and metacognitive monitoring through direct infusion. Thinking Skills and Creativity, 12,
55–68. https://doi.org/10.1016/j.tsc.2014.02.001.
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, & Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A
revision of bloom’s taxonomy of educational objectives, abridged edition. White Plains, NY: Longman.
Bailin, S. (2002). Critical thinking and science education. Science and Education, 11(4), 361–375. https://doi.org/10.1023/A:1016042608621.
Barabási, A. L. (2002). Linked. Cambridge. MA: Perseus Publishing.
Barnett, R. (1997). Higher education: A critical business. Milton Keynes: Open University Press.
Bataineh, O. (2009). Perceptions of Jordanian secondary schools teachers towards critical thinking. International Education, 38(2), 56–72.
Behar-Horenstein, L. S., & Niu, L. (2011). Teaching critical thinking skills in higher education: A review of the literature. Journal of College Teaching & Learning (TLC),
8(2), 25–42. https://doi.org/10.19030/tlc.v8i2.3554.
Belenky, D. M., & Nokes-Malach, T. J. (2013). Mastery-approach goals and knowledge transfer: An investigation into the effects of task structure and framing

16
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

instructions. Learning and Individual Differences, 25, 21–34. https://doi.org/10.1016/j.lindif.2013.02.004.

Bellei, & Morawietz (2016). Strong content, weak tools. 21st century competencies in the chilean educational reform. In Reimers, & Chung (Eds.). Teaching and learning
for the twenty-first century: Educational goals, policies, and curricula from six nations. 8 Story Street First Floor, Cambridge, MA 02138: Harvard Education Press.
Bensley, D. A., Rainey, C., Murtagh, M. P., Flinn, J. A., Maschiocchi, C., Bernhardt, P. C., & Kuehne, S. (2016). Closing the assessment loop on critical thinking: The
challenges of multidimensional testing and low test-taking motivation. Thinking Skills and Creativity, 21, 158–168. https://doi.org/10.1016/j.tsc.2016.06.006.
Butler, H. A., Pentoney, C., & Bong, M. P. (2017). Predicting real-world outcomes : Critical thinking ability is a better predictor of life decisions than intelligence.
Thinking Skills and Creativity, 25(June), 38–46. https://doi.org/10.1016/j.tsc.2017.06.005.
Cárdenas (2016). Curriculum reform and twenty-first-century skills in Mexico: Are standards and teacher training materials aligned? In Reimers, & Chung (Eds.).
Teaching and learning for the twenty-first century: Educational goals, policies, and curricula from six nations. 8 Story Street First Floor, Cambridge, MA 02138: Harvard
Education Press.
Cargas, S., Williams, S., & Rosenberg, M. (2017). An approach to teaching critical thinking across disciplines using performance tasks with a common rubric. Thinking
Skills and Creativity, 26(March), 24–37. https://doi.org/10.1016/j.tsc.2017.05.005.
Carvalho, D. P. S. R. P., Azevedo, I. C., Cruz, G. K. P., Mafra, G. A. C., Rego, A. L. C., Vitor, A. F., ... Ferreira Júnior, M. A. (2017). Strategies used for the promotion of
critical thinking in nursing undergraduate education: A systematic review. Nurse Education Today, 57(March), 103–107. https://doi.org/10.1016/j.nedt.2017.07.
010.
Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Thousand Oaks, CA: Sage.
Chen, L. (2017). Understanding critical thinking in Chinese sociocultural contexts: A case study in a Chinese college. Thinking Skills and Creativity, 24, 140–151.
https://doi.org/10.1016/j.tsc.2017.02.015.
Corbin, J., & Strauss, A. (2008). Basis of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). Thousand Oaks, CA: Sage.
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research.
Crook, C., & Gu, X. (2019). How new technology is addressed by researchers in educational studies: Approaches from high‐performing universities in China and the
UK. British Journal of Educational Technology, 50(3), 1173–1188. https://doi.org/10.1111/bjet.12750.
Davies, M., & Barnett, R. (2015). The palgrave handbook of critical thinking in higher education. The palgrave handbook of critical thinking in higher education1–25. https://
doi.org/10.1057/9781137378057.
Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12, 43–52. https://doi.
org/10.1016/j.tsc.2013.12.004.
Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed research. Educational researcher, 18(3), 4–10.
Ennis, R. H. (1985). A logical basis for measuring critical thinking skills. Educational leadership, 43(2), 44–48.
Espey, M. (2018). Enhancing critical thinking using team-based learning. Higher Education Research and Development, 37(1), 15–29. https://doi.org/10.1080/
07294360.2017.1344196.
Facione, P. A. (1990). Critical thinking : A statement of expert consensus for purposes of educational assessment and instruction executive summary “the Delphi reportThe
California Academic Press1–19. https://doi.org/10.1016/j.tsc.2009.07.002 423(c).
Farley-Ripple, E., May, H., Karpyn, A., Tilley, K., & McDonough, K. (2018). Rethinking connections between research and practice in education: A conceptual
framework. Educational Researcher(May), 0013189X1876104. https://doi.org/10.3102/0013189X18761042.
Forawi, S. A. (2016). Standard-based science education and critical thinking. Thinking Skills and Creativity, 20, 52–62. https://doi.org/10.1016/j.tsc.2016.02.005.
Gibbs, G. R. (2007). Analysing qualitative data. London: Sage.
Glaser, B. G., & Strauss, A. L. (2017). Discovery of grounded theory: Strategies for qualitative research. Routledge.
Guest, G., MacQueen, K. M., & Namey, E. E. (2012). Applied thematic analysis. Thousand Oaks, CA: Sage.
Holbert, N., & Wilensky, U. (2018). Designing educational video games to be objects-to-think-with. Journal of the Learning Sciences. https://doi.org/10.1080/
10508406.2018.1487302 0(0), 10508406.2018.1487302.
Howe, E. R. (2004). Canadian and Japanese teachers’ conceptions of critical thinking: A comparative study. Teachers and Teaching: Theory and Practice, 10(5), 505–525.
https://doi.org/10.1080/1354060042000243051.
Huang, G. C., Lindell, D., Jaffe, L. E., & Sullivan, A. M. (2016). A multi-site study of strategies to teach critical thinking: “Why do you think that?”. Medical Education,
50(2), 236–249. https://doi.org/10.1111/medu.12937.
Huber, C. R., & Kuncel, N. R. (2016). Does College teach critical thinking? A meta-analysis. Review of Educational Research, 86(2), 431–468. https://doi.org/10.3102/
0034654315605917.
Jones, A. (2015). A disciplined approach to critical thinking. In M. Davies, & R. Barnett (Eds.). The palgrave handbook of critical thinking in higher education. New York:
Palgrave Macmillan.
King, P. M., & Kitchener, K. S. (2004). Reflective judgment: Theory and research on the development of epistemic assumptions through adulthood. Educational
psychologist, 39(1), 5–18.
Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher, 28(2), 16. https://doi.org/10.2307/1177186.
Kuhn, D. (2018). Building our best future: Thinking critically about ourselves and our world. Wessex Press.
Kuhn, D., Hemberger, L., & Khait, V. (2017). Argue with me: Argument as a path to developing students’ thinking and writing. Routledge.
Kuhn, D., Iordanou, K., Pease, M., & Wirkala, C. (2008). Beyond control of variables: What needs to develop to achieve skilled scientific thinking? Cognitive devel-
opment, 23(4), 435–451.
Lai, E. R. (2011). Critical thinking: A literature review. Critical Thinking, (June), 1–49. Retrieved from http://images.pearsonassessments.com/images/tmrs/
CriticalThinkingReviewFINAL.pdf.
Larmer, J., Mergendoller, J. R., & Boss, S. (2015). Gold standard PBL: Essential project design elements. PBL blog.
Larsson, K. (2017). Understanding and teaching critical thinking—A new approach. International Journal of Educational Research, 84(December 2016), 32–42. https://
doi.org/10.1016/j.ijer.2017.05.004.
Lefever, S., Dal, M., & Matthíasdóttir, Á. (2007). Online data collection in academic research: Advantages and limitations. British Journal of Educational Technology,
38(4), 574–582. https://doi.org/10.1111/j.1467-8535.2006.00638.x.
Marin, L. M., & Halpern, D. F. (2011). Pedagogy for developing critical thinking in adolescents: Explicit instruction produces greatest gains. Thinking Skills and
Creativity, 6(1), 1–13. https://doi.org/10.1016/j.tsc.2010.08.002.
McHugh, M. L. (2013). The chi-square test of independence. Biochemia medica: Biochemia medica, 23(2), 143–149.
McIntyre, D. (2005). Bridging the gap between research and practice. Cambridge Journal of Education, 35(3), 357–382. https://doi.org/10.1080/03057640500319065.
McLaughlin, A. C., & McGill, A. E. (2017). Explicitly teaching critical thinking skills in a history course. Science and Education, 26(1–2), 93–105. https://doi.org/10.
1007/s11191-017-9878-2.
Miles, M. B., & Huberman, A. M. (1994). Quantitative data analysis (2nd ed.). Thousand Oaks, CA: Sage.
Mills, K. J., Lawlor, J. A., Watling Neal, J., Neal, Z. P., & McAlindon, K. (2019). What is research? Educators’ conceptions and alignment with United States federal
policies. Evidence & Policy: A Journal of Research, Debate and Practice.. https://doi.org/10.1332/174426419X15468576296175.
MINEDUC (2013A). Matemática, bases curriculares [pdf file]. Retrieved fromhttps://www.curriculumenlineamineduc.cl/605/articles-21321_programa.pdf.
MINEDUC (2013B). Lenguaje y comunicación, bases curriculares [pdf file]. Retrieved fromhttps://www.curriculumenlineamineduc.cl/605/articles-21322_bases.pdf.
MINEDUC (2013C). Historia y ciencias sociales, bases curriculares [pdf file]. Retrieved fromhttps://www.curriculumenlineamineduc.cl/605/articles-21320_programa.
pdf.
Moore, T. (2013). Critical thinking: Seven definitions in search of a concept. Studies in Higher Education, 38(4), 506–522. https://doi.org/10.1080/03075079.2011.
586995.
Paul, R., & Elder, L. (2009). The miniature guide to critical thinking-concepts and tools (thinker’s guide). Dillon Beach, CA: Foundation for critical thinking.
Pellegrino, J. W., & Hilton, M. L. (2012). Education for life and work. Washington, D.C: National Academies Presshttps://doi.org/10.17226/13398.

17
M. Cáceres, et al. Thinking Skills and Creativity 37 (2020) 100674

Priniski, S. J., Hecht, C. A., & Harackiewicz, J. M. (2018). Making learning personally meaningful: A new framework for relevance research. The Journal of Experimental
Education, 86(1), 11–29.
Reimers, F. M., & Chung, C. K. (2016). Teaching and learning for the twenty-first century: Educational goals, policies, and curricula from six nations. 8 Story Street First Floor,
Cambridge, MA 02138: Harvard Education Press.
Reja, U., Manfreda, K. L., Hlebec, V., & Vehovar, V. (2003). Open-ended vs. Close-ended questions in web questionnaires. Developments in applied statistics, 19(1) 160-
117.
Reynolds, S. W. (2016). Determining and exploring teachers’ perceptions on the barriers to teaching critical thinking in the classroom: A survey study (Doctoral dissertation).
Saldaña, J. (2015). The coding manual for qualitative researchers. Los Angeles: SAGE Publications.
Schoenfeld, A. H. (2016). Learning to think mathematically: Problem solving, metacognition, and sense making in mathematics (reprint). Journal of Education, 196(2),
1–38.
Shehab, H. M., & Nussbaum, E. M. (2015). Cognitive load of critical thinking strategies. Learning and Instruction, 35, 51–61. https://doi.org/10.1016/j.learninstruc.
2014.09.004.
Stupple, E. J. N., Maratos, F. A., Elander, J., Hunt, T. E., Cheung, K. Y. F., & Aubeeluck, A. V. (2017). Development of the critical thinking toolkit (CriTT): A measure of
student attitudes and beliefs about critical thinking. Thinking Skills and Creativity, 23, 91–100. https://doi.org/10.1016/j.tsc.2016.11.007.
Swartz, R. J., & Parks, S. (1994). Infusing the teaching of critical and creative thinking into content instruction: A lesson design handbook for the elementary grades. PO Box
448, Pacific Grove, CA 93950-0448: Critical Thinking Press and Software.
Tsui, L. (2001). Faculty attitudes and the development of students’ critical thinking. The Journal of General Education, 50(1), 1–28. https://doi.org/10.1353/jge.2001.
0008.
Twining, P., Rachelle, H., Nussbaum, M., & Tsai, C.-C. (2016). Some guidance on conducting and reporting qualitative studies art. Computers & Education, 106, 1–9.
https://doi.org/10.1016/j.compedu.2016.12.002.
Vanderlinde, R., & van Braak, J. (2010). The gap between educational research and practice: Views of teachers, school leaders, intermediaries and researchers. British
Educational Research Journal, 36(2), 299–316. https://doi.org/10.1080/01411920902919257.
Wechsler, S. M., Saiz, C., Rivas, S. F., Vendramini, C. M. M., Almeida, L. S., Mundim, M. C., & Franco, A. (2018). Creative and critical thinking: Independent or
overlapping components? Thinking Skills and Creativity, 27(November 2017), 114–122. https://doi.org/10.1016/j.tsc.2017.12.003.
Wiggins, G. P., & McTighe, J. (2005). Understanding by design (expanded 2nd ed). Association for Supervision and Curriculum Development.
Willingham, D. T. (2008). Critical thinking: Why is it so hard to teach? Arts Education Policy Review, 109(4), 21–32. https://doi.org/10.3200/AEPR.109.4.21-32.

18