What The Critical Thinking Data Tells Us: K-12 Education

Advancements in the Assessment of K-12 Students’ Critical Thinking

The promise of the American K-12 educational system is to develop strong numerically and scientifically literate critical thinkers who are ready for college and careers in today’s Information Age (U.S. Department of Education, 2019). This pledge is grounded in the understanding that children and adolescents, just like adults, must reason through a perpetual tidal wave of information washing over them in school and beyond through television, social media, and the Internet. Achievement of this ambitious educational goal requires the cultivation of critical literacies including higher order thinking, scientific reasoning, information literacy, and numeracy.

Evidence that critical thinking skills can be taught

In this era of Common Core State Standards, where depth of understanding, critical thinking, and real-world problem-solving receive heightened attention as learning outcomes, educators are transitioning their practice to emphasize problem-based learning to provide creative learning opportunities for all students.[1] The scholarly community similarly must critically evaluate the impacts of an enriched thinking and problem-solving focused pedagogical approach to elementary and secondary education. Rigorous educational research into the characteristics, variations, and developmental trajectories of children’s and adolescents’ critical thinking is of upmost importance to best inform our children’s teachers as they strive to nurture critical thinking in tomorrow’s citizens and leaders.

Until recently, a significant challenge has been the limited availability of valid and reliable standardized measurement tools to directly assess K-12 students’ critical thinking. However, with the introduction of the EDUCATE INSIGHT Reasoning Skills and Mindset Series, this assessment instrument gap is now closed. When teachers need an assessment of critical thinking skills or an assessment of the mindset attributes needed in a strong critical thinker, these age-calibrated assessment instruments provide evidence of individual and group strengths and areas for focused instruction.

Here is an example of collecting evidence of the effectiveness of training critical thinking in elementary students. Enrichment programs focused on improving schools take a data-based approach to evaluating their program’s value to the students. In this pilot project, students in grades 3 through 5 were assessed with EDUCATE INSIGHT to obtain evidence of the effectiveness of a curriculum designed to improve critical thinking. Participating classrooms were located in several cities across the US. The study period was one academic year of instruction.

Variable Name N Mean SE St Dev Minimum Maximum
OVERALL Pretest 504 72.8 0.2 5.1 60 90
OVERALL Posttest 345 75.0 0.3 6.0 63 94

Table 1: OVERALL score distribution – Grades 3-5 Pretest Posttest Sample

Figures 1 and 2: Distribution of EDUCATE INSIGHT OVERALL scores captured in Fall (N= 504) at left and Spring (N=345) at right.

The improvement in scores for this group of students can be seen at a glance. Pretest scores averaged 72.8 (40th percentile), while posttest scores averaged 75.0 (50th percentile) raising the group performance to a level representative of the national population of students in grades 3-5.

Another look at evidence of improvement can be seen in the table below. Here the same 168 students were assessed with EDUCATE INSIGHT at the beginning and end of Grade 4.

Variable Name N Mean SE St Dev Minimum Maximum
OVERALL Pretest 168 71.8 0.5 5.8 60 89
OVERALL Posttest 168 74.1 0.4 5.7 63 94

Table 2: OVERALL score distribution – Grade 4 – Matched Pairs Sample

This table reports the distribution of OVERALL scores in of 168 students who were assessed at the beginning and the end of Grade 4. (N=168). The results show that scores for OVERALL critical thinking skills increased by 2.30 points on average in the matched pairs sample (educationally significant gain). The observed average gain from pretest to posttest was statistically significant [T=4.43, p<.001]. The national comparison percentile score for this cohort increased from 32nd to the 41st percentile.

Can we capture evidence that middle school students are improving their critical thinking skills?

The figure below shows the Pretest and Posttest scores for a sample of 77 middle school students assessed with EDUCATE INSIGHT. Mean scores at Pretest were 80.6 (58th percentile), typical of a magnet school with selective admissions. Even though students entered this middle school year far ahead of their peers in developing their critical thinking skills, they continued to improve then over the course of their year of study. Posttest scores for the OVERALL in this same group rose to a mean of 82.4 (64th percentile). Most appear to be benefiting from an educational enrichment program focused on critical thinking.

Figures 3 and 4: Distribution of EDUCATE INSIGHT (Grade 6-8) OVERALL scores captured in Fall (left) and Spring (right), (N=77).

Other evidence gathered from schools across the US suggests that some schools are struggling when it comes to training critical thinking skills. Individual gains (or a failure to gain) can be seen here in this graphic

A graph of different colored bars Description automatically generated

Figures 5: Distribution of gain scores for EDUCATE INSIGHT Grades 6-8 – OVERALL (N=400)

Gain scores are calculated as the change in score from Pretest to Posttest. Students who made educationally significant gains have scores of 2 or above (blue bars). Many of the children in this educational program are demonstrating extraordinary gains in their reasoning skills. It is common for students who have not been previously introduced to critical thinking to make strong gains over several months of an educational program focused on reasoning skills.

Not all attempts to train critical thinking are successful, however. Being less able to engage problems that challenge us to focus, analyze, evaluate, and explain “why?” makes our lives significantly harder. EDUCATE INSIGHT can help teachers document difficulties in reasoning and problem solving.

Middle School Critical Thinking Skills Scores and Math Score of State Level Assessments

Critical thinking skill in early middle school (6th grade) has been found to be highly predictive of student mathematics achievement in later grades.[2] In a three-year longitudinal study of students in California, scores on the EDUCATE INSIGHT Grades 6-8 – Skills were strongly and significantly (p<.001) correlated with students’ math scores on the California Standardized Testing and Reporting (STAR) assessment. In the first year of testing (6th grade), the correlation between students EDUCATE INSIGHT Grades 6-8 – Skills scores and the California Standardized Test – Math (CST Math) overall score was .635 (subscale correlations = .448-.601). This relationship tells us that approximately 40% of the variance in these 6th grade students’ mathematics achievement as measured by State mandated standardized test could be predicted by their EDUCATE INSIGHT Overall score – a very strongly predictive finding. These strong and significant correlations with the CST persisted for these students in grades 7 and 8, as did an observable relationship between these scores and whether teachers’ recommended students to advance to Algebra as 8th graders. More importantly, this demonstrated relationship between critical thinking and mathematics achievement in early middle school means that an assessment of critical thinking skills can identify students whose critical thinking skills need to be improved to support their future learning in mathematics.

How early is too early to assess and train critical thinking?

A group of toys and puzzle pieces Description automatically generated In our view, it is never too early to empower children with better thinking skills.

The conventional approach has been to indirectly approximate reasoning ability using standardized measures of student achievement typically administered at the state level. Those assessments of students’ achievement are not routinely administered before the 3rd grade. International exams such as the NAEP and TIMMS are not administered younger than 4th grade.

However, even the youngest child can be taught to question their environment, and it is common for children to ask questions that test the new knowledge that they are developing. And, with careful attention to their ability to participate in an assessment process, it is possible to capture valid and reliable data about their thinking skills.

What’s known about critical thinking in children in Grades K-2?

Here is some example data from the reasoning skills portion of EDUCATE INSIGHT Grades K-2, administered to a sample of 100 male and female first and second graders. These students were attending economically and ethnically diverse public schools in the southern region of California’s San Francisco Bay Area. Teachers read questions aloud, as students followed along. Each question shows a picture.

Figure 6: Educate Insight K-2 OVERALL Skills scores in 100 students in Grades 1 and 2.

A distribution of scores for the OVERALL critical thinking scale are presented in Figure 6. The students’ scores array in a relatively normal distribution, the average score falling within the same value range as is observed with older students. While many of these scores indicate low levels of skills, others show children at the K-2 level manifesting high levels of critical thinking skills. Followed over time, studies such as this can help us learn how critical thinking is driving learning in other skill and knowledge domains. EDUCATE INSIGHT Grades K-2 asks students to Categorize, Explain, Analyze, Infer and Evaluate. It is used to capture evidence of their emerging strength in critical thinking.

The EDUCATE INSIGHT series equips educators and researchers with tools to directly measure critical thinking among students as young as kindergarten. Here is some example data collected with EDUCATE INSIGHT Grades K-2 – Mindset. We can see considerable variance in these children’s mindset for engaging and resolving problems. Learning orientation is stronger in this group. Outlier scores (orange and red bars) typically validate other formal and informal assessments.

Figures 7 and 8: Distribution of scores (n=111) for two of 4 reported mindset metrics – Educate Insight K2

Mindset matters

In addition to training reasoning skills, education for critical thinking demands nurturing a critical thinking mindset in children of all ages. The mindset portion of the EDUCATE INSIGHT assessments addresses this vital dimension — a person’s consistent internal motivation to think critically when faced with problems to solve, ideas to evaluate, or decisions to make. [3]

A graph of different colored bars Description automatically generated

Figure 9. Data collected with EDUCATE INSIGHT Mindset Metrics Grades 3 to 5.

These attitudes, inclinations, or habits of mind are dimensions of one’s personality and motivational style which relate to how likely a person is to approach decision-making contexts or problem-solving situations by engaging their reasoning skills. Here is an example of a group score report for a group of 168 students in the 4th grade (Figure 9).

Scores in the upper half of the range (above 30) indicate evidence for the disposition being present and are desired. For example, most children in this sample had a positive Learning Orientation and demonstrated Cognitive Integrity. Creative Problem Solving and Mental Focus scores ranged across a broader continuum. Creative Problem Solving, as measured here, is the habit or tendency of approaching problem solving with innovative or original ideas and solutions; feeling imaginative, ingenious, and able to solve problems that might be seen as difficult by a child of this age; engaging in activities such as puzzles, games of strategy; and striving to understand the underlying function of objects. Creative problem solving is highly desirable in today’s workforce. Many children in this very young group demonstrate this mindset attribute. If it diminishes over the course of their education, one will naturally wish to investigate why.

Teachers are models of critical thinking

At the Young Women’s Leadership School of the Bronx, computer Science teacher Marisa Shuman does this by asking her students to analyze, interpret and evaluate the information they find when conducting on-line searches, sometimes using ChatGPT. It is not their learning of content that is of concern, it is their use of critical thinking to determine whether the content they find is valuable. In one class, she asked whether the use of wearable technology is “effective or ineffective”?[4]

In 2024, it should be clear that children benefit from teachers who are strong critical thinkers,[5],[6] and that the training of critical thinking skills and mindset can be done effectively in any subject matter context.[7] If this claim seems wrong-headed, perhaps spend some time asking how students engaged in competitive sports would fare if coaches failed to explain strategies, evaluate tactics, draw inferences regarding the potential weakness of the opposition, etc. and communicate these ideas and this way of competing to their athletes. However, any previous debate should cease as to whether growth in critical thinking is an automatic result of any effective K-12 educational program. The data is clear in this regard. It is not.

A graph of a normal distribution Description automatically generated A graph of a magnetic thinking Description automatically generated with medium confidenceFigures 9 and 10: Comparing gains over one year in two elementary school cohorts.

These two comparative graphics show two different cohorts of elementary school students, one a public magnet school and the other a more comprehensive public elementary school. All students were assessed with EDUCATE INSIGHT Grades 3-5 in the Fall (blue curves) and again in the Spring (red curves). The curves show the distribution of scores for EDUCATE INSIGHT OVERALL thinking skills. In the first cohort, the curves almost entirely overlap showing no change in critical thinking skills over the year of instruction for this cohort. In Figure 10, the OVERALL scores grew decidedly stronger (scores have moved higher). Here the children were benefiting from their teacher helping them to identify and practice their thinking and reasoning skills.

The Importance of Numeracy

Research shows that Numeracy (sometimes called Mathematical Reasoning or Quantitative Literacy), is as strong a predictor of students’ school success as reading ability[8] and that the development of early mathematical skills and literacy skills is highly interrelated.[9] Wiest offers a stronger perspective, suggesting that quantitative literacy is, in essence, a gatekeeper for effective functioning today.[10] They argue that “many adults lack the “numeracy” needed to function in a maximally effective manner in their vocational, civic, and personal lives. We agree and, unfortunately, an examination of the data that we and our colleagues and clients have gathered over the past two decades shows that quantitative literacy (Numeracy) is weak in many of our educational populations. We believe schools need to foster skills in quantitative literacy (QL), an inclination and ability to make reasoned decisions using general world knowledge and fundamental mathematics in everyday circumstances. Doing this will result in more quantitatively literate students, a goal that coincides with efforts toward greater social justice.

Quantitative Literacy in the Information Age:

The development of quantitative literacy in all children is a matter of social justice in the Information Age. All EDUCATE INSIGHT reasoning skills assessments for grades 3 and higher include a reported score for Numeracy, critical thinking in a quantitative context.[11] Much more than simply adding a column of numbers or solving for x, numeracy is the ability to set up the problem in the first place, that is determining which mathematical operations to apply, and in what order, so that one might reason correctly about the quantitative information available and resolve the question at hand. Numeracy is analysis, inference, interpretation, explanation, evaluation, as well as reflection on one’s own reasoning process (metacognition and self-regulation) when numbers and other quantitative considerations matter.

We have already touched on the relationship between critical thinking and mathematics in our Middle School discussion above. This relationship becomes stronger as students advance to High School and select areas of study at Universities.[12] There is great synergy between a focus on numeracy as critical thinking applied to the context of mathematics, probability and numerical data analysis and the overarching educational goal of building students’ critical thinking in both the K-12 and postsecondary levels. Not surprisingly, the national standards reform movement in mathematics, as reflected by the Common Core State Standards Initiative (CCSI) draws heavily on the National Council of Teachers of Mathematics (NCTM) vision and recommendations and prioritizes numeracy.[13] They emphasize number sense and problem solving, abstract and quantitative reasoning, argument construction and critique, structural analysis and strategic application of tools to solve math problems, and modeling with mathematics, as vital practice-based learning outcomes at all grade levels.[14] In alignment with mathematics education, the centrality of quantitative reasoning and literacy manifesting in evidence-based explanations for science practice is emphasized in major science education documents including, the Next Generation Science Standards (NGSS).[15]

The Numeracy scale of the EDUCATE INSIGHT Grades 9-12 Skills assessment provides a focused measure that is conceptually aligned with the reasoning and problem-solving skills endorsed by NCTM, NGSS, and reflected in the language of the Common Core Standards.

This example dataset shows the assessment of a classroom of 11th Graders in a private high school (N=44). Figure 11 shows their OVERALL critical thinking skills scores, with the average at 83.9. The scores are normally distributed across the expected range for this assessment, with most students scoring in the Moderate range. Although this is a very small sample, their demonstrated strength in critical thinking skills is representative of the national comparison sample for high school students (51st percentile in the year that they were assessed: 2022).

Figure 12 shows the Numeracy subscale scores for this same group of 44 students. Although there are some students with superior (blue bars) or strong (green bars) scores for numeracy, in general these scores are weaker than the OVERALL scores. Numeracy scores are useful for determining the proportion of students who can apply their critical thinking skills to problems that involve numbers, likelihoods, rates, or other numerical considerations.

Figures 11 and 12: Private Highschool Cohort (N=44)

Overall and Numeracy scores on EDUCATE INSIGHT – Grades 9-12

The importance of numeracy skills are being widely discussed by employers and consideration of the level of numeracy skills needed for particular jobs is only growing over time. The degree of concern for improving numeracy in the workforce is well summarized in “The Essential Skill of Numeracy,” a web post by Canadian talent search firm BarrettRose & Lee Inc. (Dec 2020). “With greater numbers of workers engaging in more sophisticated tasks, numeracy is recognized as an essential employability skill. Also, it has been acknowledged as a potential employment equity issue, as adults with poor numeracy skills are more likely to have relatively low work positions with fewer promotion prospects and lower wages.”

Improvement can be achieved through the training and practice of critical thinking skills within mathematical education offerings. Assessing Numeracy skills in children throughout K-12 education allows the identification of students with strength in Numeracy and those who require more focused opportunities to acquire these skills.

Now that the assessment instruments are available for the entire K-12 educational spectrum, research on the development of critical thinking and numeracy skills and mindset attributes can be studied in greater detail. We anticipate that, as happened at the college and graduate professional school levels, equipped with these assessments, educators will significantly advance our understandings of effective pedagogies and the interrelationships in children of thinking skills, mindset attributes, and educational achievement.

There are resources for teachers and parents to better integrate critical thinking in their children’s daily lives. One of these is the Reboot Foundation’s Resources page: https://reboot-foundation.org/resources_/ . They provide practical tools for parents, teachers, employers, and others interested in cultivating a capacity for critical thinking, and we make them available for free to the public.

This report is one of a series of white papers prepared for researchers and trainers of critical thinking skills and mindset. Carol Ann G. Gittens, Peter Facione, & Noreen Facione

Additional Information for those interested in the development and scope of the EDUCATE INSIGHT Series.

The EDUCATE INSIGHT Series was first introduced under titles that referenced their K-12 companion status with the original post-secondary California Critical Thinking Skills Test (CCTST), and the CCTDI-related California Measure of Mental Motivation (CM3). The first upper elementary skills assessment, which focused on grades 3-5, was introduced in 2005, followed in 2008 by the middle school skills version, and the secondary level skills assessment in 2014. These assessments were subsequently consolidated under the EDUCATE INSIGHT Series brand in 2017 when the assessment series was augmented with the introduction of the K-2 level assessments, creating an reasoning skills and mindset series complete for all levels; early elementary, upper elementary, middle school, and secondary. Like the CCTST and CCTDI, the EDUCATE INSIGHT series of K-12 instruments is based on the Delphi study sponsored by the American Philosophical Association (APA) that produced an expert consensus definition of critical thinking as the process of purposeful, self-regulatory judgment.[16] The Delphi panelists described the characteristics of a strong thinking and included reference to both skills and dispositional mindset characteristics.

Educate Insight

Thinking Mindset

Grades

K-2

Grades

3-5

Grades

6-8

Grades

9-12

Mental Focus
Learning Orientation
Creative Problem Solving
Cognitive Integrity
Scholarly Rigor

The EDUCATE INSIGHT (EI) reasoning skills questions increase in difficulty and complexity at each higher educational level. At all levels, the questions use developmentally appropriate and relevant everyday scenarios that are common and familiar to the target test takers. Upper elementary, middle school, and secondary levels of the EI instruments have Flesch-Kincaid reading levels well below the grade level of the intended test taker. For children in K-2, it is recommended that the instrument scenarios be read aloud to test takers by a trained proctor.

Educate Insight Reasoning Skills Grades K-2 Grades 3-5 Grades 6-8 Grades 9-12
OVERALL
Categorization
Explanation
Analysis
Inference
Evaluation
Inductive Reasoning
Deductive Reasoning
Numeracy
Interpretation

Reasoning skills questions require test takers to analyze the scenario information, draw appropriate inferences, and evaluate the claims and arguments presented. The response frame on all the skills tests is multiple choice format. Any specialized information needed to select the best option from among those offered is provided in the question itself. Together, the items provide a measure of overall critical thinking and a measure of five or more skill domains (see Table 4). The K-2 level includes the scale Categorization, considered a fundamental developmental thinking skill. Categorization is the cognitive process by which objects are identified, recognized, and grouped according to a perceived common characteristic. Categorization enables understanding language, predicting the behavior of, inferring the characteristics of, and deciding how to behave toward or how to interact with objects grouped into the same category.

  1. Cash, J. (2013). Transitioning teaching and learning to embrace common core. Leadership, 43(2), 22-25. (Leadership is a publication of the Association of California School Administrators available to members. A contact point for the journal can be found at https://www.acsa.org/publications.)
  2. Gittens, C. A. (2015). Assessing numeracy in the upper elementary and middle school years. Numeracy 8(1), Article 3. DOI: http://dx.doi.org/10.5038/`936-4660.8.1.3. Available at: http://scholarcommons.usf.edu/numeracy/vol8/iss1/art3
  3. Facione, P., Facione, N., & Giancarlo (Gittens), C. (2000). The disposition toward critical thinking: Its character, measurement and relationship to critical thinking skill. Informal Logic, 20, 61–84. Giancarlo (Gittens), C. A., Blohm, S. W., & Urdan, T. (2004). Assessing secondary students’ disposition toward critical thinking: Development of the California Measure of Mental Motivation. Educational and Psychological Measurement, 64, 347–364. http://dx.doi.org/10.1177/0013164403258464.
  4. At This School, Computer Science Class Now Includes Critiquing Chatbots. Singer, N. New York Times reporter, Feb 7, 2023.
  5. Targeting critical thinking within teacher education: The potential impact on society, Williams, R 2005 The Teacher Educator 40 (3) 163-87.
  6. Critical Thinking Skills Not Emphasized By Most Middle School Teachers, Bouygues HL, Forbes Aug 17, 2022.
  7. Enhancing Critical Thinking with Aesthetic, Critical, and Creative Inquiry, Lampert N. 2015 2006 59 (%) 46-50.
  8. Perry, M. L. (2013). Strengthening early math: A high level strategy for meeting the Common Core challenge. San Jose, CA: Silicon Valley Education Foundation. Retrieved from https://svefoundation.org/wp-content/uploads/2016/10/Early-Math-SVEF-paper.pdf
  9. Purpura, D. J., Logan, J. A. R., Hassinger-Das, B. & Napoli, A. R. (2017). Why do early mathematics skills predict later reading? The role of mathematical language. Developmental Psychology, 53, 1633-1642. DOI: http://dx.doi.org.libproxy.scu.edu/10.1037/dev0000375.
  10. Wiest, L. R., H. J. Higgins, & J. H. Frost. (2007). Quantitative literacy for social justice. Equity & Excellence in Education, 40(1), 47–55.
  11. Gittens, 2015, op. cit.
  12. More on this relationship in our report – 2024 What the Data Tell Us – Higher Education
  13. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: National Council of Teachers of Mathematics. National Council of the Teachers of Mathematics. (2006). Curriculum focal points for prekindergarten through grade 8: Mathematics a quest for coherence. Reston, VA: National Council of the teachers of Mathematics. National Council of the Teachers of Mathematics. (2009). Focus in high school mathematics: Reasoning and sense making. Reston, VA: National Council of the Teachers of Mathematics.
  14. Burns, M. (2012). Go figure: Math and the Common Core. Educational Leadership, 70(4), 42–46. Common Core State Standards Initiative. (2019). Common Core Standards: Standards in your state. Retrieved from http://www.corestandards.org/standards-in-your-state/
  15. NGSS Lead States. (2019). Next Generation Science Standards: For States, By States. Retrieved from https://www.nextgenscience.org/
  16. American Philosophical Association. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. The Delphi Report Executive Summary: Research findings and recommendations prepared for the committee on pre-college philosophy. ERIC Document No. ED 315-423.