What the Critical Thinking Data Tells Us: Professional Program Students

Professional education programs tie together teaching of the content of a profession with training in the skills needed by a person practicing the profession. The curriculum is typically packed tightly with technical content and course work that builds on previous offerings, making it difficult for a student to remedy insufficient performance early in the program.

As a result, the use of critical thinking assessments follows a different pattern in professional schools. Business programs preparing professionals in Management, Marketing, Finance, and Executive Training Programs typically assess critical thinking skills and mindset as a component of program admission. Health Science Programs do the same, often using a cut score that they have observed is indicative of the ability to successfully complete the program and obtain a license to practice. The stakes are particularly high in the health sciences because a lost seat usually means one less health professional who can join the ranks of our health management teams.

How do professional education students compare with the greater population of college students?

Professional students demonstrate a wide range of critical thinking ability, but on average they demonstrate stronger critical thinking skills than the college population at large. This is true for both the undergraduate and graduate program students. An admissions sample for health science applicants who were assessed with the Health Science Reasoning Test (HSRT) might look like this (Figure 1):

Figure 1: Health Science Reasoning Test (HSRT) OVERALL SCORES for a PharmD applicant pool.

Figure 1 shows scores for HSRT OVERALL, a single score indicating overall strength in critical thinking skills. Blue bars indicate scores in the Superior range, and the green bars indicate scores in the Strong range. These are both highly desirable levels of performance, documented as strong indicators of the ability to successfully complete a rigorous curriculum and obtain licensure.[1],[2],[3],[4],[5],[6],[7],[8] In this applicant sample the median score is 79 in the Moderate (yellow) range; and almost half of these applicants demonstrated strong or superior critical thinking skills scores.

Years of assessment data collection specific to a profession allows us to provide an external criterion for evaluating the strength of applicants within a particular professional education program. For example, Health Science programs who use the HSRT to assess their applicants or admitted students can benchmark these scores against current population samples specific program types (Nursing, Medicine, Dentistry, Pharmacy. Physical Therapy, Dental Hygiene, Physician Assistant, Occupational Therapy, etc.). Applicants who are admitted and have areas of relative weakness (lower OVERALL or cognitive skills scores) can be provided with focused training to improve their clinical reasoning capability early in the program.

The Graduates of Selective Professional Programs Demonstrate Stronger Critical Thinking in their Graduates

Some of the strongest assessment samples are collected by clients whose programs are in high demand, and where the profession is highly valued. For example, this graphic (Figure 2) was created as a part of an AACSB accreditation self-study report at this type of professional education program. The graphic is a demonstration of the strength of the graduating Business school class. The grey bars represent the score distribution for the Business Critical Thinking Skills Test (BCTST) total or OVERALL score.

A graph of a normal distribution Description automatically generated

Figure 2: Critical thinking skills (BCTST TOTAL) scores at selective undergraduate Business program

The 710 undergraduate Business students’ scores mean is 26.09. The graphic is displaying where that mean score lies in relationship to mean score of undergraduate Business students in the US population (red line), the mean score of graduate students in Business (Green line) and the mean score for undergraduates in top tier Business schools in the US (Blue line). While this graduating class mean score did not quite reach the highest comparison group (Top Tier Schools), their performance far exceeded most others, demonstrating the achievement of this school’s critical thinking learning outcomes. When benchmarked using the AACSB comparison percentiles this undergraduate sample mean was at the 93rd percentile.

Curriculum evaluation – simulation training technology

Programs in military science and the health sciences have been leaders in the use of simulation technologies to expand the experience of trainees in the challenges of their future professions. Educational researchers in these disciplines have used MDCTI and HSRT to examine whether decision-making is being trained when simulators are used to educate students. [9],[10],[11] Other examples of curriculum evaluation research carried our by independent researchers using critical thinking assessments are compiled in the report “Research – Critical Thinking and Leadership” available as a downloadable free resource from www.InsightAssessment.com.

Critical thinking is trained by focusing on individual cognitive skills

Although we have always known that the different cognitive maneuvers used to make a reasoned judgement do not function discretely, but instead are iterative processes (we interpret evaluations, analyze inferences, explain deductive conclusions, etc.) we now understand that these cognitive processes are distinct within the individual. Some individuals are more skilled in inductive reasoning (they have trained their ability to think well when the situation is less certain, more probabilistic). Others lack this ability. We have also learned that individuals who have strong inference skills (those who will come to a well-grounded inference based on the information available) and strong explanation skills (those who can provide a contextually based explanation for why an analysis or a decision is accurate) are more likely pursue preparation as a health science professional, a first responder, or a professional in a STEM discipline as compared to those in the college population at large.

Here in Figure 2 are the scores of our PharmD applicant pool for Induction, Inference and Explanation.

A close-up of several graphs Description automatically generated

Figure 3: Relative strength in particular reasoning skills in a sample of PharmD applicants.

Although there are individuals in the applicant pool who do not manifest these skills (red bars), the preferred cut scores for these programs can usually be set to admit a group of first year health science students with these skills.

Some reasoning skills are more demanding than others.

Some other cognitive maneuvers that are key to having overall strong critical thinking are more difficult to achieve, because they are more complex. Interpretation, as measured by the HSRT, assesses whether the individual can discern which of the relevant significance or meaning of elements in a situation which are important for analyzing a problem, rather than focusing on the irrelevant or even the distracting or disruptive features of the situation. Evaluation itself is also a complex skill. To evaluate how a solution has been achieved or whether a rule can be applied it is necessary to interpret, explain and infer. Figure 3 is another snapshot from our PharmD admission sample. Here we can see that more of the applicant group will need to be supported to train up their skills at interpreting data accurately, evaluating specific cases and, perhaps most importantly for a Pharmacist, critical thinking well when numbers are involved (Numeracy).

Several graphs showing different colors Description automatically generated with medium confidence

Figure 4: Relative weaknesses in particular reasoning skills in a sample of PharmD applicants.

Humans cannot reason well when numbers are involved, if they are averse to or unskilled at interpreting numerical data or proportional relationships. Or if they cannot understand the meaning of probability-influenced data. As a result, Numeracy is a higher order skill reporting on the ability to engage in strong critical thinking in numerical contexts. Notice that Numeracy is not a reference to computations or mathematical theory. Numeracy refers specifically to the ability to reason well in numerical contexts (a strong overlap with the concept quantitative reasoning).

Some key findings – Age and Gender Identity

In various contexts, and over time, we have noted many incorrect assumptions about who among us is more adept at critical thinking. Any remaining prejudices regarding which groups of humans have stronger critical thinking skills need to be set aside. This is good news.

Our data, collected over more than two decades from professional education samples in training programs in every US State and in many other nations globally, demonstrate that excellence in critical thinking is a human capacity found in Professional Program students across the age groups, regardless of gender identification, and in all ethnic and cultural groups. Strong critical thinkers come in all shapes and sizes.

Does age matter?

A screen shot of a graph Description automatically generated

Figure 5: Dental Hygiene students’ Critical Thinking (HSRT OVERALL) scores by Age.

Figure 5 is a modeled national comparison sample of 3,002 Dental Hygiene students. The age of the student can be seen on the vertical axis, ranging from 18 to 61 years. The correlation between age and HSRT OVERALL score describes none of the variance in HSRT scores (r=.096). Extremely low scores (50-60) are largely in the mid-range for age. In this sample, age is likely to explain less than one percent of the variance in scores.

A graph with blue dots Description automatically generated

Figure 6: Pharmacy Students’ Critical Thinking (HSRT OVERALL) scores by age.

In a second longitudinal sample of 755 Pharmacy students, aged 18 to 62, from a single Pharmacy program (Figure 6). The correlation between age and HSRT OVERALL scores is a bit larger (r=.268). This level of relationship would estimate that about seven percent of the variance in these Pharmacy student scores might be due to the age of the student.

Although age has been shown to be relevant in working populations where older workers sometimes have weaker critical thinking skills scores, there is no evidence for age differences in critical thinking skills in the Professional Program student population.

Does gender identification matter?

Professional education, like college education in general, has become more accommodating to the adult student in recent decades. As a result, we can examine the issue of whether critical thinking skills differ by gender identity in the Professional Program student population.

Figure 7 is a graphic of the statistical test for significant difference in HSRT OVERALL scores by self-identified gender in a longitudinal sample from a single Health Sciences program in Medicine. There were 1,208 Medical students in this sample. One standard deviation in scores is shown as a vertical line for each group. There was no significant difference in OVERALL scores for any of the three groups: females (n=718, mean =23.2) males (n=482, mean = 23.4), choose not (n=9, mean = 23.0). This is the expected finding when HSRT scores are examined by gender identification.

Figure7: Medical Students’ Critical Thinking Skills (HSRT OVERALL) scores by Gender Identification (n = 1208)

In Figure 8, shows this relationship in the national comparison sample of Dental Hygiene students that was cited above in the analysis of age. Here the gender identification of the sample as female is predominant. The same findings of no significant difference in OVERALL critical thinking skills scores can be seen. There were 3,002 Dental Hygiene students in this modeled population sample. No significant difference in HSRT OVERALL scores were seen for any of the three groups: females (n=2564, mean 79.8), males (n=162, mean = 79.9), choose not (n=7, mean = 81.8) and missing data (n=269, mean = 78.5).

Figure 8: Dental Hygiene Students’ Critical Thinking Skills (HSRT OVERALL) scores by Gender Identification n=3002

The finding that self-identified females have equally strong critical thinking skill when compared to self-identified males is the common and expected finding in Professional Programs as well as in other college level educational programs. There are some exceptions to this observation in relationship to culture. For example, female university students in predominantly Muslim countries, where women cannot hold professional appointments in many sectors, often score higher than males on college level assessments of critical thinking. Professional educators at these universities have suggested that this may be a function of increased effort in self- development in their female student population and decreased effort by males in programs where male privilege is either experienced or expected.

Does Cultural Group Membership Matter?

With respect to age, gender identification and cultural group membership, there is much more variance within these levels and groups than there is between these groups. For example, in relationship to age, there is significant variance in the critical thinking scores of a group of 20-year-old Professional Program students, but not much explained variance in scores 20-year-olds when compared to 25-year-olds or 30-year-olds.

Economic and educational advantages, the kind of advantages that can easily be seen between cultural groups in society, can and do have considerable influence on the strength of a given student’s critical thinking scores, however. That said, over the past three decades our data has shown that it is possible to self-train critical thinking skills. Learning how to focus attention, systematically analyze, objectively interpret, and thoughtfully evaluate information relevant to a problem space are skills that can be self-learned. These skills can be strengthened when consciously practiced in educational programs and in everyday situations.

But there is a tremendous advantage afforded to students who are mentored and offered thinking exercises as a part of their educational program. If prospective students have not had the advantage of services offer personal tutoring or training about common human reasoning errors, these students will be more likely to make thinking errors in general and to demonstrate lower scores on objective critical thinking assessments.

One significant variable may be culture, if the culture does not offer some groups of students the same level of educational advantages that other groups have for their learning programs. Economic differences accentuate this imbalance because wealthier students can often compensate for differences in educational opportunities when they purchase additional training and mentoring. This makes it more imperative for Professional Programs to assess critical thinking and provide training support for those students who are less prepared to engage with their demanding professional preparation programs. Critical thinking skills, as the evidence amply shows, can be developed.

Socio-cultural expectations and economic advantage, not ethnicity or race. In our database of assessments completed by individuals who are members of many national and ethnic groups, critical thinking skills scores range across the entire continuum of possible scores in every Professional Program sector.

Pandemic effects

During the uncertainty of the COVID pandemic, most colleges level programs experienced unprecedented challenges to maintaining their student learning environments. This was especially true for programs where laboratory sessions and clinical care rotations were a necessary part of the training. Many students in the health sciences were participants in the caregiving needed to manage COVID patients. As a result, we can observe, but not interpret with certainty, the factors that resulted in fluctuations in the strength of critical thinking assessment data during the pandemic.

In many ways it is reasonable to expect that the intensity of the novel problems during this period would lead many Professional Program students to reflect more intensively on their thinking process and improve their reasoning process through practice as they strove to cope with the uncertainty. Like most of us, in these years nearly every well-defined procedure now required thoughtful problem-solving under conditions of uncertainty. Should we not see stronger mean scores are a result?

On the other hand, the increasing uncertainty and disruption, cognitive fatigue and physical illness, anxiety, fear, isolation, and loss, all commonly experienced by Professional Program students will certainly have influenced students’ focused attention capacity and cognitive energy. We might reasonably expect that students will have difficulty providing their best effort when completing assessments of critical thinking. Should we not expect to see weaker scores as a result?

The process of collecting assessments of critical thinking for use in training and advising proved more difficult for professional education programs during the pandemic. Most program directors developed flexible strategies that allowed them to maintain their admissions and student retention programs. The few programs that abandoned their effort to collect critical thinking data from newly admitted students and exiting students (longitudinal programs to examine teaching effectiveness) did so because of unexpected school closures, campus and hospital lockdowns and other inflexible occurrences in the early years.

In examining the aggregate data, we observed mean score fluctuations in reasoning skills for all the professional programs. We also saw more examples of incomplete assessments being submitted, particularly in the Fall 2020 through Spring of 2021. The result has been a less stable mean score for applicant pools in general. Figures 9 shows a typical example of how mean scores for applicant pools have been less constant during the pandemic period.

Figure 9: Dental Medicine applicant scores HSRT OVERALL by year.

In the case of the Dental medicine sample displayed in Figure 9 for 2019, the score range (OVERALL = 50-97) and the sample size (n = 210) were characteristic of a typical program year. The lower mean in 2020 was also for a much smaller sample (n=50). Although the range in scores was not significantly different, the distribution of scores tipped left (more scores in the weaker range).

In the years since 2020, the critical thinking skills score data for all sectors show that people are making an increased number of common reasoning errors. Historically the distribution of scores for most applicant pools has been either normal in its distribution (bell-shaped curves) or in the case of highly selective programs, a score distribution where higher scores are more plentiful (right leaning distribution). In other words, there have always been some program applicants who demonstrate very weak critical thinking skills, and some or many others who excel with strong and superior scores. Since 2020, the range of scores for many Professional Program national aggregate samples have shown a weaker distribution of scores (left leaning).

Professional Programs have seen a growth in applications in general in 2023, an encouraging sign that an increasing number of people are finding it possible to pursue training in challenging professional development programs. Although it is too soon to determine whether the demonstrated strength in applicants’ critical thinking skills will return to their past higher and more stable range.

For now, continued monitoring of the capabilities of our future leaders as they complete their professional education programs is the only option.

This report is one of a series of white papers prepared by our research team to inform researchers and trainers of critical thinking skills and mindset. By P. Facione, N. Facione, C. Gittens

More Background

We have been discussing data collected with a group of critical thinking assessments that have been scientifically developed and tested over a period of more than 30 years. Here are some of the key considerations in this process.

Four key considerations for developing and maintaining valid and reliable critical thinking assessments:

  1. The valid and reliable assessment of the reasoning skills and thinking mindset of working people in varying sectors requires tailored instruments calibrated to varying levels of decision responsibility. Today the INSIGHT Series includes assessments for executives, professionals, and two levels of support staff.
  2. Valuable input from working professionals and acknowledged leaders in each sector make it possible to create and validate instruments tailored specifically for people who work in Business, Health Care, Defense, Science and Engineering, Law, and as Educators and First Responders.
  3. Over thirty years, collaborative projects with scholars and professionals globally have extended the availability of culturally relevant language translations.
  4. Independent research and assessment projects have produced comparative data on the distribution of critical thinking skills and mindset attributes for employees from support staff to top level leadership.
  1. Exploration of Relationships Among the Health Sciences Reasoning Test, the National Physical Therapy Licensing Examination, and Cognitive Admission Variables. Huhn K & Parrott JS. J. Phys. Ther. 2016, 31(1).
  2. Assessment of Critical Thinking as a Predictor of Success in Completion of an Associate Degree Respiratory Care Program. Campbell FP. (2017) Dissertation.
  3. Impact of Students’ Strengths, Critical Thinking Skills and Disposition on Academic Success in the First Year of a PharmD Program. Comer RD, Schweiger TA, & Shektib P. 2019.Am J Pharm Educ 83(1) 6499.
  4. The health sciences reasoning test in the pharmacy admissions process. Kelsch MP, Friesner DL. College of Pharmacy, Nursing, and Allied Sciences, North Dakota State University. Am J Pharm Educ. 2014 Feb 12; 78(1): 9.
  5. Critical Thinking Disposition and Skills in Dental Students: Development and Relationship to Academic Outcomes. Whitney EM, Aleksejuniene J, Walton JN. J. Dent. Educ. 2016 Aug;80(8): 948-58.
  6. Predictive validity of critical thinking skills and disposition for the national board dental hygiene examination: a preliminary investigation. Williams KB et al. (2006). Journal of Dental Educ. 70(5), 536-44.
  7. Examining Critical Thinking Skills in Family Medicine Residents. Ross D and colleagues. Fam. Med. 2016 Feb;48(2):121-6
  8. The influence of critical thinking skills on performance and progression in a pre-registration nursing program. Pitt V, et al, Schools of Nursing, Midwifery and Psychology, University of Newcastle, Australia. Nurse Educ Today. 2015 Jan;35(1):125-31.
  9. Enhanced numeracy skills following team-based learning in United States pharmacy students: a longitudinal cohort study. Carpenter RE, Coyne L, Silberman D, Takemoto JK. 2022. J. Educ Eval Health Prof. 19. 29.
  10. An Assessment of the Command and General Staff Officer Core Course Effectiveness in Developing Student Critical Thinking. Civils, TH (Dissertation).
  11. Does Use of an Electronic Health Record with Dental Diagnostic System Terminology Promote Dental Students’ Critical Thinking? Reed S, Adibi SS, Coover M, Gellin RG, et al. J Dent Educ 2015m 79 (6), 686-696.