What the Critical Thinking Data Tell Us: Critical Thinking in the Workplace

What have we learned about the critical thinking ability and mindset of our workforce?

When human resource professionals in business and government agencies assess critical thinking skills and mindset, they are collecting real-time assessments of potential hires, and comparing them to the strength of the prospective hiring pool. Managers, commanders, and trainers are evaluating the effectiveness of employee development programs. Here are a few examples of what has been learned.

The projects discussed below administered the INSIGHT Series of assessments, tailored specifically for Business, Military and Defense, Health Care, Science, Engineering & Technology, and First Responders.

Critical Thinking Assessment in Hiring:

In our discussions with managers, executives, and employee development officers, we hear that these are primary drivers of the focus on critical thinking when hiring:

     

      • Today’s changing workplace demands adaptability, continuous reconsideration, and accurate timely action.

      • In high-stakes situations, errors in judgment are very costly events.

      • Mistakes in problem-solving and risk management are a fundamental threat to the achievement of goals.

      • Innovative leadership and creative new development require getting the problem right and keeping numbers in the mix.

      • Success demands an honest, judicious, and foresightful evaluation of options, and the resourcefulness and commitment to work until goals are achieved.   

    Corporate and government leaders know they cannot assume that designated managers and staff will have the needed strength in critical thinking required of the position. Here is an example. These graphics show scores for two of the five scores reported for reasoning skills on INSIGHT First Responder, an assessment tailored for hiring individuals who will work independently in high-stakes, high-risk, decision-intensive situations. These two scores focus on Evaluating Options and Projecting Outcomes. Scores for individuals with critical thinking scores in the green (Strong) and blue (Superior) zones are the much-preferred applicant pool.

    Figures 1 and 2. INSIGHT First Responder – 2 of the 6 Reasoning Skills Scores (Sample: 1,133 job applicants)

    If a hiring agent is inclined toward hiring a candidate who interviews well, and yet demonstrates areas of weakness in one or more reasoning skill scores (Evaluating Options, Determining Impacts, etc.), or lacks the thinking mindset attributes expected in professionals who are first responders (Foresightful, Inventive, Courageous, etc.), the assessment results can be used to set on-boarding goals for the new hire. 

    For many business leaders, having assessment scores in hand at the time of the candidate interview is the preferred option, and this method works well for integrating an assessment of reasoning skills and mindset in the hiring of many types of employees. Most assess critical thinking early in applicant screening, to screen interview candidates for “Strong” or “Superior” results. Applicants can be given a secure login to complete both the mindset and reasoning skills sections of INSIGHT Series assessment in 90-minutes or less (the optimal time frame for a cognitive assessment), The assessment results are pushed to the hiring agent for availability as needed in the interview session.

    Armed with INSIGHT assessments, a hiring professional can identify candidates who have demonstrated reasoning skills in the “Superior” range but also mindset characteristics needed in leadership, innovation, and risk management.

    Figures 3 and 4.  INSIGHT First Responder – 2 of the 9 Mindset Attribute Scores (Sample: 1,133 job applicants)

    These two graphics show the variation in this applicant pool of 1,133 applicants for a government security position. The two areas shown are the candidate’s orientation to teamwork, as well as their tolerance of diverse cultural and population groups. All Insight measures have linked mindset assessments because the requirement is that the candidate be both willing and able to think well in their decision-intensive position. Insight First Responder provides scores for nine mindset metrics, useful for hiring agents and trainers who can then determine the degree to which these attributes will be needed for excellent performance in the specific position.

    Once the value of an assessment becomes clear, because new hire test scores are consistent with subsequent supervisor ratings and department head observations, many clients roll out of a critical thinking initiative for other employee groups.

    Achieving workforce goals for improving innovation, problem-solving and risk management is possible and should be the expected standard at all levels. This is achieved by hiring individuals with strength in critical thinking skills and mindset, as managers, as team leaders, as C-suite executives. Individual reports for mentees or new hires provide a profile for each employee assisting mentors and trainers as they work individually with each candidate to reinforce strengths and address areas where growth is needed.

    What about starting at the top?

    In a different workforce population, business executives, a different set of mindset attributes are relevant, and a demonstration of strength in reasoning skills is typically important to an increasing number of organization employees as well as clients or community members. INSIGHT Executive is tailored for the assessment of executives, and used most often when executives are being mentored or trained, or an executive group is working together to tune their leadership and decision-making capability.

    In the case of this group of 113 C-suite candidates, most were strong in the ability to make judgments in uncertain conditions (left) and could accurately envision possible options (below).

    Figure 5-7. INSIGHT Executive 3 of the 6Reasoning Skills Reported (Sample: 113 Business Executives)

    Many in this group had limitations when it came to identifying logically certain constraints and consequences (Figure 7 below, yellow and red bars). The Logical Consequences score assesses a candidate’s strength in deductive reasoning, a cognitive skill needed to see the applications and the implications of regulations, policies, basic assumptions, core principles, protocols, and relationships which both shape and constrain executive decision making.

    Figure 7. INSIGHT Executive 3 of the 6Reasoning Skills Reported (Sample: 113 Business Executives)

    Executive search firms know that matching a candidate with the job is vital to the candidate’s success, and that candidate pools are highly variable. Few executives are successful in the absence of strong critical thinking skills and a thinking mindset. INSIGHT Executive provides information to inform the self-development of clients at executive prep firms, as well as the evidence that the candidates they recommend have the skills and attributes necessary for the job.

    Focusing Employee Development:

    Assessments of groups of employees provide an organization with a picture of the critical thinking strengths and weaknesses of their workforce.  This graphic displays the varying strength in overall critical thinking skills in a group of mid-career professionals responsible for monitoring the accuracy of financial reports in a government agency. Employees at this agency work independently and the expectation is that all possess the ability to identify, interpret and analyze problems, and evaluate the implicit judgments and actions represented by their reporting to the government agency. Expected overall critical thinking skills scores for this group of 3,670 working professionals should fall in the strong (green) and superior (blue) range demonstrated by approximately two-thirds of the 3670 employees in this group. And yet nearly a third of these employees were unable to demonstrate that expected level of competence.

    Figure 8. INSIGHT Business Professional Skills OVERALL Scores (Sample: 3,670 federal employees)

    A deeper dive into the score data produced by the INSIGHT Business Professional reasoning skills assessment, used in this project, shows that areas for the greatest professional development included Problem Analysis and Reasoning in Precise Contexts. Getting the problem right in the first place (Problem Analysis) is, of course, fundamental for strong critical thinking and decision-making. And being able to reason with precision is essential where applying regulations, laws, and quantitative information are elements in the job description.

    This project was focused both on improving reasoning skills and building a culture of workers with a thinking mindset. Having the habits of mind (mindset) to engage problems using strong reasoning skills is as important as having the skills themselves. The mindset portion of INSIGHT Business Professional provides scores for ten metrics describing a critical thinking mindset in the business professional.

    Figure 9. INSIGHT Business Professional Mindset – Judicious Scores (Sample: 2,790 federal employees)

    Employees who score in the Strong Positive range for judicious have these mindset descriptors:

       

        • A thoughtfulness that accurately views many problems as ill-structured and/or novel (maturity of judgment).

        • An awareness that there may be more than one reasonable option for responding to and resolving problems.

        • The habit of dispassionately examining previously made judgments, now seen as weak or poorly made, for their ability to provide insight and inform the quality of future judgments.

        • The expectation that many judgments must be made under conditions of risk and uncertainty, often when time is constrained and knowledge of the facts of the situation is sub-optimal.

      Consider this distribution of scores for the mindset characteristic of being judicious in this group of 2,790 employees. The majority of these employees scored in the higher range of Positive (yellow bars) or Strongly Positive (blue bars), demonstrating a maturity of judgment and thoughtfulness in their approach to decision-making.

      Scores in the lower Positive range and in the “Not Manifested” range (red bars) indicate a lack of discrimination regarding ill-structured problems and a rejection of the notion that there may be more than one reasonable way to resolve problems. These individuals generally disagree with the idea that there are times when a person should re-visit and revise poorly made judgments. They often commit errors of omission because they fail to make timely judgments.

      INSIGHT Business Professional reports Mindset scores for ten metrics. Areas of strength in this group of employees included: Focus, Tolerance, Foresight, and Professionalism. A broader continuum of scores were seen on several other metrics: Adaptable, Resourceful, Honorable, and Committed. These were used to guide development and engagement programs at the national level.

      Some reasoning skills cannot be absent because they are vital to the job:

      Employers are increasingly providing critical thinking professional development options as a component of on-boarding and employee advancement, but longer-term employees are less likely to have benefitted from these initiatives. This is a common finding in many employee groups where an assessment of critical thinking was not a part of the initial hiring process. In this example dataset, employees are participating in a development program for professional nurses at a very large medical center. As with all professions where training and passing a licensing examination is required to enter professional practice, these nurses have demonstrated mastery of the clinical skills and content knowledge needed for their nursing position. As a function of employee development, and as a component of an initiative to improve decision-making and decrease errors in health care delivery, this group was assessed with INSIGHT Health Professional. Desired scores are 285 and above. This group of working professionals varied widely in their demonstrated critical thinking skills.

      Figure 10. INSIGHT Health Professional – OVERALL skills (Sample: 151 nurses)

      The scale scores for the group identified Analysis, Inference and Explanation as areas of strength (all needed for diagnostic reasoning, and the ability to explain the rationale for treatment plans.  Areas of focus for subsequent employee development, where skills were sub-optimal for more than half of the cohort, centered on two specific cognitive skill areas: Evaluation and Numeracy. 

         

          1.  Evaluation refers to the ability to assess the credibility of the reasoning display when making arguments or giving explanations. People with strong evaluation skills have the practiced ability to judge the quality of their decisions and actions.

          1. Numeracy refers to the ability to make judgments based on quantitative information in a variety of contexts. Quantitative information can take the form of proportions, likelihoods, flow-rates, or many other contexts where numbers matter. Numeracy, sometimes called quantitative reasoning, includes being thoughtfully reflective while interpreting the meaning of the quantitative information expressed in charts, graphs, or text formats, analyzing those elements, drawing accurate inferences from that information, and explaining and evaluating how those conclusions were reached.

        The benefit of training critical thinking increases with time.

        Most educators and trainers believe that an effective training program focused on improving reasoning skills and mindset will have a continued impact as the learner integrates and practices what has been learned. Many subjective experiences of personal growth after training programs support this idea, but objective evidence in support of this idea is very rare. 

        One very well designed and carefully evaluated training program in critical thinking does clearly demonstrate this sustaining and evolving effect in a United States Air Force (USAF) training program for health care providers. As a component of a leadership training program for Air Force Nurses, growth in critical thinking skills over time was studied over an eight-year period using INSIGHT Health Professional.[1] Significant gains in critical thinking skills were demonstrated at the end of the 3-month leadership training program delivered at six base locations (pretest to posttest). Each base used a different training team but the same USAF training curriculum, controlling for any trainer-specific effects and allowing the evaluation of the curriculum itself, This USAF study is notable because it also required a third assessment of these same personnel after one year of deployment at hospitals located on USAF bases around the world. One year post training, the critical thinking skills of these health professionals continued to significantly improve their critical thinking skills over their demonstrated scores at the end of their training period. An effect was demonstrated in successive cohorts of trainees over the eight-year period.

        The Emphasis on Critical Thinking is Global We speak daily with agency representatives around the world seeking information about critical thinking assessment and how they can integrate this focus into their strategic planning. Because a concern for better thinking is universal and not just a consideration of the western world, our team has been working for decades to decrease language and culture barriers to critical thinking training and assessment. Our deep discounts for advanced degree candidates is one example.

        Figure 11. INSIGHT Business Professional – OVERALL skills and 5 scale scores (Sample: 199 government applicants)

        In the case of international clients who have been actively hiring and training for strength in critical thinking for more than a decade, we have begun to see evidence of their success in building a better thinking workforce. Figure 11 displays a non-US sample of applicants for government positions assessed with INSIGHT Business Professional, the strength in critical thinking skills of the applicant pool is undeniable. The center graph shows an integrated assessment of critical thinking skills (OVERALL score). The surrounding graphics show relative strength in the applicant pool in all areas. Having established their baseline expectation of strength in critical thinking and this agency has evolved an applicant pool where their hires can be expected to have “Strong” or “Superior”-level reasoning skills. The agency uses the individual report for each candidate, allowing the government agent responsible for hiring to examine each candidate’s performance. Individuals vary greatly in how they have practiced their reasoning skills, and some candidates with “Strong” scores may have areas of relative weakness (typically in making judgments in very precise contexts or ones involving strength in quantitative reasoning).

        We are proud to be partnering with agencies working toward better reasoning and we are inspired by the work our clients do in countries around the world.

        More Background

        We have been discussing data collected with a group of critical thinking assessments that have been scientifically developed and tested over a period of more than 30 years. Here are four of the key considerations in this process:

        Four key considerations for developing and maintaining valid and reliable critical thinking assessments:

           

            1. The valid and reliable assessment of the reasoning skills and thinking mindset of working people in varying sectors requires tailored instruments calibrated to varying levels of decision responsibility. Today the INSIGHT Series includes assessments for executives, professionals, and two levels of support staff.

            1. Valuable input from working professionals and acknowledged leaders in each sector, make it possible to create and validate instruments tailored specifically for people who work in Business, Health Care, Defense, Science and Engineering, Law, and as Educators and First Responders.

            1. Over thirty years, collaborative projects with scholars and professionals globally have extended the availability of culturally relevant language translations.

            1. Independent research and assessment projects have produced comparative data on the distribution of critical thinking skills and mindset attributes for employees from support staff to top level leadership.

          Anything that is valued, is measured. Employees typically attend to the metrics their supervisors use to evaluate their workplace performance. “If it is something my company values enough to measure, then I had better be doing it well”. Over the past three decades building critical thinking skills and a thinking mindset has moved from a theoretical academic discussion to a global concern for employers, educators, and those seeking to advance fair-minded truth-seeking for the sake of democracy and the reasoned pursuit of our common good.

          Published articles increasingly document the growing numbers of critical thinking-focused educational reports, research studies, corporate projects, and critical thinking focused initiatives globally.[1]  Given the ever-increasing focus on assessing and training critical thinking, and the increasing global energy around critical thinking, we can draw several confident conclusions. First, strong thinkers are valued. Second, measuring critical thinking yields actionable results. We also hope two workplace responses will follow: Those who display strength in critical thinking skills and mindset will be rewarded, and those whose deficits are objectively identified will be offered training.

          When leaders in government, the health professions, business, and the military call for greater attention to critical thinking, their focus is practical. They rightly believe that strength in critical thinking results in better decision-making and better problem-solving. And these mean better government, business, healthcare, and military outcomes. This very practical, bottom line, attention to critical thinking creates demands for effective employee development programs, including accurate empirical assessments. In reporting these general observations, we have mentioned government projects whose descriptions are available to the public. We have also presented other general information with consideration of client privacy. Many of these client organizations are societal leaders in calling for a focus on developing a culture of critical thinking in the workplace and in our government agencies. Our hope is to inform agency leaders of the potential for growth in leadership strength and overall employee performance when more emphasis is placed on assessing the reasoning skills and thinking mindset of prospective agency leaders and staff.


          [1] INSIGHT Health is a version of the Health Sciences Reasoning Test tailored specifically for the working professional, Levels of INSIGHT Health designed for various practice levels allow selection of an assessment to match the workplace roles and responsibilities. 

          [2] Aghababaein, P., Moghaddam, S.A.H., Nateghi, F., & Faghihi, A.  (2017). Investigating changing in social studies textbooks of public review (basic fourth and fifth) based on the emphasis on critical thinking skills Facione [sic] in the last three decades. International Education Studies, 10(3), 108-115. Alkharusi, H. A., Sulamani, H. A., & Neisler, O. (2019). Predicting critical thinking ability in Sultan Qaboos University students. International Journal of Instruction, 12, 491-504.  Huang, Y. C. et al. (2012). Case studies combined with or without concept maps improve critical thinking in hospital-based nurses: a randomized-controlled trial. International Journal of Nursing Studies, 49, 747-754. Jacob, S. M. (2012). Analyzing critical thinking skills using online discussion forums and CCTST. Procedia – Social and Behavioral Sciences, 31, 805-809. Paans, W., Sermeus, W., Niewsg, R., & van der Schans, C. (2010). Determinants of the accuracy of nursing diagnoses: Influence of ready knowledge, knowledge sources, disposition toward critical thinking and reasoning skills. Journal of Professional Nursing, 26(4), 232-241.  Suliman, W. A. (2008). Critical thinking and learning styles of students in conventional and accelerated programmes. International Nursing Review, 53(1), 73-79. Tiwari, A., Lai, P., So, M., & Yuen, K. (2006). A comparison of the effects of problem-based learning and lecturing on the development of students’ critical thinking. Medical Education, 40, 547-554.  Yuan, H, et al. (2008). Improvement of nursing students’ critical thinking skills through problem-based learning in the People’s Republic of China: a quasi-experimental study. Nursing & Health Sciences, 10(1), 70-76. 

          This report is one of a series of white papers prepared by our research team to inform researchers and trainers of critical thinking skills and mindset. 

          P. Facione, N. Facione, C. Gittens.

          ©2024, Measured Reasons LLC, Hermosa Beach CA USA, and C. A. Gittens, Morgan Hill, CA, USA.
          Published by Insight Assessment. www.insightassessment.com  Advancing Thinking Worldwide