Home / Resources / Research Findings / Evaluating Training Techniques

Evaluating Training Techniques

A collection of studies describing and evaluating a variety of training techniques for evidence that they effectively train critical thinking skills or mindset.

Efficacy of Skill Workshop for Doctoral Physical Therapist Students

  • The Efficacy of a Skill-Building Workshop for Reflective Critical Thinking with Graduate Students: Effect-Size Differences Based on Race. Michaels, Natalie PT, Ed.D.,  Journal of Interdisciplinary Education, Vol. 14, No. 1 – April 2017 ISSN: 1092-7069 © 2017 North American Community. The purpose of this study was to develop and evaluate the effectiveness of a workshop created to help doctoral physical therapist students build reflective critical thinking strategies. The workshop participants included 48 doctoral physical therapist students who were placed in two separate groups: the workshop group and the control group. Both groups were given the Health Sciences Reasoning Test (HSRT) before and after the workshop.  No significant differences were found between group comparisons. Within group comparisons, however, higher pre-test HSRT scores for Caucasian students were noted, but African American students achieved higher post test scores. Factors potentially effecting results included novelty of learning situation, prior exposure to tested concepts/strategies, and need for authentic practice opportunities.  Further research is recommended.

Critical Thinking in Social Studies Textbooks in Iran

  • Investigating Changing In Social Studies Textbooks of Public Review (Basic Fourth and Fifth) Based on the Emphasis on Critical Thinking Skills Facione in the Last Three Decades. Aghababaeian P, et.al.  (2017) International Education Studies.  This study investigated the changes in public school social studies textbooks in general period of Iran (fourth and fifth grades) based on the emphasis on Facione critical thinking skills in the past three decades. Content analysis of qualitative and quantitative methods was used to evaluate changes in textbook. The data gathering instruments were self-made content analysis checklist based on Facione elements, and validity was determined by experts of education science. Reliability of the instrument was calculated by the Ali Delaware formula (85% agreement coefficient). Findings indicate that in the 4th and 5th grade social studies books written in the last three decades attention to other critical thinking skills is very weak. Results of present study of all the studies mentioned above indicate lack attention from policy makers and textbook authors to discuss critical thinking skills. In fact, the new social studies books written (fourth and fifth grade) in the 90 books of fifth grade to fourth grade books written weakest in terms of attention is devoted to critical thinking skills. Recommendations include 1) Providing opportunities for participation and critical thinking skills in the classroom, 2) revision of the social studies books which have recently been developed in the ‘90s by planners and authors of textbooks in primary schools, 3) Holding in-service training courses for experts and teachers regularly, in order to provide knowledge and positive attitude towards different aspects of critical thinking and motivate them to use these skills into practice in the classroom; and 4) Designing educational package and software, especially for critical thinking skills of teachers, experts and students and present it to teachers and students to supplement the textbook content.

Impact of Virtual Patient Simulation on Critical Thinking

  • Assessing Critical Thinking Outcomes of Dental Hygiene Students Utilizing Virtual Patient Simulation: A Mixed Methods Study. Allaire JL. (2015). Journal of Dental Education. Dental hygiene educators must determine which educational practices best promote critical thinking, a quality neces­sary to translate knowledge into sound clinical decision making. The aim of this small pilot study was to determine whether virtual patient simulation had an effect on the critical thinking of dental hygiene students. A pretest-posttest design using the Health Science Reasoning Test (HSRT) was used to evaluate the critical thinking skills of senior dental hygiene students at The University of Texas School of Dentistry at Houston Dental Hygiene Program before and after their experience with computer-based patient simulation cases. Additional survey questions sought to identify the students’ perceptions of whether the experience had helped develop their critical thinking skills and improved their ability to provide competent patient care. Individual students made strong and significant gains from pretest to posttest in this convenience sample of senior dental hygiene students (N=31). Students perceived the simulator experience as an effective teaching method to promote critical thinking, problem-solving, and confidence in the clinical realm. Although the results did not show a significant increase in mean scores, the results of this pilot study may have implications to support the use of virtual patient simulations in dental hygiene education. Future research could include a larger controlled study to validate findings from this study.

Writing in Biology Education

  • Learning to Improve: Using Writing to Increase Critical Thinking Performance in General Education Biology. Quitadamo IJ, Kurtz M. (2007). Life Sciences Education. The Researchers compared critical thinking performance ( CCTST Scores) of students who experienced a laboratory writing treatment with those who experienced traditional quiz-based laboratory in a general education biology course. The writing group significantly improved critical thinking skills whereas the non-writing group did not. Specifically Analysis and Inference scores improved significantly. Other co-variables such as gender, ethnicity and age were not significant.

Involving Undergraduates in Research

  • The Relationship between Undergraduate Research and Critical Thinking Skills. Denny JP (Univ Florida Dissertation 2012) The purpose of this study was to examine the relationships between participation in an undergraduate research program, critical thinking skills, common educational outcomes and academic performance. The primary significant relationship found, using a Wilcoxon Rank Sum, was that students who had four or more semesters of experience with undergraduate research tended to have higher scores on the California Critical Thinking Skills Test ( CCTST ) than those with three or less semesters of experience. Analysis revealed a significant correlation between collaborative learning and CCTST score, as well as active learning and CCTST score. There were no significant correlations between CCTST score and prompt feedback, time on task, or respect for diverse ways of learning.

Case-Based Learning

  • Critical thinking skills of nursing students in lecture-based teaching and case-based learning.  Kaddoura M (2011). International Journal for the Scholarship of Teaching and Learning. A strategy that promotes students' active learning is case-based learning (CBL). The purpose of this study was to examine critical thinking (CT) abilities of nursing students from two different curricular approaches, CBL and didactic teaching. The sample (N=103) included 65 students from the CBL nursing program and 38 students from the didactic nursing program offered by the MOH Schools of Nursing in the United Arab Emirates (UAE). Data were collected using the California Critical Thinking Skills Test ( CCTST ) to measure the CT abilities of the participants. The independent t-test results revealed that the CBL participants performed better in the total CT score and all CT subscales than the didactic program participants.

Patient Simulators

  • Measuring critical thinking dispositions of novice nursing students using human patient simulators. Wood RY, et al. Boston College. J Nursing Education. (2012). This study assessed the influence of human patient simulator (HPS) practice on critical thinking dispositions in a sample of novice baccalaureate nursing students. Eighty-five second-year nursing students were randomly assigned to an experimental (n = 42) or a control (n = 43) group based on exposure to a 2-hour HPS practice session prior to a course competency examination. The California Critical Thinking Disposition Inventory ( CCTDI ) was administered before and after the competency examination. No between-group differences were found on overall or subscale CCTDI mean scores. Within-group differences for the HPS practice group were significant for overall scores (p < 0.05) and the truth-seeking (p < 0.01) and judiciousness or maturity of judgment (p < 0.01) subscales. This preliminary data analysis suggests disposition gains for individual students practicing critical assessment skills using HPS. The cohort will be followed for 2 years to assess long-term critical thinking outcomes following practice with HPS.

Evaluating a Focused Program

  • Effects of an experiential learning program on the clinical reasoning and critical thinking skills of occupational therapy students. Coker P. (2010). Journal of Allied Health. This study examined the effects of participation in a 1-week, experiential, hands-on learning program on the critical thinking and clinical reasoning skills of occupational therapy students (N=25). The students had completed three semesters of didactic lecture coursework in a master's level OT educational program prior to participation in a hands-on therapy program for children with hemiplegic cerebral palsy. Changes in critical thinking and clinical reasoning skills were evaluated using the Self-Assessment of Clinical Reflection and Reasoning (SACRR) and the California Critical Thinking Skills Test ( CCTST ). Changes in pretest and posttest scores on the SACRR and the CCTST were statistically significant (p>0.05) following completion of the experiential learning program.

Questioning

  • Development of critical thinking in occupational therapy students. Velde BP, et al. (2006). Occupational Therapy International. Do students who use the Guided Reciprocal Peer Questioning (GRPQ) method increase their ability to think critically? When compared to students in a traditional senior seminar course, the results of this study indicated no significant difference between the groups regarding changes in scores on the CCTST . However, the students in the experimental group asked more questions labeled as critical thinking than the seminar control group. Future research regarding the role of questions in stimulating critical analysis and the role of context in the learning environment is warranted.

Reflective Blogs

  • Reflective blogs in clinical education to promote critical thinking in dental hygiene students. Wetmore AO, et al. Eastern Washington University. J Dent Educ. (2010). One challenge facing dental hygiene, as well as dental, education is to identify clinical teaching strategies promoting critical thinking and clinical reasoning. These skills are crucial elements in the practice of dental hygiene. A two-group design (intervention, n=28, and control, n=30) assessed first-year dental hygiene students using pre-and post-Health Science Reasoning Test ( HSRT ) scores to evaluate the effect of reflective blogging on critical thinking skills. A reflective blog rubric, based on Mezirow's levels of reflection, determined if reflective blogging increased the level of reflection for dental hygiene students. The results suggest within this nonprobability sample that reflective blogging did not produce a significant change in students' HSRT scores (p>0.05). However, analyses of reflective blog rubric scores demonstrated statistically significant improvements (p<0.05) in students' levels of reflection. Furthermore, data analysis revealed a correlation (p<0.05) between HSRT subscale scores and the element of reflection scores for the intervention group. This study addressed needs of the dental and dental hygiene education community by examining the use of blogs, an emerging technology, as a tool for reflecting on clinical experiences and, in turn, for promoting critical thinking.

Narratives

  • Cognitive and ethical maturity in baccalaureate nursing students: did a class using Narrative Pedagogy make a difference? Evans BC, Bendel R. (2004). Nursing Education Perspectives.This project examined the effect of Narrative Pedagogy in nursing education on students' CT disposition as a measure of their cognitive and ethical maturity and increased autonomy in nursing practice. Exit scores were marginally and statistically significant improved over entry scores for the program (p < 0.05) for both the Measure of Intellectual Development and the CCTDI . There was minor improvement on both scales in cognitive and ethical maturity and the disposition to think critically in the intervention group.

Problem-Based Learning

  • A comparison of problem-based and traditional education on nursing students' critical thinking dispositions. Ozturk C, et al. (2008). Nurse Education Today. This descriptive analytic study compared levels of critical thinking among senior nursing students (N=147) in two educational programs in Turkey, one of which used a problem-based learning (PBL) model while the other used a traditional model. Using the California Critical Thinking Disposition Inventory ( CCTDI ) to measure CT disposition, there were significantly higher (p<0.05) scores in Truthseeking and Openmindedness in seniors in the PBL school when compared to those in the school implementing the traditional model.

Problem-Based Learning

  • A comparison of the effects of problem-based learning and lecturing on the development of students' critical thinking. Tiwari A, et al. (2006). Medical Education. The aim of this study was to compare the effects of problem-based learning (PBL) and lecturing approaches on the development of students' critical thinking. The primary outcome measure was students' critical thinking disposition as measured by the California Critical Thinking Disposition Inventory ( CCTDI ). Data were collected at 4 time points spanning 3 years. Overall CCTDI and subscale scores were not significantly different by group at pretest. Compared with lecture students, PBL students showed significantly greater improvement in overall CCTDI (P = 0.0048), Truthseeking (P = 0.0008), Analyticity (P = 0.0368) and Critical Thinking Self-confidence (P = 0.0342) subscale scores from the first to the second time points; in overall CCTDI (P = 0.0083), Truthseeking (P = 0.0090) and Analyticity (P = 0.0354) subscale scores from the first to the third time points; and in Truthseeking (P = 0.0173) and Systematicity (P = 0.0440) subscale scores from the first to the fourth time points. CONCLUSIONS: There were significant differences in the development of students' critical thinking dispositions between those who undertook the PBL and lecture courses, respectively.

Problem-Based Learning

  • Improvement of nursing students' critical thinking skills through problem-based learning in the People's Republic of China. Yuan H, et al. (2008). Nursing Health Science. A quasi-experimental, two-group pretest-post-test design was conducted to examine the effect of problem-based learning on the critical thinking skills of 46 Year 2 undergraduate nursing students in the People's Republic of China using the CCTST (Chinese-Taiwanese version). Groups were not significantly different in CT skills at pretest, whereas, the problem-based learning students had a significantly greater improvement on the overall CCTST total score, analysis subscale, and induction subscale scores compared with the lecture students at posttest.

Questioning as a Training Technique

  • Teaching students to question. Loy GL, et al. (2004). American Journal of Obstetrics and Gynecology. Workshops were developed to teach medical students how to systematically ask critical questions. Sixty-two consenting students in their third-year obstetrics and gynecology clerkship were divided according to alternate rotations to either attend the workshops (n = 28) or not (n = 41). Medical students who attended the workshops scored higher on the CCTST mean total score (study group 25.1 [ 0.7 SEM] vs control group 22.9 [ 0.6 SEM], P = .028), subscales of inference (12.6 [ 0.3 SEM] vs 11.2 [ 0.3 SEM], P = .003), and of deductive reasoning (12.7 [ 0.4 SEM] vs 10.9 [ 0.3 SEM], P = .001). CONCLUSION: Teaching students to ask critical questions improves critical thinking as measured by the California Critical Thinking Skills Test.

Concept Mapping

  • Evaluating critical thinking in clinical concept maps: a pilot study. Pastirik PJ. (2006). International Journal of Nursing Education Scholarship. The purpose of this pilot study was to determine the level of critical thinking in the clinical concept maps developed by second year baccalaureate nursing students. The data for the study included eighteen concept maps, 1 student focus group and 1 instructor focus group. The Holistic Critical Thinking Scoring Rubric ( HCTSR ) was used to measure levels of critical thinking, and content analysis was used to analyze focus group data. Results indicated that developing concept maps in the clinical setting fostered critical thinking and improved clinical preparedness.

Concept Mapping

  • The influence of concept mapping on critical thinking in baccalaureate nursing students. Wheeler LA, Collins SK. (2003). Journal of Professional Nursing. Pretest-posttest design with a control group to evaluate the effectiveness of concept mapping in developing critical thinking skills (n = 76). Participants were randomly assigned to experimental (n = 44) and control (n = 32) groups. The experimental group was taught to use concept mapping of patient information to prepare for clinical experiences. The control group was taught to use traditional nursing care plans. Critical thinking skills were measured with the CCTST . Experimental-group scores improved significantly (p <.05) on the overall score and the analysis and evaluation subscales, while control-group scores improved significantly only on the evaluation subscale and declined significantly on the inference subscale.

Patient Simulators

  • The effects of simulation on nursing students' critical thinking scores: a quantitative study. Sullivan-Mann J, et al. (2009). Newborn and Infant Nursing Reviews. Simulation has become the innovative method of incorporating clinical and theoretical knowledge and experiences for nursing students. It is essential for educators to offer strategies that develop students' critical-thinking abilities. This experiment (N=53) used the Health Sciences Reasoning Test ( HSRT ) to determine if critical-thinking scores improved in the associate degree in nursing student after exposure to multiple clinical simulation scenarios. Control participants received two simulation scenarios, and experimental participants received five scenarios. As predicted, experimental participants showed a greater increase in critical-thinking scores, demonstrating that simulation can enhance the quality of nursing education providing guaranteed clinical experiences that increase critical-thinking skills, as well as providing high-tech training for nursing students. Implications for nursing education and further research are discussed.

Problem-Solving Discussion Forums

  • Analyzing critical thinking skills using online discussion forums and CCTST. Jacob SM. (2012) Swinburne University of Technology Malaysia, Procedia - Social and Behavioral Sciences. This research study examined the progress of critical thinking (CT) skills in a first year mathematics unit for two batches of students. Problem solving sessions in engineering mathematics were activated through two online Discussion Forums (DFs) and the student postings analysed for CT skills. Changes in participants’ general CT Skills were investigated using the California Critical Thinking Skills Test ( CCTST ) instrument, before and after participation in the DFs. The CT skills identified through the postings from the DFs showed slight improvement over the semester. The general CT skills, as measured by the CCTST showed a progress in the total and the component scores. The study proved that online technology could facilitate development of CT skills in a slow yet steady manner, under the efficient supervision of the instructor.

Peer Led Team Learning

  • Peer led team learning in introductory biology: effects on peer leader critical thinking skills. Snyder JJ, Wiles JR. Departments of Biology and Science Teaching, Syracuse University, PLoS One (2015).  This study evaluated hypothesized effects of the Peer-Led Team Learning (PLTL) instructional model on undergraduate peer leaders' critical thinking skills. This investigation also explored peer leaders' perceptions of their critical thinking skills. A quasi-experimental pre-test/post-test with control group design was used to determine critical thinking gains in PLTL/non-PLTL groups. Critical thinking was assessed using the California Critical Thinking Skills Test ( CCTST ) among participants who had previously completed and been successful in a mixed-majors introductory biology course at a large, private research university in the American Northeast. Qualitative data from open-ended questionnaires confirmed that factors thought to improve critical thinking skills such as interaction with peers, problem solving, and discussion were perceived by participants to have an impact on critical thinking gains. However, no significant quantitative differences in peer leaders' critical thinking skills were found between pre- and post-experience CCTST measurements or between experimental and control groups.

Reflective Blogs

  • Reflective blogs in clinical education to promote critical thinking in dental hygiene students. Welmore AO, et al. (2010) Journal of Dental Education. A two-group design (intervention, n=28, and control, n=30) assessed first-year dental hygiene students using pre-and post-Health Science Reasoning Test ( HSRT ) scores to evaluate the effect of reflective blogging on critical thinking skills. A reflective blog rubric, based on Mezirow's levels of reflection, determined if reflective blogging increased the level of reflection for dental hygiene students. The results suggest within this nonprobability sample that reflective blogging did not produce a significant change in students' HSRT scores (p>0.05). However, analyses of reflective blog rubric scores demonstrated statistically significant improvements (p<0.05) in students' levels of reflection. Furthermore, data analysis revealed a correlation (p<0.05) between HSRT subscale scores and the element of reflection scores for the intervention group. This study addressed needs of the dental and dental hygiene education community by examining the use of blogs, an emerging technology, as a tool for reflecting on clinical experiences and, in turn, for promoting critical thinking.

 Problem-Based Learning

  • The effectiveness of problem-based learning on development of nursing students' critical thinking: a systematic review and meta-analysis. Kong LN, et al. Department of Infectious Diseases, The First Affiliated Hospital and Nursing College of Chongqing Medical University, PR China. Int J Nurs Stud. (2014). The objective of this systematic review and meta-analysis was to estimate the effectiveness of problem-based learning in developing nursing students' critical thinking. Nine articles representing eight randomized controlled trials were included in the meta-analysis. Most studies were of low risk of bias. The pooled effect size showed problem-based learning was able to improve nursing students' critical thinking (overall critical thinking scores SMD=0.33, 95%CI=0.13-0.52, P=0.0009), compared with traditional lectures.

Case Studies and Concept Maps

  • Case studies combined with or without concept maps improve critical thinking in hospital-based nurses: a randomized-controlled trial. Huang YC, et al. Taitung Hospital Taiwan, ROC. International Journal of Nursing Studies. (2012). This study was a randomized control trial to evaluate the effects of a 16 week program of case studies alone (CS)(n=67) or combined with concept maps (CSCM)(n=67), on improving CT in clinical nurses. After the programs, there were significant differences between the two groups in the critical thinking skills ( CCTST ) of analysis, evaluation, inference, deduction, and induction. There was also an overall significant difference, and a significant difference in the CCTDI scores, specifically the disposition of open-mindedness.

Reflective Writing

  • The effect of reflective writing interventions on the critical thinking skills and dispositions of baccalaureate nursing students. Naber J, Wyatt TH, Nurse Educ Today. (2014). This study used an experimental, pretest-posttest design. The convenience sample included 70 fourth-semester students in baccalaureate nursing programs. Randomly assigned control and experimental groups completed the California Critical Thinking Skills Test ( CCTST ) and the California Critical Thinking Dispositions Inventory Test (CCTDI). The experimental group completed six reflective writing assignments. Both groups completed the two tests again. Results showed that the experimental group had a significant increase (p=0.03) on the truthseeking subscale of the CCTDI when compared to the control group. The experimental group's scores increased on four CCTST subscales and were higher than the control group's on three CCTST subscales. If future studies, testing over longer periods of time, show significant increases in critical thinking, those interventions could be incorporated into nursing curriculum and change the way nurse educators evaluate students.

Socratic Questioning

  • Quantitative Assessment of a Critical Thinking Skills Intervention within an Accelerated MBA Capstone Course (Dissertation Wilmington University). Morlino, K. (2012).The need for graduate business education, and in particular Masters of Business Administration (MBA) programs, to foster the development of the critical thinking skills of its graduates has increased. Employers of MBA graduates desire sound critical thinking and decision making skills in their MBA hires. An increasing body of academic literature questions the effectiveness of MBA curricula and teaching pedagogy in developing students' critical thinking skills. This study measured the impact of a pedagogical intervention comprised of an application of Bloom's taxonomy and Socratic questioning pedagogy to a series of case studies within the MBA program capstone course. The Business Critical Thinking Skills Test ( BCTST ), a standardized assessment of critical thinking skills, was used to measure the sampled students' critical thinking skills. A pre- and post-test research design compared the BCTST assessment scores of an intervention group exposed to the intervention compared to that of a control group not exposed to the intervention. Results indicated that the critical thinking skills of the full intervention sample did not improve but did improve for an intervention sub-sample with BCTST post-test outliers removed. A statistically significant correlation between undergraduate GPA and students' BCTST scores was determined. Correlations between student age and undergraduate mathematics record and students' BCTST scores were not statistically significant.

Case Studies and Presentations

  • Externalizing the Critical Thinking in Knowledge Development and Clinical Judgment. Facione, NC, Facione, PA, (1996). Nursing Outlook. Critical thinking, defined as purposeful, self-regulatory judgment, is centrally evident in nursing knowledge development and expert clinical judgment. A holistic critical thinking scoring rubric, a framework for critical thinking individual and group presentations, and a case study strategy for training and nurturing critical thinking in students illustrate that the critical thinking in nursing knowledge development and clinical judgment can be externalized, taught, modeled, and measured. The approaches suggested here can be adapted to other professional practice programs and fields.

Good thinking is in demand. Download Critical Thinking Insight from your app store today:

Contact Us

* Required fields

Insight Assessment will not share your data with anyone. Click here to view our privacy statement.

[ Hide ]

Contact Us

* Required fields

Insight Assessment will not share your data with anyone. Click here to view our privacy statement.


C
O
N
T
A
C
T

U
S