Home / Resources / Research Findings / Learning Outcomes Assessment

Learning Outcomes Assessment

A collection of studies reporting the outcome of assessment projects. Papers from general education projects, STEM education studies, health sciences training projects and business education curriculum evaluation projects are included.

Overview of scholarly literature about critical thinking, its instruction, and its assessment

  • Don’t Reinvent the Critical Thinking Wheel: What Scholarly Literature Tells Us about Critical Thinking Instruction. Eigenauer JD, (2017) National Institute of Staff and Organizational Development.  As colleges face accreditation requirements or as departments undergo program review, they commonly seek to integrate formal statements about critical thinking into their documentation. It is not uncommon for these schools to be unsure about how to do so. Frequently, faculty meet to seek consensus on the meaning of critical thinking, as well as how it should be taught and measured. Because faculty have discipline-specific expertise, they may not be familiar with the literature on critical thinking that has emerged from the fields of cognitive science, educational, developmental, and social psychology, and even neuroscience. This can result in well-intentioned but sometimes misinformed efforts. The goal of this article is to present an overview of what scholarly literature tells us about critical thinking, its instruction, and its assessment in order to assist schools in aligning their efforts in this important field with the most current scientific knowledge and best pedagogical practices. The scholarly literature on critical thinking, its instruction, and its assessment is quite large. While the conclusions that various researchers draw are somewhat diverse, one finding predominates. It is that measurable gains in critical thinking among college students are obtainable by implementing proven methods and best practices, which always include explicit instruction in critical thinking.  Executive Summary of the Delphi Report

Numeracy in Middle School Students

  • Assessing Numeracy in the Upper Elementary and Middle School Years. Gittens CA. NUMERACY: Advancing Education in Quantitative Literacy (2015). Numeracy is the ability or tendency to reason critically about quantitative information. The preponderance of published research on numeracy examines this construct among either pre-K or early elementary samples, students with developmental challenges, or is focused on post-secondary and adult cohorts. The numeracy skills of upper-elementary and middle school students are less well documented and understood, most notably because of the lack of valid instruments that are developmentally appropriate for the age range. The numeracy scale on the CCTST MIB and CCTST M25 critical thinking skills tests was used in upper-elementary and middle school students in this study. The scale was validated in a gender-balanced, racially and ethnically diverse sample of 3rd through 8th grade students (N=197) from a private, Catholic K-8 school in Santa Clara County, California. Construct validity was supported by strong, positive correlations with the three subscales of the Cognitive Abilities Test (CogAT) as well as the Iowa Test of Basic Skills Mathematics test, a standardized academic achievement domain assessment. A preliminary exploration of the critical-thinking dispositional correlates of numeracy suggests a positive relationship with students’ self-reported creative problem solving, diligence, systematicity, and fairmindedness.Contact us for further information about the CCTST MlB and CCTST M25.

Critical thinking assessment in Business Education

  •  Understanding the Nature and Determinants of Critical Thinking Among Senior Business Undergraduate Students Brown FW &  Bielinska-Kwapisz, (2015) Journal of Education for Business.The authors examine the dimensions and determinants of critical thinking skills, as measured by the California Critical Thinking Skills Test ( CCTST ) among graduating senior students enrolled in an Association to Advance Collegiate Schools of Business–accredited undergraduate business program. Utilizing explanatory variables, a methodology for predicting student cognitive development while in an academic program is described and was utilized to identify the percentage of students who were over- and underachieving expected developmental levels across fields of study in the focal group. Implications of the results and suggestions for further study are provided.

Direct training achieves results

  • Instructional Interventions Affecting Critical Thinking Skills and Dispositions: A Stage 1 Meta-Analysis. Abrami PC, et al., Review of Educational Research. (2008): “Critical thinking (CT), or the ability to engage in purposeful, self-regulatory judgment, is widely recognized as an important, even essential skill. This article describes an ongoing meta-analysis that summarizes available empirical evidence on the impact of instruction on the development and enhancement of critical thinking skills and dispositions.  We found 117 studies based on 20,698 participants, yielded 161 effects with an average effect size (g+) of 0.341 and a standard deviation of 061. Type of CT intervention and pedagogical grounding were substantially related to fluctuations in CT effects sizes, together accounting for 32% of the variance. These findings make it clear that improvement in students’ CT skills and dispositions cannot be a matter of implicit expectation.  As important as the development of CT skills is considered to be, educations must take steps to make CT objectives explicit in courses and also to include them in both pre-service and in-service training and faculty development.”

Curricular Effectiveness and Assessment

  • Measuring change in students' critical thinking ability: implications for health care education. Bartlett DJ, Cox PD. (2002). Journal of Allied Health. The purpose of this study was to determine the change in critical thinking (CT) dispositions and skills among physical therapy students (N=28) over academic and clinical portions of a year and to determine correlates of change in CT abilities. Participants completed the CCTDI and the CCTST before the academic year, after the academic year, and after their clinical placements. A one-way repeated measures analysis of variance was conducted on the subscales and total scores of the CCTDI and CCTST. Statistically significant improvements in all subscales and both total scores were obtained over the year. Greatest improvements on the CCTDI were observed for truth-seeking (p < 0.001) and CT self-confidence (p < 0.001). Greatest improvement on the CCTST was noted for deduction (p < 0.001).

Assessing Curriculum Innovation

  • Evaluation of critical thinking outcomes of a BSN program. Spelic SS, et al. (2001). Holistic Nursing Practice. Following a curriculum revision, which emphasized critical thinking, this paper compares entry and exit scores on the CCTST to study students' gains in critical thinking skills (N = 136). Results for students in each of the three program tracks demonstrated significantly improved (P < or = .05) CCTST scores on all subscales and total scores, with one exception. RN to BSN students' scores on the Analysis subscale approached but did not reach significance (P = .058).

Athletic Training Programs

  • Assessment of Critical Thinking Skills Among Undergraduate Athletic Training Students. German, N. (2008) Dissertation. Scores generated from this study provided a critical thinking baseline score for undergraduate athletic training students. Using a 95% confidence interval, results show that undergraduate athletic training students' CCTST scores are comparable to students in other allied health professions. This study also indicates that the CCTST is an appropriate measurement tool for athletic training. An analysis of variance revealed no significant difference between academic level and critical thinking scores (p = 0.05).

Growth in Mindset

  • Disposition toward critical thinking among occupational therapy students. Lederer JM. (2007). Amer J Occupational Therapy. The disposition, or internal motivation, to think critically strongly influences the development of critical thought. Students (N = 79) at three levels of education in one program were administered the CCTDI . Results indicated no differences in the disposition to think critically related to length of time spent in the program. Differences in the dispositions of open-mindedness and maturity of judgment were found between undergraduate and graduate students.

Aggregate Nursing Study (Foundational Study)

Assessing Curriculum Innovation

  • Assessing critical thinking in baccalaureate nursing students: a longitudinal study. Beckie TM et al. (2001). Holistic Nursing Practice. The purpose of this study was to evaluate the attainment of critical thinking skills of students before and after curriculum revision of a baccalaureate nursing program at the University of South Florida. The CCTST was used to measure the critical thinking ability at program entry, midpoint, and at exit. The sample consisted of three cohorts of students: cohort 1 (n = 55) was the baseline class before curriculum revision, whereas cohorts 2 (n = 55) and 3 (n = 73) were the first two classes to experience the revised curriculum. The results revealed that cohort 2 achieved significantly higher critical thinking scores than the baseline cohort. Cohort 2 also improved dramatically on all subscales from test 1 to test 3. However, cohort 3 failed to demonstrate improved critical thinking scores over time. Findings have implications (for study design) when measuring critical thinking.

Collaborative Assessment Project

  • Outcomes of ADN-BSN partnerships to increase baccalaureate prepared nurses. Sizemore MH, Robbins LK, Hoke MM, Billings DM. (2007). International Journal of Nursing Education Scholarship. To address a statewide need, a BSN Program and 3 ADN Programs formed a partnership to take BSN education to rural and medically underserved areas in New Mexico. This article describes the program planning, implementation, and evaluation using an adapted assessment framework with partnership principles as its foundation. Interactive television, internet education components, local clinical experiences, and distant nursing faculty liaisons were used. The nursing course sequence was completed by 101 of 102 students. Hall's Professionalism Scale, the CCTDI and the CCTST measured the increases found in professional socialization and critical thinking. Use of the adapted theoretical framework represented a strategic approach to developing a distance delivered nursing education program.

Evaluating a Training Program

  • Critical Thinking Skills during a Physical Therapist Professional Education Program. Vendrely A. (2005). Journal of Physical Therapy Education. The CCTST pretest was given to each incoming PT student cohort during orientation day prior to beginning coursework in a classroom on campus and the CCTST posttest was administered during the last week in the program. After completing all of the didactic and clinical experiences, the students returned to campus for 1 week to complete several standard evaluations, including the CCTST, which was administered and scored in the same manner as the pretest. There was a positive change in pretest CCTST-C scores (mean=19.32) when compared to posttest CCTST-C scores (mean=20.61); however, the magnitude of the change was not great enough to constitute statistical significance.

Assessing Skills and Mindset

  • Evaluation of critical thinking in a baccalaureate nursing program. McCarthy P, et al. (1999). Nursing Education. There were significant gains in critical thinking skills ( CCTST scores) from the sophomore year to the senior year. There also were significant gains in CT disposition between sophomores and seniors on the overall score for the CCTDI, truth-seeking, analyticity, self-confidence, and inquisitiveness.

Curricular Effectiveness in Prosthetics/Orthotics

  • A prospective study on the development of critical thinking skills for student prosthetists and orthotists in Hong Kong. Wong MS. (2007). Prosthet Orthot International. This study was to evaluate prospectively the development of critical thinking disposition of the student prosthetists and orthotists in Hong Kong. The results showed that there was significant improvement in 5 out of the 7 domains of the CCTDI , namely Truthseeking, Open-mindedness, Systematicity, Analyticity, Critical thinking self-confidence, Inquisitiveness and Maturity of judgment in 3 years' time. Further curriculum enhancements were suggested as the sum of all the domains was just slightly above the threshold of positive tendency.

Longitudinal  Study of Undergraduates Thinking Mindset

  • A Look across Four Years at the Disposition toward Critical Thinking Among Undergraduate Students. Giancarlo (AKA Gittens) CA, Facione PA. (1999). Journal of General Education. This article examines the critical thinking (CT) dispositions, as measured by the CCTDI of students at a four-year, private, liberal arts, comprehensive university. This paper follows up results first published in 1995. The present findings represent another snapshot of CT dispositions among students who participated in 1996 and during the original investigation in 1992. Longitudinal results about students tested as freshman in 1992 and again as seniors in 1996 are presented. Cross sectional results are reported as well. Questions explored include the relationship between the disposition toward critical thinking, as measured by the CCTDI, and students’ major, gender, class level, and grade point average.

Curriculum Evaluation

  • The assessment of graduating students' critical thinking skills and habits of mind challenges and rewards those who approach the task from a critical thinking perspective. Facione, NC, Facione, PA, (1996). Holistic Nursing Practice. This paper identifies and discusses issues in the design of authentic assessments of critical thinking as an educational outcome predictive of competent professional judgment in professional practice programs. The paper uses as its running example programs in nursing, but is applicable to programs in business, engineering, social work, teacher preparation, and other areas of professional practice.. Authentic assessment implies a multiple methods design which address the diverse contexts within which judgments must be made by professional nurses. Most important, it implies a concern for validity and reliability of measurement, selection of appropriate data-points, and attention to a number of logistical and practical concerns. Keywords: Outcomes assessment, critical thinking, portfolio assessment, testing, clinical judgment, professional judgment, nursing education.

Discussion of Assessment Methods

  • Assessment design issues for evaluating critical thinking in nursing. Facione NC, Facione PA. (1996). Holistic Nursing Practice.  This article identifies and discusses issues in the design of authentic assessments of critical thinking as an educational outcome predictive of competent professional judgment in nursing practice. Authentic assessment implies a multiple methods design that addresses the diverse contexts within which judgments must be made by professional nurses. Most important, it implies a concern for validity and reliability of measurement, selection of appropriate data points, and attention to a number of logistic and practical concerns.

Anxiety and Self-Esteem

  • Critical thinking, self-esteem, and state anxiety of nursing students. Suliman WA, Halabi J. (2007). Nurse Education Today. This study used a cross-sectional correlational design to explore critical thinking (CT) disposition in a convenience sample of first year (n=105) and fourth year (n=60) nursing students in Saudi Arabia. Of interest was the relationship between CT disposition ( CCTDI ), self-esteem (Rosenberg SE), and state anxiety (Spielberger SAI). CCTDI scores showed no serious deficiency with the exception of a weakness in CT confidence. Scores for Self-esteem scale were average, and scores on the SAI were relatively high. Beginning students reported significantly poorer CT self-confidence (t=-2.053, p=.04).

Curriculum Evaluation

  • Study of critical thinking skills in nursing students and nurses in Japan. Kawashima A, Petrini MA. (2004). Nurse Education Today. The purpose of this study was to measure the dimensions of critical thinking ( CCTDI scores) of nursing students at baccalaureate nursing program and registered nurses at general hospital in Japan. A convenience sample of three small groups: generic students (n=82) including freshmen and juniors; transfer students (n=16) at selected baccalaureate nursing program; and registered nurses (n=67) at selected general hospitals. A one-way ANOVA indicated that registered nurses scored lower than the two groups of baccalaureate students on the total score and several sub-scale scores. The outcomes of this study propose recommendations regarding curriculum review for Japanese nursing education and reflection on professional boundaries for Japanese nursing practice.

Measuring Post Graduates

  • Exploring critical thinking in critical care nursing education: a pilot study. Rogal SM, Young J. (2008). Journal of Continuing Education in Nursing. Critical care nurses process vast amounts of information and require well developed critical-thinking skills to make clinical decisions. Using a pretest posttest design, the critical-thinking skills of 31 postgraduate nurses in Western Australia were evaluated using the  CCTST . For the total group, mean critical-thinking scores improved slightly over time. The CCTST revealed a mean pretest score of 18.5 and a mean posttest score of 19.7, both of which were higher than the established norms for the test. Despite the small sample, the majority of the postgraduate nurses improved their critical-thinking skills during the course of their study.

Gains with Online Education

  • Critical Thinking in Online vs. Face-to-Face Higher Education. Derwin EB Media Psychology Review (2009). This study compares critical thinking skills for adult learners in online and face-to-face liberal studies classes at a university with locations in California and Washington (N=150). In a between subjects design, the study analyzed students’ score gains from pre- to post-tests on the California Critical Thinking Skills Test ( CCTST ). The study also compared students’ grades on critical thinking assignments required at the end of the course. Results showed that there were no significant differences between face-to-face and online learners for the CCTST score gains or the grades on the final assignments. Results are consistent with previous “no significant difference” studies. The research adds to the literature by specifically addressing outcomes in critical thinking. Future studies may benefit from selecting a variety of critical thinking measures and identifying characteristics required to demonstrate higher-level thinking skills.

Community and Technical College

  • A Study to Determine Whether The California Critical Thinking Skills Test will Discriminate Between the Critical Thinking Skills of First Semester Students and Fourth Semester Students in a Two year Community Technical College. Raykovich TF. (Dissertation University of Wisconsin 2000). Students were selected randomly for participation (N=103). The results of this study indicate the average mean test score of Nicolet College students enrolled in their fourth semester was significantly higher on the California Critical Thinking Skills Test ( CCTST ) than the average mean test score earned by first semester Nicolet College students. Additionally, there was a statistically significant increase in the mean test scores of fourth semester students on the Analysis, Evaluation, Deductive and Inductive sub-scale areas of the CCTST.

Good thinking is in demand. Download Critical Thinking Insight from your app store today:

Contact Us

* Required fields

Insight Assessment will not share your data with anyone. Click here to view our privacy statement.

[ Hide ]

Contact Us

* Required fields

Insight Assessment will not share your data with anyone. Click here to view our privacy statement.


C
O
N
T
A
C
T

U
S