Home / BLOG

BLOG

Red pencil next to paper with boxes to check

Rubrics are rating forms designed to capture evidence of a particular quality or construct. The quality of the measure obtained from a rubric depends on how well it is designed. If the rubric is poorly designed, the rating is confounded or inaccurate. The quality of the rubric is also affected by the skill of the rater using the rubric.  When using a rubric it is necessary to train and calibrate the raters to use the rubric well to assure that ratings are accurate and consistent across all raters. Ratings using rubrics cannot be benchmarked against national comparison groups or compared to other ratings made by other rater groups.

Rubrics are a popular approach when the goal is largely developmental; rubrics are good pedagogical tools.   Issues arise, however, when rubrics are used for summative assessment.

To read more about measuring critical thinking with rubrics

Graphic lists Academic uses for Insight Assessment student critical thinking skills and mindset data including admissions, accreditation, documenting student learning, quality improvement initiatives & Institutional effectiveness

It is increasingly important for educational organizations to document achievement of students learning outcomes. Many of Insight Assessment’s customers use one or more of our test instruments to gather the detailed objective evidence required to refine curricular approaches and to demonstrate educational success for accrediting bodies. Effective assessment programs first identify goals for the attainment of critical thinking skills and habits of mind learning outcomes and then create a plan to assess their achievement.

To read more about documenting achievement of evidence based student learning outcomes .

Custom Questions

Custom questions enable test administrators to better organize their results.  Custom questions are used to gather the classification and demographic metrics needed to support analysis and grouping of test results. 

Insight Assessment's testing system enables clients to create up to ten custom demographic items.  Response options include selecting choices provided by the client or free response. There are also five default demographic items which clients may elect to eliminate or retain. Here are some frequently asked questions about getting the most out of our custom question option:

1. What kinds of questions can I ask the people I’m testing using the custom demographics items?

Just about anything that it is legal to ask.  For example, you can ask about educational background, employment background, which department or work group a person is a member of, or how much time a person spends on a given activity.  If you know the choices you want test takers to select from, you can supply them. If you want the test takers to enter a free text response, that works too.

2. What are some examples of custom demographic items?

Here are four examples.  You are able to create your own custom demographic items to best fit with your assessment plans.

  •  Type your student ID number here:  _______________
  •  What is your highest level of formal education?
    • Advanced degree
    • Baccalaureate degree
    • Associates degree
    • High School Diploma or GED
    • Eighth Grade
    • Fourth Grade
    • None
  • Which professor assigned you to take this test?
    • Adams
    • Gomez
    • McDonnell
    • Smith
  • Which Division of our Company do you work in?
    • Accounting
    • Customer Relations
    • Human Resources
    • Marketing
    • Physical Plant & Security
    • Research & Development
    • Upper Management

3. I want to do a pretest and posttest analysis , but anonymity is important.  How can I set that up in the IA testing system?

There is a very straightforward procedure using the custom demographics option. First create an identifier that will link a given person’s pretest with that person’s posttest.  The identifier can be a four digit pin number, for example, that you create. Assign each individual their unique identifier.  You will then ask them to insert that unique identifier as a free response answer to a custom question.  Each person will do that when they login to take their pretest and again when they login to take their posttest. When you download test results in spreadsheet form, use that code to sort the data and, thereby, associate each person’s pretest and posttest. The custom demographic item you would create might look like this:

                Write the code number you were assigned here __________________

4. Does Insight Assessment supply any demographic items automatically?

Yes, there are five optional default demographic questions. Clients can elect to eliminate or retain any or all of these five.  The first three are free response items, which enable the responder to enter anything they wish whether it is accurate or not.  The last two on the list provide responders with the option of declining to provide the requested information.  

  • Name
  • Email address
  • Age
  • Gender
  • Ethnicity 

5. How can I learn more about creating custom demographic items or eliminating one or more of the default demographic items?

Easy, just contact Insight Assessment. We are happy to assist clients in getting the most helpful information possible when using our testing instruments and our powerfully versatile testing system.

The words Better Results on blue graph showing an upward trend cause by better decision-making

Measuring Changes and Improvement in Thinking Skills

Thinking skill matters; it is the springboard for problem-solving, decision-making and innovation. Building strong employee critical thinking skills is an important investment in the workforce of the future.  Employees who think well are more productive, dependable and creative. 

Performance-based funding, however, is a fact of life for HR managers and training specialists. You must constantly prove that your training and development methods, curriculum or instruction are working. 

To prove the value of your initiatives, objective, validated criterion-based evidence will be required to demonstrate success.  

The best way is to provide evidence of changes in your targeted individual and/or group thinking skills.

Effective training should result in measurable improvements. Thinking should be assessed before the training starts and again after the training is completed. It’s important to document the participants’ baseline as early as possible in the project.  Pre-training assessment is best. This will help faculty and trainers to learn about strengths as well as areas for improvement.  Then critical thinking skills should be assessed again, post-training, to measure the changes in thinking skills resulting from the training program. 

Use pre and post testing to tell if training is working 

Pre and post testing is an assessment model designed to examine the change in overall critical thinking skills or mindset attributes in a group of test-takers.  Comparison of individual or group results will highlight areas of improvement. Identification of areas that still need improvement can be targeted in subsequent training or mentoring.

For business, the pretest might be done before an employee training program is begun. The post-test could be set for weeks or months after the program has been completed. It's reasonable to post-test as soon as a few weeks after a focused training program in critical thinking, but most often a post-test is gathered months, or even years, after the pretest to measure program improvement.  More about pre and post testing

Report graphic showing pre and post test results for INSIGHT Business Professional

Data to assess impact of employee development program 

Only the highest quality assessment tools can deliver statistical evidence of value added to groups and individuals.

INSIGHT validated business test instruments deliver quantifiable metrics for the essential skills of critical thinking and quantitative reasoning. INSIGHT assessment tools are based on more than 30 years of experience in measuring thinking and reasoning skills.  These tools were specifically designed to provide businesses with the metrics they need to create teams of people able to learn their jobs quickly and to perform successfully in situations requiring problem-solving. 

INSIGHT assessments meet the highest standards of validity and reliability.

INSIGHT measures and reports on core business skills (such as analyzing problems; evaluating options; managing consequences; managing uncertainty; managing numbers) plus associated essential business thinking mindsets (such as job commitment; work ethic; honesty and focus).

INSIGHT Development Program is designed to help employers build critical thinking in teams as well as in individuals.  The program includes assessment and critical thinking training modules that can be integrated into your training programs or can be implemented as a new initiative. This proven solution will address your talent enrichment challenges.

Contact us to make sure you can document individual and/or group progress in attaining your learning goals.

For further information:

Insight Assessment HSRT Difference Scores in pretest posttest critical thinking skills assessments

Critical thinking can be analyzed, measured and improved. The more we are supported and challenged to exercise our core critical thinking skills, the stronger these skills become. 

In order to obtain high quality data about the effectiveness of a critical thinking program, you will need obtain baseline data by assessing thinking skills both before and after the training. Scores on Insight Assessment test instruments will improve with the effective training of reasoning skills.

There are a number of ways to document quality improvement in critical thinking skills for your institution or organization:

  • Group Difference Scores: One method is compare the difference in the mean scores reported for your group before and after a training program or course.  This method compares the scores as a whole rather than looks at changes within the individual. The expected positive group difference change from pretest to posttest will not tell you how many of the individuals have improved, nor will it describe the range of improvement made by each individual.
  • Individual Difference Scores: To learn more about the proportion of individuals who have improved and the degree to which they have improved, the best method is to calculate the difference between the pretest and posttest scores for each person you have assessed. Individuals will have made different progress as the result of an educational offering aimed at building critical thinking skills. Some individuals may not have improved their critical thinking skills during the interim time period. If the difference score is at or near zero, they have shown no effect as a result of the educational training program.  The OVERALL Score for Insight Assessment test instruments is the score that is best used to calculate difference scores.
  • Comparison Percentile Difference Scores: A third option is to calculate difference scores for the comparison percentiles reported for each individual at pretest and posttest. Example: Improving your group’s overall percentile score from 52nd to 58th percentile demonstrates a significant overall gain in critical thinking scores. Insight Assessment offers a wide range of options of external criterion groups for. percentile score comparisons .

Individual difference scores provide the most information on the effectiveness of training techniques. In any given sample when individuals test twice, individual difference scores will demonstrate individual gains in critical thinking scores. For example, the demonstration of a two-point gain in CCTST OVERALL Score from pretest to posttest is associated with becoming more metacognitive about the reasoning process and with observable evidence of efforts to actively build reasoning skills. Large individual difference scores are observed in most pretest posttest studies for at least some individuals.

Longitudinal studies of individual critical thinking gains over time are rare. In one longitudinal study of practicing U.S. military nurses, demonstrated gains at posttest were retained in three consecutive cohorts at one-year post completion of the specified training program. Additional gains were also seen in many of these practicing professionals, suggesting that individuals who have become more reflective about, and evaluative of, their reasoning process continue to make gains.  

Scores or percentiles that drop significantly at posttest are very rare, as critical thinking skills are not lost over a short period of time in the absence of cognitive injury, chemical impairment, or a failure to give or to be able to give best effort. 

The fact that Insight Assessment test instruments measure only critical thinking and not content knowledge makes it possible to use these instruments as a pretest and posttest to measure improvement in critical thinking that occurs during any educational program or staff development exercise.

Contact Us to discuss how the data reported on Insight Assessment test instruments will help you meet your assessment program goals.

  • It’s difficult to develop an internal program for improving thinking skills. INSIGHT Development Program includes assessment and critical thinking training modules that can be integrated into your training programs or can be implemented as new initiative. Contact us to learn more.

Important Critical Thinking Skills in the Workplace

Are you prepared for the upcoming wave of resumes from college seniors? So much enthusiasm, so little experience. Completing a four year degree program doesn’t mean that they are prepared to succeed in the realities of your workplace.

Yellow sign reading SUCCESS against a bright blue sky

Employer expectations are increasing

Now, even entry level professionals must be able to identify and resolve anomalies. They'll have to think their way through technological barricades and job challenges of all kinds. What you need is reliable data on the thinking skills of your new recruits.

Wanted: critical thinking and data analysis skills

You need to find talented professionals with the reasoning skills and mindset to drive current systems to the next level, the eagerness for relentless problem solving, and an eye to seize opportunities while managing risks.  

Imagine the positive impact on your bottom line if you could be certain that:

  • problems were correctly diagnosed and risks were effectively managed
  • employees were committed to excellence and to the success of the organization
  • opportunities were anticipated and captured at all levels
  • decisions were driven by data relevant to company goals
  • employees regularly contributed practical and affordable suggestions
INSIGHT Business Professional

There is a better way to screen the best job candidates

Save time and money by prescreening for the most important predictors of on the job success: the capacity and desire to engage complex problems using sound critical thinking and quantitative reasoning. INSIGHT Business Professional delivers objective analysis of the candidate’s strengths and weaknesses in 15 core business skills and mindset attributes.

Contact us to preview INSIGHT Business Professional or to learn more about additional INSIGHT tools.

Study Group working around a table

Many two year academic institutions choose to measure what really matters: good thinking. 

Community and technical colleges, certificate programs and other associate degree programs realize that student success is based on the strength of individual critical thinking skills and personal mindset attributes.  They depend on Insight Assessment tools to provide the highest quality, cost effective critical thinking metrics needed to support admissions, student retention and student success.

The measurement of the critical thinking skills and mindset of incoming students, individually and as a group, will provide vital data for tailoring student success services to the needs of specific students. Exit data on student critical thinking is valuable for other educational purposes such as program evaluation, accreditation or learning outcomes assessment.

In today’s world, nothing is more important than strength in problem analysis and strong decision-making skills. Critical thinking assessment is invaluable whether your project involves selective admissions to STEM and Heath Sciences programs, quality in admissions to business education, student retention initiatives, or the documenting of learning outcomes. Here are some two-year college success stories:

  • With an abundance of applicants for the Engineering Technology degree program, the program faculty found that the Test of Everyday Reasoning (TER)   and the Adult Level California Measure of Mental Motivation (CM3 III) identified their most qualified applicants. They used national percentile benchmarks to document the strength of their program graduates.
  • An urban college linked their SACS QEP initiative to a Statewide concern for improving the percentage of students who successfully complete their associate degree level career training.  The college’s QEP project using the TER and College Student Success (CSS) assessments to identify students at risk in their first semester, assisting their teaching and learning professionals to tailor the college’s support services.
  • Directors of the Nursing and the Physical Therapy Assistant programs selected the Health Sciences Reasoning Test – Associate Degree (HSRT-AD)  and the California Critical Thinking Disposition Inventory (CCTDI) to augment their admissions screening process. Strong scores on skills and mindset attributes long acknowledged as desired qualities and abilities in these clinical disciplines improved degree completion and licensure rates.
  •  A college transition director at another community college uses the College Student Success (CSS) instrument to guide students through a smooth transfer to their Bachelors completion program. The individual reports generated by the CSS give the advisor and the student a clear picture of the transfer student’s critical thinking skill set, motivation to learn, drive to succeed and much.

As the leader in reasoning assessment worldwide, our full service solution includes presentation-ready group statistics and graphics , scores spreadsheets, optional individual student reports, multiple language testing options, client customizable demographic question options, and same-day score reporting with 24/7 online, mobile or desktop testing , plus educational discounts and much more.  Visit our website for details about these services and to access our Resources bank of critical thinking teaching support materials.

Call us at 650-697-5628 or use our website’s “Contact us” link for a free quote for your project. Talk with an Insight Assessment representative about how other 2 year programs have added critical thinking assessment to their admissions and early cohort assessment procedures and how they document students’ strength in critical thinking as a criterion for receiving performance-based funding.

Follow our blog, Thinking INSIGHT , or RSS feed for more stories of assessment projects that help educators at all levels achieve their goals.              

Building under construction with text: How to BUILD an Employee Development Program

Maximize your investment in training

Billions of dollars are invested annually in employee development and training. This is seen in dismal retention rates, poor engagement, stagnant leadership development and meager customer service.

Investing in employee development takes planning, but it doesn’t have to be hard. Some key tactics:

1.  Start with goals for your learning and development strategy

Take the time to decide which employee strengths and weaknesses most impact your company potential. Your continuing professional development should, then, address how to improve these strengths and reduce those weaknesses.

Are bad decisions and missed opportunities the issue? Are there specific skills or motivational attributes that hold your company back? Is poor motivation derailing corporate growth?  Do team members have the mental discipline and mindset to drive new ideas from concept to completion?

Consider the benefits of increasing the quality of decision-making. The foundation for success is your employees knowing what actions to take.

Part of the skill set for decision-making includes being able to analyze data.  Due to the increasing use of technology, there is a skill gap for many employees.  Per a 2018 survey by edX, the online learning portal for Harvard and MIT, “39% of people reported feeling less than proficient in data skills with an emphasis on analytics and computer science.”  

Prioritizing the development of strong decision-making skills and motivation will deliver ongoing improving throughout an organization.

2. Align and integrate business goals and training

Once you’ve determined your goals, the next step is to identify the necessary skills, knowledge and competencies to deliver the desired results. 

If you’ve decided that good decision-making is critical to the success of your group/organization, then prioritize specific components of critical thinking.  This will enable enable you to carefully select a training program that focuses on those skills. To boost the analytic skills used in decision-making, for example, you should select a program that develops Quantitative Reasoning. 

To optimize achievement of your program goals, it is necessary to select the right L&D and assessment tools.  

Bonus Hint: Use an assessment to learn from your best.   Start by measuring the skills and mindsets of your high performers; discover exactly what makes them excel. Once you've determined the core reasoning characteristics, you can establish benchmarks for hiring and talent development. This information will help you define the reliable metrics needed to document achievement in improving the targeted skills.

More about how to choose a thinking assessment

3.  Prioritize relevant data for employee development

High integrity data is the foundation of employee development plans.  You will need objective, affordable metrics on how prospective employees will perform when faced with new problems in high stakes and time-limited situations.

Reliable data about the strengths and weaknesses in critical business reasoning skills and mindset attributes are valuable when applied to:  

4.  When training is completed, evaluate ROI

So did employees learn anything they will apply to doing their job better?

Build in an evaluation of the impact of your plans and initiatives. There’s no substitute for high quality objective and validated data.  You will need statistical evidence of the value added by your training and development program.

Funding always much easier when you have a demonstrated track record.  It’s important to measure changes in individual and/or group skills. Do employees demonstrate improvement? Comparing pre and post training data confirms training and program effectiveness. The difference between the pre and post test results will demonstrate the effectiveness of your curriculum.   

5.  Use it or lose it (time, money and motivation)! 

You must use the data to get value. An employee development program should help a business create an integrated approach to thinking improvement.  Both individual and group report data can be applied throughout your operations in areas such as prescreening, hiring, personal development, talent development.

6.  Be proactive.

Ultimately, employee development plans should assist in maximizing your most valuable resource--your employees, managers and leadership.  Be proactive, but make sure your actions are governed by data and metrics.

Logo for INSIGHT Developmebt Program, an online critical thinking training program

How to build a better employee development program?

It’s difficult to develop an effective internal program for improving thinking skills. Insight Assessment delivers the entire turn-key package: training and objective assessment tools.  

INSIGHT Development Program is a suite of thinking skills and mindset enrichment modules that focus on improving the quality of employee decision-making. Designed to be used as an independent study by employees, it can also be incorporated into existing training programs. INSIGHT Development Program is online, confidential, and available at staff member convenience.

INSIGHT Development Program features a unique integration of assessment and development. An analysis of skills and mindset attributes using our industry leading assessments is included for each trainee. Individual and group report provide comprehensive analysis of the strengths and weaknesses on 5 critical thinking skills and 10 essential mindset attributes that are characteristic of contributing members of a team. 

These INSIGHT tools have a proven record of reliability and meet statistical requirements of validity.  Assessments are administered via our secure, encrypted online interface available 24-7, easily administered at single or multiple sites, at test centers, to groups & to individuals.  Results will be delivered to you quickly and confidentially.

Contact us to preview this powerful tool that can support new employee on-boarding, department level enrichment programs, and company-wide training initiatives.

 

Boy Thinking in Class

K-12 schools use EDUCATE INSIGHT assessments to assess critical thinking because they need objective information to assist them in identifying applicants who are ready, willing, and able to think and to learn.

Assessing Critical thinking for Selective Admissions

Today the pressure on selective schools is immense, as parents, eager to provide the best possible start in life for their children, strive to have them admitted to the best schools possible.  But a tsunami of applications can actually present problems for selective K-12 schools.   Schools know that it is a disservice to admit a child who is not prepared to succeed academically. And schools know that their admissions processes need to be as fair, objective, and thorough as possible. 

To address these twin problems, selective schools are adding critical thinking mindset and critical thinking skills measures to their applications package.

EDUCATE INSIGHT provides a comprehensive thinking assessment program with instruments calibrated for students from kindergarten to high school.

EDUCATE INSIGHT K-12 Thinking Mindset 

  • As a first screen on admissions, schools use EDUCATE INSIGHT thinking mindset tools to assess a child’s expectations and attitudes toward using thinking to learn things and to solve problems. Parents and teachers know that a positive desire to engage in thinking and learning is very important to a child’s educational success.  By contrast, students who are unwilling to apply themselves are not only difficult to teach, they take valuable places away from more deserving applicants. Used as part of an admissions package, EDUCATE INSIGHT thinking mindset assessments provide objective information about the applicant’s mental focus, learning orientation, and approach to problem solving. 

EDUCATE INSIGHT K-12 Reasoning Skills 

  • Children with stronger critical thinking skills have a huge advantage at every educational level. That is why schools use the EDUCATE INSIGHT reasoning skills series as part of the second, more refined, admissions screening to identify strengths and weaknesses in core thinking skills.

Critical thinking skills empower learning.  By engaging their skills children learn important content and, at the same time, they exercise and strengthen those skills.

  • Just like learning a sport or a musical instrument, critical thinking skills grow stronger when guided by the teacher’s watchful coaching. Learning is so much more than memorizing facts.  Ultimately education means learning to think well. Through education, as contrasted with rote memorization, a child learns to be analytical, to draw reasonable inferences, and to explain why and how things work. Through learning to think, the child becomes able to correctly evaluate the credibility and the quality of reasons and information.  Whether educators are using a Montessori approach or some other method to ask “Why?” and “How?” questions, engagement in thinking is the key to learning.

EDUCATE INSIGHT assessments are calibrated for use with children at different grade levels, are standardized, objective measures of critical thinking skills. Using scenarios that are familiar and age-appropriate, these tests invite children to apply their reasoning skills in order to analyze information, to draw inferences, to evaluate options and explanations. 

It is important to note that no specific test prep is needed before taking a critical thinking skills test beyond general education focused on the development of critical thinking skills.  A good critical thinking test is not going to be a test of factual knowledge about critical thinking. Memorizing information and definitions is not the key. A good reasoning test will engage you in using your thinking skills and mindset because critical thinking is a process. 

Bottom Line: selective K-12 schools incorporate EDUCATE INSIGHT into their application process because they know strong critical thinking predicts academic and professional success. Making the measurement of critical thinking at program entry strengthens comprehensive selective admission and placement programs.

Contact us today to learn more about how you can use the EDUCATE INSIGHT K-12 series to measure what matters most – critical thinking.

Follow our blog, Thinking INSIGHT , for resources and further discussion of the measurement of thinking skills and mindset.

Critical thinking is essential for workplace decision making.

Is there immediate value from an initiative to strengthen critical thinking in new hires and/or current employees?

Just consider these examples of how critical thinking is applied on the job:

Woman executive thinking as she looks at the words: 18 examples of using critical thinking

Business Critical Thinking in Action:

  • analyzing and explaining risks
  • designing & implementing initiatives
  • allocating resources
  • anticipating & preventing errors
  • analyzing markets
  • using & managing information systems
  • assessing opportunities & competition
  • evaluating the effects of initiatives
  • identifying & analyzing emergent concerns
  • developing data security protocols
  • evolving long term goals and plans
  • communicating effectively with team members
  • evaluating customer service
  • anticipating the implications of actions taken
  • hiring and promoting leaders
  • analyzing and resolving personnel issues
  • explaining policy and procedures
  • designing & evaluating departmental reports

The list of business applications for critical thinking goes on and on, almost indefinitely. It is particularly relevant in high-stakes workplace settings. 

Employees in healthcare services, engineering and law firms and other operationally complex organizations are required to engage unexpected problems and make time sensitive judgments.

Business owners cannot afford to be mistaken about the critical thinking capabilities of employees.  In cases where error rates are already in question, difficult problems must be addressed.  Poor judgments could lead to irreparable damage and even death. The cost is too high.

It’s easy to see why critical thinking is important. Building critical thinking strength throughout a business is an investment that pays off every day.

3 team members thinking together as they look at a computer monitor

Using INSIGHT business assessments and INSIGHT Development Program 

Insight Assessment strives to improve the quality of decision-making in business.

Our critical thinking assessments measure the thinking skills and mindset required for success in the workplace. The INSIGHT suite  includes assessments  tailored for Business, Health, Defense, First Responder, Educator, Government/Law, and Science/Engineering uses.  INSIGHT objective, online assessments provide  individual and group metrics on the strengths and weaknesses of key aspects of decision-making.

INSIGHT Development Program is an integrated solution that incorporates online training modules and industry leading assessment tools.  It can be implemented to supplement existing employee improvement plans or as a stand-alone development program.

Contact us to begin your strategic investment to improve the quality of decision-making in your organization. Follow our blog, Thinking INSIGHT , for resources and further discussion of the measurement of thinking skills and mindset.

Good thinking is in demand. Download Critical Thinking Insight from your app store today:

Contact Us

* Required fields

Insight Assessment will not share your data with anyone. Click here to view our privacy statement.

[ Hide ]

Contact Us

* Required fields

Insight Assessment will not share your data with anyone. Click here to view our privacy statement.


C
O
N
T
A
C
T

U
S