Non-Cognitive Competencies in Graduate Admissions: Enhancing Validity and Diversity

By  - 
Casey W. Miller, PhD
Casey W. Miller, PhD

Although the importance of diversity in STEM graduate education is rising in the U.S. — notably with commentary from National Science Foundation leadership — the standard graduate admissions process is lagging behind the rhetoric.

Julie R. Posselt’s work at the University of Michigan has shown that most graduate admissions requirements include the GRE exam, and many admissions committees require a minimum score for admission.

Unfortunately, these tests reveal strong race and gender differences. For example, the percentage of women who make it over a 64th percentile cutoff is only 38 percent, while it is 58 for men; 62 percent of Asian Americans and 53 percent of whites pass this cut, but only 24 percent of all underrepresented minorities (African Americans, Hispanics, and Native Americans) meet or exceed this percentile. These differences can significantly affect the representation in STEM graduate programs. For instance, relative to the demographics of test takers, a 64th percentile cutoff shrinks the number of women by 25 percent and inflates the percentage of men by 14; the number of Asian Americans and whites are inflated by 22 percent and 6 percent, respectively, while the representation of underrepresented minorities is reduced by 47 percent.   Consequently, admissions protocols using minimum acceptable scores will disproportionately filter out underrepresented minorities and women.

It is important to note that these score differences do not depend much on undergraduate GPA or undergraduate major: Students with GPAs in the A range show the same race and gender gaps. In fact, these score differences exist on all standardized exams down to fourth grade math achievement tests. The Educational Testing Service (ETS), which administers the GRE, suggests that the score differences are rooted in educational opportunity and access early in life, which raises the question: Should the zip code of your grammar school determine whether you are able to attend graduate school?

Ironically, the admissions process that indiscriminately filters women and underrepresented minorities out of the applicant pool doesn’t work better than random chance in identifying students who will actually earn the PhD. Indeed, the Council of Graduate Schools has shown that the seven-year PhD completion rate in STEM fields is only 52 percent.

Fortunately, there is hope for simultaneously addressing both of these issues. Decades of management and industrial-organizational psychology research have shown that non-cognitive competencies, or non-cogs, can enhance both validity and diversity in selection processes. Non-cogs are psychological constructs that help us successfully navigate life; examples include conscientiousness, achievement orientation, initiative, trustworthiness, adaptability, emotional self-control, and optimism — all qualities I would love to see in my students, my colleagues, and myself.

Importantly, non-cogs do not show measurable differences based on race, gender, culture, or language. Researchers, such as Richard Boyatzis from Case Western Reserve University, have concluded that cognitive ability and disciplinary knowledge are thresholds; they are necessary but insufficient for outstanding performance. It is these non-cognitive competencies that separate exemplars from the subpar, the finishers from the non-finishers. The Council of Graduate Schools recently announced a program to further explore these concepts in graduate admissions.

It is clear that to begin addressing diversity in graduate education on a large scale, the standard admissions protocol needs to be altered.

Despite their clear potential, non-cognitive assessments are presently difficult to use with no standard protocols. Admissions committees typically perform disorganized evaluations of letters of recommendation, personal statements, and in some fields, interviews. This attempt at holistic evaluation, unfortunately, is applied to only a small fraction of the applicant pool — those who passed the minimum GRE hurdle — because it takes a lot of the faculty’s time.

For non-cogs to become widely adopted, their evaluation must be as easy as that of GPA and test scores. Social scientists can help in this regard through the design of Situational Judgment Tests to measure non-cogs (with surprisingly few questions), which result in a set of simple scores. These could be administered at the time the application is submitted. To assess non-cogs evident in written application materials, social scientists can help develop rubrics to make the evaluation more efficient and robust. Rubrics increase fairness, mitigate implicit bias, and combat reviewer fatigue by ensuring all factors are assessed for each applicant.

Using non-cognitive constructs will be integral to helping diversify STEM graduate education. They uniquely offer both a lack of gaps between different demographic groups and the ability to predict performance in their own right. Non-cogs will thus lower barriers to participation for women and underrepresented minorities while enhancing the validity of the currently poor admissions process used by many STEM graduate programs. Their inclusion has the potential to revolutionize the graduate admissions process along the path to meeting national goals for diversity in STEM.●

Casey W. Miller, PhD, is a professor and the director of the Materials Science Engineering Program at Rochester Institute of Technology. Funding for this project came from the National Science Foundation and Rochester Institute of Technology’s Division of Diversity and Inclusion.