The GRE vs. grit: The changing face of graduate school admissions
Originally published March 3, 2015
If you want to go to graduate school in America, more than likely, you will have to complete the Graduate Record Examination (GRE). Since World War II, this test has been a dominate factor in the decision-making process of most graduate admissions officials. Today, the GRE still holds an inordinate amount of weight in the application process and higher education and business are beginning to suffer for it.
SFL recently spoke with Keivan Stassun, Professor of Physics & Astronomy at Vanderbilt University and Co-Director at Fisk-Vanderbilt Masters-to-PhD Bridge Program, to discuss not only his thoughts and work in the area of graduate admissions processes, but also to learn more about how schools ought to give extra consideration to gathering supplemental information to create more complete assessments of student candidates.
SFL: Please describe the Fisk-Vanderbilt Master-to-PhD Bridge Program.
Keivan Stassun: In a nutshell, the program is for students who have completed undergraduate degrees in a science or engineering field, and who aspire to a PhD in science or engineering, but who might not be ready to just jump immediately into a PhD program. The students first complete a two-year Master’s degree training period where they get intensive research training, a customized curriculum, lots of mentoring and professional development. At the end of the two-year Master’s phase, students transition seamlessly into the PhD program at Vanderbilt.
SFL: Generally speaking, how long does it generally take a student to go through the PhD program at Vanderbilt?
Stassun: A good average for students who come straight into our PhD program is five years. For students who come through our bridge program, generally they will complete the PhD phase in about four years. If you add to that time the two years in the Master’s training period, their total time to PhD for students in our bridge program is usually about six years.
SFL: Does any of the work in the Masters level to the PhD level requirements, or is the bridge program an acclamation process, or a transitional process, as opposed to actually working towards the credits a student need to earn for a PhD?
Stassun: No, we designed it so that the work done during the Master’s phase, a lot, if not most of it, does transfer to satisfy the PhD requirements. For example, for a typical student coming through our bridge program, when they join the Vanderbilt PhD program two years later, they will have fulfilled most, or all, of the PhD course requirements along the way. They’ll also have a jumpstart on their dissertation research as well.
SFL: Given that you have two feet in two fairly different institutions, do you see a difference in the level of preparation from students that come straight from Vanderbilt’s Master’s program and go into the PhD program, versus those that go through the bridge program?
Stassun: Yeah. There definitely are differences.
One difference is that the coursework and research work that the students do while in the Master’s phase of the program is Master’s level, as opposed to PhD level. That’s intentional. We see that as a real benefit and feature of the bridge program. For those students who, as I said before, might not have felt quite prepared to jump into a PhD program, those two years that they spent in the Master’s phase really ramps them up to be able to succeed at the PhD level.
The other thing to say is that, for better or for worse, in most PhD programs, when students come into the PhD program, they’re thrown in head first. It’s kind of a baptism by fire. Generally speaking, there’s not a lot of transition, easing in, professional development or acclimating to the rigors and expectations of PhD research.
I would say that students who come through our bridge program really get a leg up in that respect, because they spend two years in the Master’s phase getting a lot of that acclimation, preparation and mentoring for those expectations so that they can really succeed, whereas, the national average PhD completion rate for PhD students in the sciences and engineering is about 50 percent. In our bridge program, we’re seeing an 81 percent PhD completion rate. We think that there are real advantages, potentially for any student in terms of their long-term success to go through a program like ours.
SFL: Regarding your study and the use of the GRE during graduate admission processes, would you say that even though 80 percent are successful, that the GRE, in many ways, could prevent some of that 80 percent of moving forwards based upon its limitations?
Stassun: Yes, but just to be clear, we do not consider GRE scores as part of our admission criteria to the bridge program. We only collect the GRE scores for statistical purposes.
That being said, when we look at the GRE scores that the students in our bridge program have had, more than 90 percent of those students would have been filtered out by most PhD program admission criteria.
SFL: Let’s just talk about the GRE in general. It started in 1949 and it still possesses, from a pedagogical standpoint, a very World War II mentality. From your perspective, as to its history, what has been its primary function, and is its function valid today?
Stassun: My perspective is that, as you said, the GRE and the SAT, as well as a number of other standardized exams, really came to life in that era around World War II. At that time, the GRE and SAT and other standardized tests represented a hugely positive step forward from the standpoint of democratization of higher education and opening doors of opportunity. The reason being, prior to the advent of standardized tests, many, if not all, elite institutions did admissions in the way that completely shut the door to students from low socioeconomic status. If you wanted to go to an Ivy League school, your parents would have had to have gone to that particular Ivy school. To have had a way for students from poor backgrounds, or racial/ethnic minority backgrounds, to have a way to demonstrate they merited access to those elite institutions, at the time, was a big breakthrough. It was a big step forward. I think that’s important to say.
Fast forward 60 years, the world has changed. We now have a somewhat more nuanced situation where students and society as a whole really expect merit as the fundamental basis for selecting people. What we know from the decades of data that have been gathered from the GRE, as well as other standardized tests, is that, for reasons that are not completely understood, people from different groups, people from different genders, people from different ethnic groups, people from different socioeconomic status will perform differently on these exams in ways that do not fully reflect their true merit or their true capabilities.
Unfortunately, despite the very important gains that were made 60 years ago with the introduction of these exams, 60 years later we haven’t evolved in the ways in which we use these tools. We still use them as really blunt instruments, even though, as we show in our nature article, highly capable students‚ African-Americans, for example‚ students who have every promise of success, those students will perform 200 points lower, on average, than white or Asian students. We don’t factor that at all, generally speaking, in our PhD admissions.
There’s an unevenness in the playing field that’s sort of built into these standardized tests. It’s not for any pernicious or nefarious reasons, as far as I can tell, but nonetheless, it’s there. I believe fully that ETS (Educational Testing Service) does due diligence to create the fairest and best exams that they can. The onus is on those of us in universities who are doing admissions and using these tools to use them in a much more informed way than we have been for the past few decades.
SFL: It seems as though that one objective of standardized tests was to objectify people, but not in a negative way, in a positive way. It’s a blind evaluation. The idea is that you look at the cold, hard numbers and that’s it. However, what that has led to, especially with the K-12 environment, are the numerical benchmarks or standards that we’re fixated upon today. We’re trying to get numbers up, test score results, not necessarily measuring how students are truly developing. It’s a very Cartesian approach to assessment where you dice something up like learning to such a degree that it loses its humanity. I’m I making sense?
Stassun: Yes, you are.
I would say, for example, if you look in the corporate world, talk to any CEO of a Fortune 500 company about how they do hiring, about how they recruit for and identify talent within their company. You basically never hear them talking about some numerical score. They talk a lot about very human characteristics. In education, as with the business world, there are certain things that have to be there. There’s a certain level of intelligence or smarts required. We can’t pretend that we can grab anybody off the street and stick them into a PhD program, no matter what their human characteristics may be, and expect them to be successful. We definitely do have to be assessing various aspects of prior preparations and academic ability.
To get to your question, I think that what we have not been doing very well at all in PhD admissions in recognizing that there are these non-cognitive, but human and very real and very important, attributes that people have that really matter when it comes to potential for academic and business success. In our program, that’s something that we’ve really been focusing on. Even as we do screens for basic academic preparation in the students that we admit, instead of looking at a number like a GRE score, we interview them, we probe for specific skills and attributes that students have that we know are predictive of their potential for long-term success. It does have an effect on the students themselves. When they are admitted into our program, they know that the reason they were picked is not in spite of a low GRE score, they were picked because they bring a set of human characteristics and attributes that make them valuable to our program and highly likely to succeed when given our prescribed academic tasks.
SFL: What are some of the human attributes that you’re looking for in the interview?
Stassun: In a word: grit.
There’s an emerging research literature in psychology around this idea of grit, and that’s actually what it’s called. The word is a simple, single word that encapsulates a handful of more specific attributes that predict student success.
For example, one of the more specific attributes that is encapsulated in grit is an orientation toward long term goals. The idea there is that you can have somebody who is very smart, but if they’re just following their nose and focused on achieving the thing that’s right in front of them? What you risk with such a person is if they stumble on that next step in front of them and they don’t have their eyes on the prize that’s on the horizon. When they stumble on that one step in front of them, they might just give up. They’re not motivated towards something bigger that’s on the horizon. People who are gritty tend to know what they’re aiming for long-term, and that allows them to, if they do get tripped up by something, they can stand up, dust off, and keep going. They know that the stumble was only a temporary problem and it’s not ultimately the thing that they’re wanting to achieve. So, orientation toward long-term goals is a very important characteristic.
Another attribute we seek in students is the ability to accurately self-assess. Gritty people are people who understand their own imperfections and weaknesses, and regard those not as character flaws or as intrinsic limitations to their ability to achieve long-term goals. Rather, they look at those things as specific areas that they need to hone and improve. It makes their path toward that long-term goal much more efficient, because they can spend more energy improving things that they accurately assess to be in need of improvement.
I’ll mention just one more specific attribute, which, in essence, is networking and connectivity. Current research literature tells us that gritty people, successful people, tend to be highly networked people. They’re connected to lots of other people in important ways. They have mentoring relationships with others. They have peer support with friends and family. They are involved in communities that matter to them, whether they are faith communities or cultural communities, and that networking allows them to draw upon a large set of human resources when the going gets tough. If they start to feel lost or if they start to stumble, they’re not alone. They can draw upon this network of resources. That turns out to also be an important aspect behind what makes someone gritty.
Those are three specific attributes that are examples of this grit factor we should seek.
When we interview students, it’s not just an interview. It’s a carefully crafted interview that’s designed to probe for these attributes. It’s not a test of performance. It’s not a test of how polished or well-spoken somebody is. In fact, some of our best and grittiest students are a bit shy. We’re probing for these specific attributes that tell us that this is somebody who can be mentored and who is very likely to succeed, because they have what it takes.
SFL: What would you say to somebody who makes the claim that, in the STEM areas, there shouldn’t be so much subjectivity when evaluating someone for a position in a graduate program? They say the process should be more objective, due to the nature of the curriculum you’re dealing with in the STEM areas. What would you say to that line of thought?
Stassun: I wouldn’t entirely disagree.
I would say that denying that these attributes, such as grit, are important, to deny that is to be subjective. The research tells us, in fact, that those attributes matter a lot. In fact, I would argue that one of the big problems that we’ve had in graduate education in STEM is that we’ve operated on faith that we can look at these numbers and accurately determine who is going to be successful. That’s a very subjective way to operate, when objectively the research tells us that these other attributes are very, very important.
SFL: So admissions officials can use the GRE. They can conduct interviews. What other components should be used to evaluate someone’s graduate admissions application?
Stassun: We look at their undergraduate preparation. By that, I do not mean simply looking at their undergraduate GPA. That would just be looking at another number. We look very carefully at their undergraduate transcripts and read the letters from their undergraduate mentors in order to understand exactly where is the student in their academic preparation. For example, if we’re looking to admit a student into the physics track of our bridge program, we’ll want to make sure that they took an appropriate set of math courses as an undergraduate, that they took an appropriate set of introductory and intermediate physics courses. Very often, their grades were okay. Maybe they had a few courses where they got a bad grade, but, if on the whole, we can see that they’ve got the basic, necessary preparation under their belt, then that allows us to begin to correctly place them in terms of what additional coursework preparation‚ tutoring, or whatever‚ that they might need in order to ramp them up. If we can combine the past course work with a high grade on our grit scale, then we know that the student candidate has what it takes to be mentored, and geared up to fill in any of gaps in their undergraduate background. That’s how we look at the whole package.
SFL: Some could say that, by including this interview and by probing more in a holistic fashion that you might be opening yourself back up to those old ways of interviewer-prejudice, whether they are conscious or unconscious, bubbling forth and impacting the final admission decision. What sorts of safeguards have you put into place to prevent that from occurring within your process?
Stassun: That’s a good point, and I think that’s a very valid criticism.
Most of us have had experiences with interviews, either as the interviewee or the interviewer, where we were, in essence, having a conversation with somebody and we were asking questions on the fly and trying to develop a general impression. That’s a pretty subjective approach. What we’ve done is we’ve worked with researchers who are experts in this area of grit and these human attributes and how to probe them. We’ve developed an interview protocol that we follow. Think of it as a script. Now, we don’t ask exactly the same questions in exactly the same way every time. It’s not scripted to that extent, but we have a protocol that we’ve published in the American Journal of Physics that serves as a concrete guide to the interview process so that we’re asking certain kinds of questions and so that we know how to interpret the answers that we get to those questions in the context of these specific grit attributes that we’re trying to measure.
SFL: It seems as though everything you’ve mentioned is logical and reasonable, but in terms of practicality, if applied in a broad fashion, you almost have to have a dedicated set of staff to come together to pound out admissions policies and processes, and then make sure the new requirements are sustained. It seems like that could be a difficult thing to accomplish in a higher education setting, given that there can be a lot of faculty turnover. What you’re describing in terms of front-end planning is pretty labor-intensive in terms of sweat equity.
Stassun: I’m going to disagree somewhat with the way you presented it there, and I understand you’re just framing the question.
Graduate education is one of the highest-stake things we do in a university. Graduate education is expensive. The students we bring into our PhD program, we’re investing, both in terms of dollars and in terms of time, a great deal. It’s an incredibly expensive proposition to have a student not make it through our PhD programs. Because we all recognize that, we already invest a lot of time and effort into the admissions process.
A department’s Graduate Admissions Committee can meet many times during the admissions season to discuss applicants, to think about which are the very best students to try to admit, to go through the application materials, to invite students to visit. It is a labor-intensive process, but it’s labor that we’re already putting in for very good reasons. What our process does is it simply systematizes that work.
To be very concrete, for our bridge program every year, we will narrow down the pool of applicant to approximately three candidates for each slot that we expect to be able to admit. So if we expect to be able to admit a dozen students, we’ll interview 36. Each interview, because we have a protocol and because we have a systematized process, takes approximately 30 minutes. There are two faculty members who participate in each interviews. In a program like ours, let’s say that a third of those candidates are in one area like physics. You’ve got 12 candidates to interview, 30 minutes a piece. That’s six hours of interviewing for two faculty members. The way that we do it is we block out an entire day for those two faculty. They conduct their six hours of interviews back to back to back to back. Then they spend another hour and a half or so collecting their notes on these interviews and turning their grit assessments into numerical scores. The upshot is that what it costs in time, I can tell you exactly, is two faculty members in a given department, one full day of their time. That’s not nothing, but I think most people would recognize that that’s not an enormous amount of time or labor compared to what we already invest in the admissions process.
SFL: Some think that these sorts of evaluative considerations need to occur much earlier in a student’s academic career, like at the high school level. Student could go through similar benchmarks and get much better ideas of whether or not they can succeed at the following stage of their academic career. Let’s say starting around the 10th grade, this approach is how people are going to be evaluated, and feedback will be given to them. What do you think would happen over the course of that student’s academic career by the time they got to you?
Stassun: I don’t think I’m qualified to comment or make recommendations about whether or how this kind of process should be done at other places in the educational pipeline. I will say that I’m pretty convinced that there’s a positive impact on the student from learning to think about assessing themselves along this dimension, in addition to the other dimensions. Student should be thinking at early stages of questions like: ‘What grades am I getting? How am I performing on standardized tests? How am I doing in developing my grit?’
There is some very nice research that has been done by Carol Dweck about what she calls‚ ‘growth mindset’. Her idea is that successful individuals learn early, on one way or another, that failure is a temporary condition, and that when they fail at something or struggle with something that it doesn’t necessarily mean that they’re not capable. Rather, they have a growth mindset, which is, ‘Okay, I’ve just learned something. I need to improve in concrete ways.’ I think it would be tremendously positive for students and, downstream, for those of us looking to bring talented people into science, if students were learning from a much younger age to think of themselves in that way and if we were also assessing them in that way. That approach then would be ideal.
SFL: Moving to a final point in dealing with diversity which is the notion that, for whatever reason, this process of looking at the GRE in a strict fashion like many admissions officials do is creating an undiversified environment within STEM. Your research brings up the point that you need diversity within STEM because it helps with critical thinking and creativity. Can you talk about how STEM is hurt by a lack of diversity and how students are harmed in the process? If I get in as a graduate student because I earn a high score on my GRE, I’m not going to be exposed to those people from diverse backgrounds. That’s impacting me and my development, not just the person that didn’t get in the program.
Stassun: There’s great emerging literature and a general understanding that diversity is a key component to innovation and problem-solving. It seems to be the case that as human beings, our brains are wired to perform best when tackling a difficult task‚ when trying to solve a difficult or big problem. We’re wired to do that best in the context of assembling multiple perspectives from different directions.
In the business world, this has come to be really deeply understood and recognized that you don’t just hire the best. You assemble great teams and an essential component of that is to attend to very different backgrounds and experiences and, therefore, perspectives that different people bring. When you do that, the whole is greater than the sum of its parts. Everybody benefits.
2021 Update: Since 2014, Stassun has received numerous awards and produced a multitude of academic publications. He has also been involved in a pilot program at Vanderbilt- the Frist Center for Autism and Innovation. Stassun continues to support de-emphasizing the GRE in the graduate admissions process. He gave a lecture at Cornell University in Oct 2018 titled‚ ‘Holistic Admissions: Achieving Diversity at the Ph.D. Level’‚ that speaks to this. In April 2021, he gave a lecture at Vanderbilt titled‚ ‘Model for Dramatically Increasing Diversity at the Ph.D. Level in Science and Engineering’. He continues to advocate for replacing the GRE in admissions with indicators that are more predictive of long-term success.