University of Arkansas testing use of math model to award aid, offer mentoring

FAYETTEVILLE -- A new mentoring and grant program being tested by the University of Arkansas, Fayetteville makes use of models predicting student outcomes to decide who receives help.

The university's Student Talent Enrichment Program provides one-time grants for already enrolled first-year students from Arkansas with at least $2,500 in unmet financial need in the fall semester. Other eligibility requirements include a high school grade-point average of at least 3.0.

Predictive data models developed by UA determined the amount of one-time awards of up to $10,000 given to 151 students, according to documents released by the university and a UA spokesman. Another 150 students are enrolled in a mentoring component of the pilot program.

Using an algorithm to determine financial awards, while not unique, differs from typical college programs and raises questions about transparency, higher-education and data researchers said.

UA officials said the pilot program's mentoring and need-based grant components will be studied for their effectiveness.

"These approaches have shown success at other schools, and we're trying to tailor them to the University of Arkansas and the state of Arkansas," said Charles Robinson, UA's vice chancellor for student affairs.

The university declined to release details of the predictive models for the one-time grant program beyond the eligibility requirements, saying "selection for invitation into the Student Talent Enrichment Program (STEP Grants) includes consideration of many factors, some of which are weighted."

Grant amounts were based "on a measure that projects probability of one-year retention," a reference to the likelihood of students returning for their second year, according to documents released by UA. Grants were awarded in such a way "that most improved the group's sum of projected probabilities of retention," based on an email from Trevor Francis, UA's associate vice provost and director or student success, included in records released by UA.

UA released some technical details for models used to calculate students' anticipated first term grade-point average, which is how selections were made for the mentoring program.

A document released by UA stated: "The categorical variables that contributed to the predictions of first-term GPA for one or more groups include initial housing type, whether any college-level credit has been transferred, possession of a first-year scholarship, sex, initial college," referring to colleges within UA.

Parental college completion, the state where the student graduated from high school, and a student's high school grade-point average adjusted for several factors were among other variables listed, with the document stating that "high school GPA was easily the best single predictor of freshman GPA."

To better evaluate the program, some students with "similar demographics" were selected for a control group not receiving help from the program, Robinson said.

Mark Rushing, a UA spokesman, said a control group was included in the program design "so that we can statistically assess the impact of STEP investments." UA's most recent first-year retention rate is 82.2 percent, according to university data.

UA is spending $650,000 in one-time grants and $150,000 on mentoring, Rushing said, with no other student-aid programs reduced. The school's student affairs and central administration offices, among other units, are funding the program.

Chancellor Joe Steinmetz in a September speech said the pilot program is a way to help with financial hurdles faced by some students.

"Small grants of a few thousand dollars are likely sufficient to bridge the gap between what a student has and what he or she may need to stay in school," Steinmetz said. "The need-based component is similar to a University of Kentucky initiative which has been very successful, which uses predictive analytics to identify students who might struggle and reaches out to them before that happens."

Lisa Wilson, University of Kentucky's associate provost for finance and administration, in an email said students last fall as well as this fall "were identified by a predictive model" to receive one-time grants.

Wilson said "approximately 20 different academic, demographic and financial variables" were used in the model.

However, Will Doyle, an associate professor of higher education at Vanderbilt University, said the way UA is using predictive analytics is "pretty unusual."

Doyle said institutional grants typically are either based on financial need or based on academic achievement. In those cases, "the criteria for receiving the aid is really clear," Doyle said.

The UA pilot program's approach "might make sense from the institution's perspective, to target aid in a way that's most likely to increase the retention rate from one year to the next," Doyle said. But "it's unlikely that's going to be clear at all to the student," he said.

Claire Fontaine, a researcher with the New York-based Data & Society research institute, said "there's potential with any algorithm of it leading to increasingly unequal outcomes." She added that a potential benefit of an algorithm is that it can bring consistency to decisions, but said it depends on how the model is constructed and that the model should be made public.

TyCamRon Mack, 18, said he was told in November that he had been selected for the mentoring portion of the program, which Rushing said will involve student mentors meeting biweekly with the first-year students to talk about academics. Each student being mentored also will receive $1,000 in university funds, Rushing said.

"I'm paying for college on my own," said Mack, from Dumas, which is about 40 miles southeast of Pine Bluff. He said the $1,000 award "really caught my attention."

Robinson said students chosen for mentoring have potential academic challenges that mentoring aims to address. In contrast, students in the need-based grant component of the program are receiving extra consultation about financial matters, Robinson said.

"When we first started thinking about this, we thought that the data analytics would show that these students were pretty much the same. And in some case they were, but in many cases they were not," Robinson said.

Mack, a member of the Razorback Spirit Squad, said he's confident he'll continue with his studies at UA.

"I feel like I have every resource at the University of Arkansas, and if I ever doubted not coming back, there is somewhere or someone that I could talk to about getting help," Mack said.

Metro on 12/03/2017