Since 2021, Calbright has had a $4.1 million research partnership with UC Irvine’s School of Education and non-profit behavioral design firm ideas42 to find ways to make online learning for adults more effective.
By 2023, this partnership had led to significant innovation ideas to support student persistence, which were tested that year and found to have a significant positive impact: the “Pace and Progress Timelines” approach was applied across the College, with extremely positive student feedback and completion outcomes.
As Calbright begins 2024, the research effort continues – confirming prior findings and offering new approaches to enhance student success.
Helping Students Set a Schedule That Works For Them, and Sticking to it
As an online college, Calbright is able to follow student progress in real time, which provides significant data to drive design interventions and improvements. So even after the College tested the Pace and Progress Timelines, it continued to monitor whether they were having a positive impact across programs and student populations.
The news continues to be good.
“Timelines are yielding positive results, and we’re seeing that across the board,” said Calbright’s Dean of Student Journey Ava Burns. “That impact is relatively small, just a few percentage points, in the first weeks after students enroll. But the longer they stay with the program, the greater the impact the timelines have. Looking across programs that now offer timelines, students are completing an average of 27% of assignments within their first 90 days, compared to just 12.5% before the timelines were implemented. While part of that difference may be attributed to other changes at Calbright over time, it’s clear that timelines are making a meaningful contribution to student progress.”
The more Calbright tests the approach the more confident we are that these results are robust and apply across programs and populations.
Kristen Krenz, a student support specialist at Calbright, said that the important thing about timelines isn’t the actual schedule students choose for themselves, but the way in which they use the timeline to measure their progress.
“Students are still learning at their own pace,” she said. “Having a timeline is the difference between a student saying ‘I’m going to finish this program some day’ and saying ‘I’m going to finish this program in 120 days.’ That gives them a goal in mind, something to measure themselves against and keep them accountable, and if it doesn’t work, we’ll adjust the timeline with them. It’s not about them keeping a schedule that works for us, but staying on track with a schedule that works for them.”
It’s a big win, both for Calbright students and for online adult education.
Videos or it Didn’t Happen
As Calbright confirms the benefits of the Pace and Progress Timelines, it has also tested new approaches to student engagement. In the process, it’s discovered that big isn’t necessarily better: often small interventions can have a significant impact.
While Calbright seeks to make the college experience as logistically simple as possible for all its students, there are some key student responsibilities: completing their applications, signing up in their own accounts, and reaching out if they’re struggling with their coursework.
There are things Calbright can do to support students in all of these processes, but students also need to motivate themselves. And these are often points where some students may struggle, especially as many adult learners have work or family responsibilities.
Late in 2023, as part of the research partnership, Calbright’s Student Success team started a pilot program to see if adding short, TikTok style videos about specific tasks at key points in their communications with students might help students at these critical points.
To find out, they tested three videos:
- A video to help students complete their enrollment
- A video introducing students to their success counselor and encouraging them to schedule an appointment within the first month of their studies
- A video encouraging students who have completed at least 40% of their coursework to visit Calbright’s Career Services website and download resources there.
The videos were sent to test groups of students who seemed like they were struggling at key points for these metrics, and over time the results were compared with students who did not receive videos.
The results were not as strong as the timelines, but the videos clearly had an impact across the board:
- Prospective students who received the video as part of their communications from Calbright finished their enrollment between 4-6% more than students in the control group.
- New students who received the video introduction to their academic success counselor as part of their communication from Calbright scheduled an appointment five percent more often.
- Students who received the video about Calbright’s Career Services page as part of their communication visited the page 37% more often than the average of students in the control group.
These videos aren’t major interventions, they’re less than one minute communications with students, just using a new medium and approach – but they do have an impact. Factored over thousands of students, those nudges can help our students clear hurdles on their path to program completion and career success.
Calbright is different from a traditional college in many major ways: we’re free, our students take classes at their own pace, and we don’t use grades or credit hours, to name just a few. All of those big changes are research based: we do them because they work. But our research also focuses on small changes and interventions that work. On human interactions and high touch social connection, on little things that each help students in small ways. Over time, they add up to changed lives.
