Examples of visuals produced by the IAD that highlight student demographics

Creating meaningful communication with students in a large course is a familiar challenge for Adam Brummett. With nearly 1,110 students in his Principles of Chemistry I course, he was searching for a new method to effectively communicate with hundreds of students.

Brummett, a lecturer in chemistry, collaborated with OTLT Research & Analytics to take a data-informed approach to gather more information about students’ learning behaviors. The idea was simple: if they could identify students who were struggling early in the semester and have insight into their behavior, they could offer targeted feedback to motivate them.

The team designed an instructional analytics dashboard (IAD) for Principles of Chemistry I in fall 2020. The dashboard aggregates data from multiple technology tools used in a course (e.g., ICON, Zoom, Panopto, and online homework platforms) to generate near real-time reports that offer insight into students’ engagement patterns and interactions with materials and tools.

The team’s work expanded into spring 2021. Targeted feedback messages were developed for the course based on data from the dashboard and sent to students via mail merge. The feedback consisted of data specific to the student and was craft

Adam Brummett
ed to encourage metacognition related to studying and performance while maintaining a positive tone. For instance, a message might recognize students for completing practice problems and invite them to discuss strategies for studying and other available support.

“I can make an announcement in class or send a mass email, but it doesn’t resonate until the message is more personal, and the dashboard helped us create the targeted emails,” explains Brummett.

The response was overwhelming. While some students replied with a brief “thank you,” others shared the challenges they’d experienced in the past year and how the message made them feel connected to the course.

“Personalized messages can initiate and establish a connection, which helped open new ways of communicating with students and reducing the barriers to understanding chemistry and science,” says Brummett. “Research also demonstrates that when students feel more connected to a community, there’s a higher likelihood that they’ll persist in a field.”

By the end of the spring semester, 72 percent of the 650 students enrolled  had received at least one feedback email. The team traced student behavior to see which group changed their behaviors after receiving the feedback emails. In addition, they collected students’ perceptions of the helpfulness of the messages at the end of the semester.

“I’m excited to see how the dashboard continues to progress and to disseminate what we’ve learned,” says Brummett. “I believe that the dashboard can provide useful insight into courses in many other disciplines.”

Join Brummett and the OTLT Research & Analytics team at an upcoming presentation, Hello? Effective Messaging and Connecting to Students in Large Courses, from 11 a.m. to noon on Thursday, Oct. 28. Learn more about the process of developing personalized messages and how they affected students’ behavior. They’ll also introduce the instructional analytics dashboard and explain how it informed the creation of the targeted messages.