Jackson Holt ’26 did more than study computer science. He combined it with psychology, built a working system that responds to human emotion and tested it in real time.
What if you could rehearse confidence?
Imagine practicing a conversation before it matters.
“You can create interactions that would allow them to practice checking out at a store or ordering in a restaurant,” Holt said. “Some of those situations make people really nervous.”
That is where his project ended. It is not where it started. Holt began the summer with a different goal: building a poker-playing robot that could detect bluffs by reading emotional cues.
“My original idea was to set it out to be able to play poker with you and be able to tell whether you’re bluffing or not,” he said.
The concept worked, but the timeline did not. When Holt brought it to his professor, Kenny Moorman, the response was not to shut it down but to scale it back.
“Let’s just see if you can get the robot to alter its behavior based on the emotions that it’s picking up,” Moorman said.
That shift changed everything. Before detecting a bluff, a machine has to recognize emotion. What started as a clear goal became something more open-ended, something Holt would have to figure out as he built it.
Building something that did not exist yet
Each stage of the project required Holt to make decisions without knowing whether they would work, often building and testing ideas from scratch. Progress came through trial and error, forcing him to adapt, rethink and move forward without a clear path.
“There were several moments where it would click,” Holt said. “And then I’d be like, oh man, I really don’t know what I’m doing again.”
He worked with Furhat, a social robot designed for human interaction. It could look at a user, listen and respond, but only if someone built the system behind it.
He taught himself Kotlin while learning to build a real-time system that could interpret facial expressions, process speech and respond in the moment.
“Almost every day of the project would involve problem solving and learning new things,” Moorman said.
“In our courses, there’s scaffolding,” Moorman said. “We tell students what they need to do. In research, that scaffolding goes away.”
That kind of open-ended work is what makes research at Transylvania different. Students are not following a set path. They are defining it as they go.
“You’re in the driver’s seat for the first time,” Holt said.
By the end of the summer, he had built and tested a system that could read facial expressions, compare them to spoken words and determine a basic emotional state before responding in real time.
Where disciplines come together
The project quickly moved beyond computer science. To interpret emotion, Holt had to understand how people express it. He turned to psychology, studying how facial expressions and speech patterns signal emotional states, then translated those ideas into code.
“I had to read psychology papers. I didn’t think this is where this was going to go,” he said. “But that interdisciplinary approach is really cool.”
Instead of keeping subjects separate, he combined them. Computer science, psychology and communication became pieces of the same problem, each shaping how the system worked.
When it didn’t go as planned
Most of the time, the system fell short of expectations, producing responses that were awkward, inaccurate or unusable. Early tests often revealed gaps between what worked in theory and what functioned in practice, forcing Holt to revisit his assumptions, troubleshoot failures and rebuild key parts of the system before trying again.
“In theory, it was elegant,” Holt said. “In practice, it broke constantly.”
The first interaction made that clear.
“It wasn’t great,” he said. “It didn’t go well. It kind of crashed and burned.”
He adjusted and tried again, reworking how the system interpreted emotion and how it responded.
“The hardest problem for me during this project was probably when you’re trying to simulate that conversation, you’re trying to decide, you know, if they’re happy, how do you respond?” he said.
Because the system was not fully automated, Holt built those responses himself. He tested them by pulling in his roommates.
“I had a few of my roommates actually come in and say, all right, can you talk to this and just tell me if this sounds weird,” he said.
Their feedback was immediate. Some responses were bluntly negative, others more encouraging. He revised and tried again. Even the robot’s appearance became part of the process. When it looked too realistic, people pulled back. He adjusted it to look more clearly robotic, and the interaction improved.
Gradually, something shifted. What started as skepticism turned into surprise.
“They’d be a little skeptical,” Holt said. “And then as the robot started talking and they were like, wait, it’s kind of understanding me. That surprise is really cool.”
The same lesson, in a different place
That process of adjusting, trying again and continuing forward was familiar. As a goalkeeper on the men’s soccer team, Holt’s first season did not go as planned. In an early game, he gave up six goals and did not play again that year.
“Sometimes things aren’t going to go your way. You’re going to have to learn to lose,” he said.
He stayed, improved and eventually earned the starting role, helping lead his team to a conference championship and earning Defensive Player of the Year honors.
The same mindset showed up in the lab, working through setbacks without stepping away from the problem.
What the project became
By the end of the summer, the project looked nothing like the one he started. Holt had built a system he designed and tested himself that reads emotion and responds in real time.
“I think this kind of social interaction with a robot or a humanoid robot or whatever it may be has some really beneficial uses,” he said.
He thinks about people who find everyday interactions difficult, including those with social anxiety, ADHD or anyone who feels unsure in those moments. A system like this could give them a way to practice without pressure.
“I had a few family members that have trouble interacting or get a little nervous in certain interactions,” he said. “So being able to be a part of that kind of research was pretty cool to me.”
What comes next
Holt is preparing for graduate school in artificial intelligence and human-computer interaction, building on work that connects technical systems with human behavior.
“When a student is the primary on a project, it completely changes their view of how things work,” Moorman said. “When they get that experience, it is truly life changing.”
By the end of the summer, Holt was not guessing what he could do next.
He had built something he could point to and a clearer sense of where it could take him.