Professor Koo Myoung-wan’s Team from the Artificial Intelligence Department 

Wins Top Prize in ‘2022 ETRI Human-Understanding AI Thesis Competition’

- Thesis “Multimodal Recognition of Emotion in Conversation Using MLP-Mixer Structure” wins Minister of Science and ICT Award and prize of KRW 5 million -

- Methods for multi-modal recognition of emotions in conversation presented as part of efforts to develop emotionally interactive AI –

▲ (Clockwise from top left) Professor Koo Myoung-wan, master’s student Bang Na-mo, master’s student Lee Jee-hyun, and master’s student Yeen Heui-yeen

On Thursday, June 30, 2022, Sogang-ISDS, a research team led by Professor Koo Myoung-wan of the Computer Science and Engineering Department (Artificial Intelligence Program within the Graduate School), won top prize in the ‘2022 ETRI Human-Understanding Artificial Intelligence (AI) Thesis Competition,’ hosted by the Electronics and Telecommunications Research Institute (ETRI) in Seogwipo, Jeju Province, Korea. The team was recognized with the Minister of Science and ICT Award.

Sogang-ISDS, comprising of Bang Na-mo, Yeen Heui-yeen, and Lee Jee-hyun, master’s students at Sogang’s Graduate School of Artificial Intelligence, submitted a thesis titled “Multimodal Recognition of Emotion in Conversation Using MLP-Mixer Structure.” The team was granted the Minister of Science and ICT Award and a prize of KRW 5 million. The award-winning thesis will be published in the dissertation collection of ‘Korea Computer Congress (KCC) 2022,’ organized by the Korean Institute of Information Scientists and Engineers (KIISE).

The research team presented a first-of-its-kind idea through its thesis that applied the MLP-Mixer structure to the recognition of emotions using multi-modal data to combine speech and text. The team’s multi-modal MLP Mixer (MMM) structure combines the characteristics of speech and text data, extracted with Wav2vec 2.0 (speech pre-training model) and KcELECTRA (Korean comments pre-training language model) with MLP-Mixer to classify final emotions expressed in speech and text.

▲ Structural chart

For the thesis competition, ETRI had invited submissions for research findings in the area of AI technologies for communicating with humans. Out of 36 contenders, only the teams that survived the first round of thesis screening, held from March 16 to April 22, were given a chance to make presentations in the second round of screening. A total of seven teams, including Professor Koo Myoung-wan’s team, competed in the second round, with the screening session held on June 30 as part of the KCC 2022 programs.

Yeen Heui-yeen of the Sogang research team remarked, “At the start of the first semester of the master’s program, Professor Koo instructed us to always take on challenges and think up new ideas based on thesis studies. As we held thesis seminars that encompassed various fields, we applied MLP-Mixer – a cutting-edge technology from a different field – to the emotion-recognition field. I feel even greater pride in receiving this award for the fact that the thesis was only possible as a result produced by our team members’ concerted efforts and passions, coupled with Professor Koo’s valuable advice along the way.”

▲ Celebrating the top prize in the 2022 ETRI Human-Understanding AI Thesis Competition

The award-winning students are all in the master’s program of the Artificial Intelligence Department, newly established in 2022 within the Graduate School of Sogang University. With academic backgrounds in liberal arts, they have emerged as interdisciplinary talents majoring in artificial intelligence, of all things. They are on full scholarships from Smilegate, a global video-game developer and publisher in Korea. In the spring semester of 2022, the Artificial Intelligence Department within the Graduate School welcomed 19 freshmen. Going forward, Sogang University is planning to establish the Artificial Intelligence Department for undergraduate studies in 2023, with a view to fostering interdisciplinary talents in artificial intelligence to play a pioneering role in the Fourth Industrial Revolution. The university also intends to promote AI-based interdisciplinary education for all university students.