Publications

2024

# Multimodal # RLAIF

Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback

Daechul Ahn, Yura Choi, Youngjae Yu, Dongyeop Kang, Jonghyun Choi

Arxiv

# NLP # Reward Modeling

Aligning Large Language Models by On-Policy Self-Judgment

Sangkyu Lee, Sungdong Kim, Ashkan Yousefpour, Minjoon Seo, Kang Min Yoo, Youngjae Yu

Arxiv

# NLP # Conversation # Recommendation

Pearl: A Review-driven Persona-Knowledge Grounded Conversational Recommendation Dataset

Minjin Kim, Minju Kim, Hana Kim, Beong-woo Kwak, Soyeon Chun, Hyunseo Kim, SeongKu Kang, Youngjae Yu, Jinyoung Yeo, Dongha Lee

Arxiv

# NLP # Conversation

Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support Conversation

Dongjin Kang, Sunghwan Kim, Taeyoon Kwon, Seungjun Moon, Hyunsouk Cho, Youngjae Yu, Dongha Lee, Jinyoung Yeo

Arxiv

# korean-LLM # Naver

HyperCLOVA X Technical Report

Jiwan Chung, Sangkyu Lee, Youngjae Yu contributed.

Arxiv

# NLP # Reasoning # Code Generation

Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models

Hyungjoo Chae, Yeonghyeon Kim, Seungone Kim, Kai Tzu-iunn Ong, Beong-woo Kwak, Seonghwan Kim, Taeyoon Kwon, Jiwan Chung, Youngjae Yu, Jinyoung Yeo

Arxiv

NAACL2024

# multimodal # Commonsense # Video Understaning

SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models

Hyun Lee, Kim Sung-Bin, Seungju Han, Youngjae Yu, Tae-Hyun Oh

Arxiv

ICLR2024

# Text-to-Image # PEFT

Navigating Text-To-Image Customization:From LyCORIS Fine-Tuning to Model Evaluation

Shin-Ying Yeh, Yu-Guan Hsieh, Zhidong Gao, Bernard B W Yang, Giyeong Oh, Yanmin Gong

Arxiv

2023

Neurips2023

# multimodal # Commonsense

Localized Symbolic Knowledge Distillation for Visual Commonsense Models

Jae Sung Park, Jack Hessel, Khyathi Chandu, Paul Pu Liang, Ximing Lu, Youngjae Yu, Qiuyuan Huang, Peter West, Jianfeng Gao, Ali Farhadi, Yejin Choi, Qiuyuan Huang

Arxiv

EMNLP2023 (Oral)

# multimodal # Commonsense # Social Norm

Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms

Seungju Han, Junhyeok Kim, Jack Hessel, Liwei Jiang, Jiwan Chung, Yejin Son, Yejin Choi, Youngjae Yu

Arxiv

EMNLP2023

# multimodal

VLIS: Unimodal Language Models Guide Multimodal Language Generation

Jiwan Chung, Youngjae Yu

Arxiv

EMNLP2023 (Oral)

# NLP # Commonsense Knowledge # Distillation # Dialog

SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization

Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, Yejin Choi

Arxiv

EMNLP2023

# NLP # Commonsense Knowledge # Dialog

Dialogue Chain-of-Thought Distillation for Commonsense-aware Conversational Agents

Hyungjoo Chae, Yongho Song, Kai Tzu-iunn Ong, Taeyoon Kwon, Minjin Kim, Youngjae Yu, Dongha Lee, Dongyeop Kang, Jinyoung Yeo

Arxiv

BMVC2023

# VideoQA # multimodal

Long Story Short: a Summarize-then-Search Method for Long Video Question Answering

Jiwan Chung, Youngjae Yu

Arxiv

RA-L/ICRA2024

# Robotics # NLP # uncertainty estimation

CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agents

Jeongeun Park, Seungwon Lim, Joonhyung Lee, Sangbeom Park, Minsuk Chang, Youngjae Yu, Sungjoon Choi

Arxiv

ACL2023

# NLP # Chain-of-thought

Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step

Liunian Harold Li, Jack Hessel, Youngjae Yu, Xiang Ren, Kai-Wei Chang, Yejin Choi

Arxiv

NeurIPS2023

# Computer Vision # NLP # augmentation # interleaved image+text

Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved With Text

Wanrong Zhu, Jack Hessel, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Youngjae Yu, Ludwig Schmidt, William Yang Wang, Yejin Choi

Arxiv

ICCV2023

# Computer Vision # NLP # Conversation # Visual Context

CHAMPAGNE: Learning Real-world Conversation from Large-Scale Web Videos

Seungju Han, Jack Hessel, Nouha Dziri, Yejin Choi, Youngjae Yu

Arxiv

CVPR2023

# NLP # Computer Vision # Reinforcement Learning # Zero-shot

Fusing Pre-trained Language Models with Multimodal Prompts through Reinforcement Learning

Youngjae Yu, Jiwan Chung, Heeseung Yun, Jack Hessel, Jae Sung Park, Ximing Lu, Prithviraj Ammanabrolu, Rowan Zellers, Ronan Le Bras, Gunhee Kim, Yejin Choi

Arxiv

ICRA2023

# Computer Vision # Robotics # Object Detection

Zero-shot Active Visual Search (ZAVIS): Intelligent Object Search for Robotic Assistants

Jeongeun Park, Taerim Yoon, Jejoon Hong, Youngjae Yu, Matthew Pan, Sungjoon Choi

Arxiv