Undergraduate Research Mentorship
Status: Completed
Two undergraduate mentoring programs examined how models perform across languages and cultural settings. Eight teams worked on multilingual evaluation; four students studied cultural representation. Four of the projects grew into published benchmarks.
Multilingual evaluation
This program ran for a semester in 2023. Each of the eight teams paired its own low-resource or non-English language, from Urdu and Bengali to Korean, Russian, and Indonesian, with a capability to test: chain-of-thought reasoning, sentiment analysis, question answering, named-entity recognition, or standardized-exam performance. The evaluations revealed weaknesses in model performance and cultural grounding across the languages studied. Two teams carried their work to published benchmarks:
- BEnQA: A Question Answering Benchmark for Bengali and English (Findings of ACL 2024)
- CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean (LREC-COLING 2024)
Cultural evaluation
The second program moved from language to culture. Four students evaluated cultural representation in text-to-image and multimodal models, and their work became two publications:
- Diffusion Models Through a Global Lens: Are They Culturally Inclusive? (ACL 2025, Oral)
- When Tom Eats Kimchi: Evaluating Cultural Awareness of Multimodal Large Language Models in Cultural Mixture Contexts (C3NLP @ NAACL 2025, Outstanding Paper)