Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

EXAONE 3.5: Series of Large Language Models for Real-world Use Cases

About

This technical report introduces the EXAONE 3.5 instruction-tuned language models, developed and released by LG AI Research. The EXAONE 3.5 language models are offered in three configurations: 32B, 7.8B, and 2.4B. These models feature several standout capabilities: 1) exceptional instruction following capabilities in real-world scenarios, achieving the highest scores across seven benchmarks, 2) outstanding long-context comprehension, attaining the top performance in four benchmarks, and 3) competitive results compared to state-of-the-art open models of similar sizes across nine general benchmarks. The EXAONE 3.5 language models are open to anyone for research purposes and can be downloaded from https://huggingface.co/LGAI-EXAONE. For commercial use, please reach out to the official contact point of LG AI Research: contact_us@lgresearch.ai.

Soyoung An, Kyunghoon Bae, Eunbi Choi, Kibong Choi, Stanley Jungkyu Choi, Seokhee Hong, Junwon Hwang, Hyojin Jeon, Gerrard Jeongwon Jo, Hyunjik Jo, Jiyeon Jung, Yountae Jung, Hyosang Kim, Joonkee Kim, Seonghwan Kim, Soyeon Kim, Sunkyoung Kim, Yireun Kim, Yongil Kim, Youchul Kim, Edward Hwayoung Lee, Haeju Lee, Honglak Lee, Jinsik Lee, Kyungmin Lee, Woohyung Lim, Sangha Park, Sooyoun Park, Yongmin Park, Sihoon Yang, Heuiyeen Yeen, Hyeongu Yun• 2024

Related benchmarks

TaskDatasetResultRank
Instruction FollowingIFEval
IFEval Accuracy83.6
625
Paraphrase IdentificationPAWS-X
Accuracy85.24
66
CodingMBPP+
Pass@179.4
52
MathematicsGSM8K
GSM8K Score82.5
39
Trustworthiness evaluationLLM Trustworthiness Benchmark
Bias Score84.5
17
Bias EvaluationKoBBQ
Ambiguous Context Score87.9
17
Natural Language UnderstandingKoBEST
BoolQ Score92.59
13
Multiple-choice Question AnsweringMMLU Redux (test)
Accuracy79.26
13
LLM-generated text detectionKatFish Paper Abstract
AUC-ROC (Solar)70.8
12
LLM-generated text detectionKatFish Essay
AUC-ROC (Solar)92.08
12
Showing 10 of 31 rows

Other info

Follow for update