2024年2月21日 · Social value alignment evaluates how well the model understands nation-specific social values, while common knowledge alignment examines how well the model captures basic knowledge related to the nation. We constructed KorNAT, the first benchmark that measures national alignment with South Korea.

Kornat (pronounced [kɔ̌rnaːt]) is an island in the Croatian part of the Adriatic Sea, in central Dalmatia. With an area of 32.44 km 2 it is the 16th largest island in Croatia and the largest island in the eponymous Kornati islands archipelago. [1] It is part of the Kornati National Park, which is composed of a total of 89 islands, islets and ...

6 天之前 · We constructed KorNAT, the first benchmark that measures national alignment between LLMs and South Korea. KorNat contains 4K and 6K multiple-choice questions for social value and common knowledge, respectively.

국내 최초로 개발된 한국형 llm 평가 데이터셋 'kornat'을 소개합니다. 한국어 자연어 처리 모델의 성능을 체계적으로 평가하며, 한국 AI 연구의 새로운 기준을 제시하는 중요한 자료입니다.

We construct KorNAT (Korean National Alignment Test), the first benchmark that measures national alignment with South Korea from social values and common knowledge. Here, we only show small set of samples from KorNAT in Korean and their English translations.

We constructed KorNAT, the first benchmark that measures national alignment with South Korea. For the social value dataset, we obtained ground truth labels from a large-scale survey involving 6,174 unique Korean participants.

KorNAT (Korean National Alignment Test) is a benchmark dataset developed to assess the alignment of Large Language Models (LLMs) with South Korean social values and common knowledge. It evaluates models across two primary dimensions:

In this paper, we constructed KorNAT (Korean National Alignment Test), the first benchmark that measures national alignment with South Korea. Samples are in a multiple choice question format, offering five answer choices for social values and four answer choices for common knowledge, as shown in Figure1. For the social value dataset, we

Kornat je najveći otok u otočju Kornati u Jadranskom moru. Otok je dug 25,2 km, a širok je do 2,5 km.

社会价值一致性评估模型对特定国家社会价值观的理解程度，而常识一致性则检查模型捕获与国家相关的基本知识的程度。我们构建了 KorNAT，这是衡量国家与韩国一致性的第一个基准。