ETS published an analysis in a paper titled LLM-based conversational assessments introducing semantic variability in LLM-generated replies to the same discourse, both across repeated calls to the same LLM and across different LLMs.
Published
Signal category
Research & Knowledge
Quote
“LLM-based conversational assessments introduce a fundamental tension between adaptivity and standardization. In this paper (https://lnkd.in/gup3RMGq), we provide an initial analysis of the semantic variability of LLM-generated replies to the same discourse, both across repeated calls to the same LLM and across different LLMs.”
— Jiangang Hao|ETS team
Company
ETS
Advancing the science of measurement to power human progress.
- Industry
- Education Administration Programs
- Location
- Princeton, US
- Company size
- 8,348 employees
ETS is a global education and talent solutions organization, enabling lifelong learners to be future ready. We advance the science of measurement to build the benchmarks for fair and valid skill assessment. We are committed to powering human progress by promoting skill proficiency, empowering upward mobility, and unlocking more opportunities for everyone, everywhere. For more than 75 years, we have developed products and services based on rigorous research and our belief in the power of learning. Our portfolio consists of some of the world’s most trusted assessments - including the TOEFL® and TOEIC® exams, the GRE® and the Praxis® family of assessments - along with research, development and innovation that explores new frontiers in learning and measurement. Our educational measurement solutions and research contribute to the development of new methods and tools, inform important dialogue with education policymakers, and shed light on critical issues and potential solutions - all with the aim of creating a world where all learners are able to improve their lives through education. Visit www.ets.org to learn more.