Penn Center on Media, Technology and Democracy published research by Dan Roth digging into why asking an LLM the same question two different ways is a cheap, effective way to catch when it doesn't actually know the answer.

The Penn Center on Media, Technology and Democracy is committed to independent research on the information ecosystem and its impact on democracy. Our mission is to advance our collective understanding of the information landscape through cutting-edge science—then leverage that research to foster a more informed society and strengthen the foundations of democracy. We focus on the media organizations and the online platforms, the incentives and the algorithms, that determine which ideas and beliefs spread, and which don’t. The Center’s faculty and scholars are distinguished by their use of emerging technology for data-driven research of the information ecosystem, including novel data collections, large-scale cloud computing, and innovative applications of AI.