AI Models Still Can't Get Visual Jokes: Top Systems Miss 28% of Contradictory Humor

This is a Plain English Papers summary of a research paper called AI Models Still Can't Get Visual Jokes: Top Systems Miss 28% of Contradictory Humor. If you like these kinds of analysis, you should join AImodels.fyi or follow us on Twitter. Overview Research examines if AI models can understand contradictory humor in images Introduces YesBut, a dataset of 1,080 image pairs with contradictory elements Tests ability of AI models to perform visual comparative reasoning Evaluates 11 leading vision-language models like GPT-4V and Claude 3 Models struggle with recognizing contradictions, with best accuracy only 72% Proposes new protocol for improved evaluation of visual understanding Plain English Explanation AI systems are getting better at understanding images, but can they catch subtle jokes or contradictions? This paper explores whether modern AI models can understand humor that relies on contradictions - the kind where something appears one way but actually contradicts itself. ... Click here to read the full summary of this paper

Apr 7, 2025 - 11:25
 0
AI Models Still Can't Get Visual Jokes: Top Systems Miss 28% of Contradictory Humor

This is a Plain English Papers summary of a research paper called AI Models Still Can't Get Visual Jokes: Top Systems Miss 28% of Contradictory Humor. If you like these kinds of analysis, you should join AImodels.fyi or follow us on Twitter.

Overview

  • Research examines if AI models can understand contradictory humor in images
  • Introduces YesBut, a dataset of 1,080 image pairs with contradictory elements
  • Tests ability of AI models to perform visual comparative reasoning
  • Evaluates 11 leading vision-language models like GPT-4V and Claude 3
  • Models struggle with recognizing contradictions, with best accuracy only 72%
  • Proposes new protocol for improved evaluation of visual understanding

Plain English Explanation

AI systems are getting better at understanding images, but can they catch subtle jokes or contradictions? This paper explores whether modern AI models can understand humor that relies on contradictions - the kind where something appears one way but actually contradicts itself.
...

Click here to read the full summary of this paper