[Ads-l] Phrase: Jagged Intelligence (description applied to AI systems)

[ADSGarson O'Toole]

Al systems currently display an odd mixture of superhuman capabilities
and pathetic incompetence. In 2024 researcher Andrej Karpathy employed
the term "jagged intelligence" to describe this behavior. In reply to
Karpathy's message Ethan Mollick pointed to a 2023 paper he
co-authored which contained a pertinent phrase:

[Begin excerpt]
We suggest that the capabilities of AI create a "jagged technological
frontier" where some tasks are easily done by AI, while others, though
seemingly similar in difficulty level, are outside the current
capability of AI.
[End excerpt]

Thus, Karpathy's coinage was probably inspired by Mollick et al's paper.

Xtweet from: Andrej Karpathy @karpathy
Timestamp: 1:50 PM  Jul 25, 2024
https://x.com/karpathy/status/1816531576228053133
[Begin xtweet excerpt]
Jagged Intelligence

The word I came up with to describe the (strange, unintuitive) fact
that state of the art LLMs can both perform extremely impressive tasks
(e.g. solve complex math problems) while simultaneously struggle with
some very dumb problems.

E.g. example from two days ago - which number is bigger, 9.11 or  9.9? Wrong.

https://x.com/karpathy/status/1815549255354089752
[End xtweet excerpt]

Xtweet from: Ethan Mollick @emollick
Timestamp: 1:53 PM Jul 25, 2024
https://x.com/emollick/status/1816532255076204799
[Begin xtweet]
We wrote a paper on this last year based on data from a randomized
controlled trial calling this the "jagged frontier" of AI.
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4573321
[End xtweet]

Paper title: Navigating the Jagged Technological Frontier: Field
Experimental Evidence of the Effects of AI on Knowledge Worker
Productivity and Quality
Posting date: 18 Sep 2023
Authors: Fabrizio Dell'Acqua, Edward McFowland III, Ethan R. Mollick,
Hila Lifshitz-Assaf, Katherine Kellogg, Saran Rajendran, Lisa Krayer,
François Candelon, Karim R. Lakhani
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4573321

Xtweet from: Andrej Karpathy @karpathy
Timestamp: 2:13 PM Jul 25, 2024
https://x.com/karpathy/status/1816537376212369688
[Begin xtweet]
Hah nice!! Yes exactly. I'm really doubting myself now maybe I had
come across it :D
[End xtweet]

Garson O'Toole

------------------------------------------------------------
The American Dialect Society - http://www.americandialect.org