Competence vs. Performance: Summary

[Gary Marcus]

The interesting cases in chess are not ones in which experts violate  
the rules (exceedingly rare),  but in which they violate their own  
standards of behavior (ie, fail to take into account his/her own  
stored representations), which is to say that blunders are (virtually  
by definition) cases in which performance deviates from competence.  
Here's a nice description from wikipedia:
In chess, a blunder is a very bad move which is quickly recognised as  
a very bad move by the player who made it, typically before or  
directly after his opponent has made his reply move. They are usually  
caused by some oversight, whether from time pressure ...,  
overconfidence or carelessness.

A theory of pure use would not allow for such a distinction; my only  
claim is that an adequate theory of psychology ought to.

If there's further interest in this topic, I suggest we move it to a  
Google group.

-- gfm

On Oct 17, 2007, at 7:33 PM, Anat Ninio wrote:

> I'm sure one can come up with a well-articulated theory of  
> psychology that won't scare us all with the dark shadow of  
> behaviorism and still won't make an elementary mistake of  
> identifying the rules of chess with a player's playing skill.  This  
> is what the competence/performance distinction is really about,  
> isn't it?  The pure rules versus the impure human behaviour?
>
> Take note that even when a player is making a mistake, he is using  
> -- not terribly successfully -- his stored representation of the  
> game's rules and his cumulated playing strategies.  Why would you  
> not see this as competence, I don't really understand.  Stored  
> competence can be partial, full of holes, context-dependent -- in  
> fact, it is just (messy) potential for action, nothing more rule- 
> like than that.  I'm sure you'll agree to that, at the least.
>
> Anat Ninio
>
>
> Gary Marcus wrote:
>> So far as I know, the only well-articulated theory of psychology  
>> in which a competence-performance distinction does not arise is  
>> behaviorism, and in that connection I still find Chomsky's 1959  
>> arguments to be compelling.  If we are to talk of internal  
>> representations -- and I believe we must -- we must have a theory  
>> of how those representations relate to behavior; I don't see how  
>> to do so coherently without a competence-performance distinction.
>>
>> Please note, by the way, that a commitment to distinguish  
>> competence from performance per se does not commit one to any  
>> particular theoretical apparatus; one need not, for example, be a  
>> nativist about the rules of chess to think that a competence- 
>> performance distinction applies -- and does useful work there   
>> (e.g., in distinguishing "blunders" from more considered  
>> miscalculations).
>>
>> When we discuss the merits of Chomsky's various proposals, it is  
>> essential that we judge each of those hypotheses on its own merits.
>>
>>  -- gfm
>>
>>
>> On Oct 17, 2007, at 6:39 PM, Anat Ninio wrote:
>>
>>> I disagree strongly, dear Gary, with your untested assumption  
>>> that "in psychology, and in linguistics, we are continually left  
>>> with the daunting challenge of inferring underlying  
>>> representations from surface behavior".  Just imagine for a  
>>> minute that this rationalistic perception of psychology and  
>>> linguistics is in effect incorrect and misleading -- as well as  
>>> leading us to wasted decades of research on transformations, deep  
>>> structures, and innate parameters.  There are other ways to look  
>>> at the organization of reality, and my only hope is that the next  
>>> generation of theoreticians of both disciplines will not repeat  
>>> the mistakes of the last one.
>>>
>>> Anat Ninio
>>> Professor of Psychology
>>> The Hebrew University of Jerusalem
>>>
>>>
>>>
>>> Gary Marcus wrote:
>>>>
>>>> I couldn't disagree more strenuously with this last posting. To  
>>>> ascribe both "spinned" and "spun" to competence is to miss the  
>>>> point of the distinction, not to undermine it. "Spinned" and  
>>>> "spun" simply aren't on a par; people accept the latter as  
>>>> correct and ascribe the former to a mistake.
>>>>
>>>> It makes perfect sense to collect data on both (I did just that  
>>>> in 1992 SRCD Monograph on overregularizations, and I have cited  
>>>> Stemberger's data in this connection), but no sense whatsoever  
>>>> to treat the two as if they are on equal footing. And I say this  
>>>> not as a matter of prescriptive linguistics (what the rules of  
>>>> language "ought to be", according to some self-appointed  
>>>> grammarian) but as a matter of descriptive psycholinguistics, as  
>>>> a characterization of what speakers themselves believe.
>>>>
>>>> When I inadvertently forget to carry a 1 and report that the sum  
>>>> of 87 and 24 is 101, we need to able to distinguish my transient  
>>>> error from my general understanding of what would constitute a  
>>>> correct sum (viz. 111).  A competence-performance distinction  
>>>> gives us a theoretical tool with which to make that distinction;  
>>>> to say that 101 and 111 are equally good answers ("since I have  
>>>> the 'competence' to utter either number") would be nothing more  
>>>> than sophistry.
>>>>
>>>> The competence-performance distinction does, as others have  
>>>> noted, have any number of difficulties, but most are  
>>>> methodological ("how can I tell which is which", a fact that  
>>>> will only reveal itself once we have properly carved nature at  
>>>> it joints), not theoretical.
>>>>
>>>> In psychology, and in linguistics, we are continually left with  
>>>> the daunting challenge of inferring underlying representations  
>>>> from surface behavior; there is no way we can conceivably  
>>>> succeed at that task without a firm understanding of the fact  
>>>> that the mapping between the two is often indirect: behavior is  
>>>> dictated not only by underlying representations but by a host of  
>>>> other factors.  Inasmuch as the competence-performance  
>>>> distinction encapsulates that fundamental truth, it is a  
>>>> valuable tool that we cannot afford to dismiss.
>>>>
>>>> -- Gary Marcus
>>>> Professor of Psychology
>>>> New York University
>>>>
>>>>
>>>>
>>>> On Oct 17, 2007, at 1:26 PM, Anat Ninio wrote:
>>>>
>>>>> Hello Everybody,
>>>>>
>>>>> I have a feeling -- following Joe Stemberger's excellent  
>>>>> contributions to this strand -- that we're all a little scared  
>>>>> stating the obvious, which is that the notion of competence as  
>>>>> a separate entity from performance is a philosophical error,  
>>>>> pure and simple. If under time stress people say in 4% of the  
>>>>> time "spinned" instead of "spun", then they possess a  
>>>>> competence to do so. Competence is -- in plain English -- an a  
>>>>> priori stored potential or ability to produce some behaviour,  
>>>>> and any other treatment of it, by Chomsky or whoever, is at  
>>>>> best a mystification of the obvious.
>>>>>
>>>>> I agree with Joe that we should simply proceed with doing  
>>>>> research and collect information on what people actually say,  
>>>>> whether children or adults, and don't heed the voices saying  
>>>>> that what we hear is "merely" this or "merely" that, when any  
>>>>> philosopher major in their first year would tell us how  
>>>>> confused the whole idea of competence/performance distinction is.
>>>>>
>>>>> The best,
>>>>>
>>>>> Anat Ninio
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Joe Stemberger wrote:
>>>>>> Hello, everyone.
>>>>>>
>>>>>> Just to start, I'd like to mention that I've spent all of my  
>>>>>> career working on speech errors in adults (both spontaneous  
>>>>>> speech and experimental tasks), and so I accept the notion  
>>>>>> that there are outputs that are "correct" but that performance  
>>>>>> factors can lead to "incorrect" outputs. But that doesn't mean  
>>>>>> that "the" competence-performance distinction gets us anywhere  
>>>>>> practical.
>>>>>>
>>>>>> Defenses of "the" competence-performance distinction are  
>>>>>> missing two main points:
>>>>>> (1) that the exact division between what is competence and  
>>>>>> what is performance, as well as the criteria that distinguish  
>>>>>> them, are largely unknown after more than 40 years.
>>>>>> (2) There is no clear way to test competence except through  
>>>>>> performance.
>>>>>>
>>>>>> Consider John Limber's quote:
>>>>>> "Inferences about linguistic competence in children are  
>>>>>> typically based on spontaneous speech.
>>>>>> This poses a problem since we know that other factors are also  
>>>>>> involved in speech production."
>>>>>>
>>>>>> Well, yes, there are performance factors in language production.
>>>>>> Just as there are performance factors in language perception,  
>>>>>> language comprehension, grammaticality judgments, etc.
>>>>>> In the real world, we can observe only performance, and all  
>>>>>> acts of any sort are "contaminated" by performance.
>>>>>> And it follows from that that inferences about competence must  
>>>>>> come from performance, only and always.
>>>>>>
>>>>>>
>>>>>> OK, so now let's get concrete.
>>>>>> Observe a native speaker of English of any age for long  
>>>>>> enough, and you will find them saying *SPINNED instead of SPUN.
>>>>>> Ask them about it, and they'll tell you that it was an error.
>>>>>> In typical speeded experimental situations, undergraduates  
>>>>>> produce such errors about 4% of the time in neutral contexts.
>>>>>> (And in an experiment that I'm just finishing up, where they  
>>>>>> produce coordinated verbs, the error GRINNED AND *SPINNED  
>>>>>> (with rhyming regular in the first word) shoots up to about  
>>>>>> 25% of tokens.)
>>>>>> Young children produce such outputs from an early age, but  
>>>>>> every study has shown that such overregularizations are in the  
>>>>>> minority for most children (and most irregular verbs) from the  
>>>>>> beginning, and that, like in adult speech, the frequency of  
>>>>>> the verb is one of the predictors of error rate.
>>>>>> This is a generally viewed as a performance phenomenon at all  
>>>>>> ages.
>>>>>> But yet it is cited all the time as a nice deomstration that  
>>>>>> patterns are being extracted and generalized.
>>>>>>
>>>>>> At the same time, speakers will also occasionally produce,  
>>>>>> instead of SPUN, *SPAN. In experimental situations, such  
>>>>>> errors are easy to come by.
>>>>>> As well as things like *GRUN instead of regular GRINNED. (In  
>>>>>> that experiment that I'm just finishing up, *GRUN occurs about  
>>>>>> 2% of the time in neutral contexts,
>>>>>> rising to about 6% in SPUN AND *GRUN (with a rhyming irregular  
>>>>>> in the first word).)
>>>>>> Bybee & Moder 1983 showed that irregular patterns generalize  
>>>>>> to novel words at much greater rates.
>>>>>> So irregular patterns also show generalization.
>>>>>>
>>>>>> Consider another of John's quotes:
>>>>>> "Does anyone really doubt that the language one observes is  
>>>>>> but a subset of the language one might observe under such and  
>>>>>> such conditions?"
>>>>>>
>>>>>> So, if we MUST conclude that competence extracts and  
>>>>>> encapsulates explicit procedures (such as rules) to occount  
>>>>>> for generalization, then competence contains such procedures  
>>>>>> for creating both regular and irregular forms, because both  
>>>>>> types of patterns overgeneralize, right?
>>>>>>
>>>>>> Steve Pinker, Harald Clausen, and others taking a similar  
>>>>>> approach, have argued that the overgeneralization of the -/ed/  
>>>>>> pattern involves the use of a rule after failure to access an  
>>>>>> irregular form, but that overregularization of irregular  
>>>>>> patterns is a performance phenomenon, based on the way that  
>>>>>> irregular forms are stored in and accessed from the lexicon.
>>>>>> Which leads to the conclusion that generalization of patterns  
>>>>>> can occur for performance reasons, even in approaches that  
>>>>>> accept "the" competence-performance distinction.
>>>>>>
>>>>>> Which leads to this possibility:
>>>>>> all inflected forms that a speaker has been been exposed to  
>>>>>> are simply stored in the lexicon.
>>>>>> All generalization, even of regular patterns, is a performance  
>>>>>> phenomenon.
>>>>>> And while it's true that the -/ED/ pattern overgeneralizes  
>>>>>> more than any other pattern, we could attribute /that/  
>>>>>> difference to performance factors, right?
>>>>>>
>>>>>> Or even
>>>>>> The purpose of a grammar is to enumerate the sentences of the  
>>>>>> language.
>>>>>> Actual grammars consist of storing every sentence that has  
>>>>>> ever been observed.
>>>>>> Generalization is just a performance phenomenon across stored  
>>>>>> exemplars.
>>>>>> You want to know in detail how generalization occurs?
>>>>>> Hey, that's a performance phenomenon. It lies outside the  
>>>>>> proper bounds of linguistic theory, and so it's not my  
>>>>>> responsibility to show how it works.
>>>>>>
>>>>>> And what I personally would want to see is some formal proof  
>>>>>> that "the" competence-performance distinction couldn't lead us  
>>>>>> to that sort of system just as easily as it has led us  
>>>>>> anywhere else.
>>>>>>
>>>>>> >From a practical perspective, it seems to me that, since we  
>>>>>> have to work with performance data anyway, we want to work  
>>>>>> with as wide a range of types of data as possible.
>>>>>> And we want to develop theories that account for all of those  
>>>>>> data, in detail.
>>>>>> If there's a distinction between competence and performance,  
>>>>>> that's fine, but it has to be explicit, and our theories need  
>>>>>> to explain exactly which phenomena are due to competence,  
>>>>>> which are due to performance, and why. And if there are  
>>>>>> aspects of data that aren't accounted for in detail, it means  
>>>>>> that we should be uncomfortable, because we need to account  
>>>>>> for it all.
>>>>>>
>>>>>> That isn't the way that "the" competence-performance  
>>>>>> distinction has been used in the past.
>>>>>> If it had been, that would've been fine, as far as I'm concerned.
>>>>>> While some theoretical linguists feel it's fine to apply  
>>>>>> linguistic theory to child language, few are open to using  
>>>>>> child language to provide the tie-breaker for choosing between  
>>>>>> two theoretical mechanisms.
>>>>>>
>>>>>> In the meantime, I'll happily go on studying performance and  
>>>>>> working on theories of how human language works, including  
>>>>>> claims about the fundamental orgainization of the language  
>>>>>> system.
>>>>>>
>>>>>> As should we all.
>>>>>>
>>>>>>
>>>>>> ---Joe Stemberger
>>>>>> Linguistics
>>>>>> UBC
>>>>>>
>>>>>>
>>>>>> john limber pravi:
>>>>>>>
>>>>>>> On 10/16/07 6:46 AM, "Matthew Saxton" <M.Saxton at ioe.ac.uk>  
>>>>>>> <mailto:M.Saxton at ioe.ac.uk> wrote:
>>>>>>>
>>>>>>>     “No-one has stepped forward to defend the competence- 
>>>>>>> performance
>>>>>>>     distinction, or even to offer supportive references.”
>>>>>>>
>>>>>>> OK— try this—with references too!
>>>>>>>
>>>>>>> The basic idea is so widespread in science that it hardly  
>>>>>>> needs defense— frictionless bodies, stimulus generalization,  
>>>>>>> latent learning, stereotype bias.... and linguistic  
>>>>>>> competence all are more or less scientific concepts designed  
>>>>>>> to variously explain conditional performance.
>>>>>>>
>>>>>>> Does anyone really doubt that the language one observes is  
>>>>>>> but a subset of the language one might observe under such and  
>>>>>>> such conditions? And that much of that observed language is  
>>>>>>> fragmentary and ill-formed? While every case demands its own  
>>>>>>> explanatory story, to toss out the whole idea of competence  
>>>>>>> or similar concepts sounds like a lame return to behaviorism.
>>>>>>>
>>>>>>> Years ago I spent a lot of time on the development of complex  
>>>>>>> sentences (Limber, 1973). There was one gap in the thousands  
>>>>>>> of two to three year old children’s utterances I observed --  
>>>>>>> a lack of relative clauses attached to subject NPs.
>>>>>>>
>>>>>>> Did this mean these kids didn’t have the potential for those  
>>>>>>> structures in their behavioral repertoire—their linguistic  
>>>>>>> competence? Here’s the abstract of my answer (Limber, 1976)— 
>>>>>>> which curiously in connection with this current discussion,  
>>>>>>> involves pragmatics.
>>>>>>>
>>>>>>> “Inferences about linguistic competence in children are  
>>>>>>> typically based on spontaneous speech.
>>>>>>> This poses a problem since we know that other factors are  
>>>>>>> also involved in speech production.
>>>>>>> Children who may use complex object and adverbial NPs do not  
>>>>>>> use complex subject NPs. Is
>>>>>>> this a competence deficit, a performance problem, or simply a  
>>>>>>> reflection of pragmatic factors?
>>>>>>> Evidence presented here suggests that children probably do  
>>>>>>> not need complex subjects. An
>>>>>>> extensive use of pronouns in subject but not object position  
>>>>>>> indicates that pragmatics may
>>>>>>> account for the distribution of clauses in their speech. A  
>>>>>>> similar pattern in adult speech indicates there is no warrant  
>>>>>>> to conclude children's lack of subject clauses reflects  
>>>>>>> anything more than the nature of spontaneous speech.”
>>>>>>>
>>>>>>>
>>>>>>> In fact, in all my data of several thousand utterances of  
>>>>>>> children and adults, only TWO subject NPs showed up—one shaky  
>>>>>>> example from a three-year old and another from an adult. The  
>>>>>>> probability that a child is exposed to a subject NP is, from  
>>>>>>> my data, less than 1/1000. Here are the two:
>>>>>>>
>>>>>>> Adult: “ Well these buses that I've had today have been  
>>>>>>> really weird.”
>>>>>>>
>>>>>>> Child: “I think that the girl ... that's here ... doesn't ...  
>>>>>>> she doesn't want me to open it. “
>>>>>>>
>>>>>>> I wonder how the Nuevo-Statistical approaches to language  
>>>>>>> acquisition would handle this?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Limber, J. (1973). The genesis of complex sentences. In T.  
>>>>>>> Moore (Ed.), Cognitive Development and the Acquisition of  
>>>>>>> Language (pp. 169-186). New York: Academic Press.
>>>>>>> http://pubpages.unh.edu/~jel/JLimber/ 
>>>>>>> Genesis_complex_sentences.pdf <http://pubpages.unh.edu/%7Ejel/ 
>>>>>>> JLimber/Genesis_complex_sentences.pdf>
>>>>>>>
>>>>>>> Limber, J. (1976). Unravelling competence, performance, and  
>>>>>>> pragmatics in the speech of young children. Journal of Child  
>>>>>>> Language, 3, 309-318.
>>>>>>> http://pubpages.unh.edu/~jel/JLimber/ 
>>>>>>> pragmatics_performance.pdf <http://pubpages.unh.edu/%7Ejel/ 
>>>>>>> JLimber/pragmatics_performance.pdf>
>>>>>>>
>>>>>>> John Limber
>>>>>>> University of New Hampshire
>>>>>>> Durham NH
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I think that the girl ... that's here ...
>>>>>>> doesn't ... she doesn't want me to open it.
>>>>>>>
>>>>>>> Well these
>>>>>>> buses that I've had today have been really weird.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/info-childes/attachments/20071017/5a44999c/attachment.htm>