Competence vs. Performance: Summary

[Gary Marcus]

So far as I know, the only well-articulated theory of psychology in  
which a competence-performance distinction does not arise is  
behaviorism, and in that connection I still find Chomsky's 1959  
arguments to be compelling.  If we are to talk of internal  
representations -- and I believe we must -- we must have a theory of  
how those representations relate to behavior; I don't see how to do  
so coherently without a competence-performance distinction.

Please note, by the way, that a commitment to distinguish competence  
from performance per se does not commit one to any particular  
theoretical apparatus; one need not, for example, be a nativist about  
the rules of chess to think that a competence-performance distinction  
applies -- and does useful work there  (e.g., in distinguishing  
"blunders" from more considered miscalculations).

When we discuss the merits of Chomsky's various proposals, it is  
essential that we judge each of those hypotheses on its own merits.

  -- gfm


On Oct 17, 2007, at 6:39 PM, Anat Ninio wrote:

> I disagree strongly, dear Gary, with your untested assumption that  
> "in psychology, and in linguistics, we are continually left with  
> the daunting challenge of inferring underlying representations from  
> surface behavior".  Just imagine for a minute that this  
> rationalistic perception of psychology and linguistics is in effect  
> incorrect and misleading -- as well as leading us to wasted decades  
> of research on transformations, deep structures, and innate  
> parameters.  There are other ways to look at the organization of  
> reality, and my only hope is that the next generation of  
> theoreticians of both disciplines will not repeat the mistakes of  
> the last one.
>
> Anat Ninio
> Professor of Psychology
> The Hebrew University of Jerusalem
>
>
>
> Gary Marcus wrote:
>>
>> I couldn't disagree more strenuously with this last posting. To  
>> ascribe both "spinned" and "spun" to competence is to miss the  
>> point of the distinction, not to undermine it. "Spinned" and  
>> "spun" simply aren't on a par; people accept the latter as correct  
>> and ascribe the former to a mistake.
>>
>> It makes perfect sense to collect data on both (I did just that in  
>> 1992 SRCD Monograph on overregularizations, and I have cited  
>> Stemberger's data in this connection), but no sense whatsoever to  
>> treat the two as if they are on equal footing. And I say this not  
>> as a matter of prescriptive linguistics (what the rules of  
>> language "ought to be", according to some self-appointed  
>> grammarian) but as a matter of descriptive psycholinguistics, as a  
>> characterization of what speakers themselves believe.
>>
>> When I inadvertently forget to carry a 1 and report that the sum  
>> of 87 and 24 is 101, we need to able to distinguish my transient  
>> error from my general understanding of what would constitute a  
>> correct sum (viz. 111).  A competence-performance distinction  
>> gives us a theoretical tool with which to make that distinction;  
>> to say that 101 and 111 are equally good answers ("since I have  
>> the 'competence' to utter either number") would be nothing more  
>> than sophistry.
>>
>> The competence-performance distinction does, as others have noted,  
>> have any number of difficulties, but most are methodological ("how  
>> can I tell which is which", a fact that will only reveal itself  
>> once we have properly carved nature at it joints), not theoretical.
>>
>> In psychology, and in linguistics, we are continually left with  
>> the daunting challenge of inferring underlying representations  
>> from surface behavior; there is no way we can conceivably succeed  
>> at that task without a firm understanding of the fact that the  
>> mapping between the two is often indirect: behavior is dictated  
>> not only by underlying representations but by a host of other  
>> factors.  Inasmuch as the competence-performance distinction  
>> encapsulates that fundamental truth, it is a valuable tool that we  
>> cannot afford to dismiss.
>>
>> -- Gary Marcus
>> Professor of Psychology
>> New York University
>>
>>
>>
>> On Oct 17, 2007, at 1:26 PM, Anat Ninio wrote:
>>
>>> Hello Everybody,
>>>
>>> I have a feeling -- following Joe Stemberger's excellent  
>>> contributions to this strand -- that we're all a little scared  
>>> stating the obvious, which is that the notion of competence as a  
>>> separate entity from performance is a philosophical error, pure  
>>> and simple. If under time stress people say in 4% of the time  
>>> "spinned" instead of "spun", then they possess a competence to do  
>>> so. Competence is -- in plain English -- an a priori stored  
>>> potential or ability to produce some behaviour, and any other  
>>> treatment of it, by Chomsky or whoever, is at best a  
>>> mystification of the obvious.
>>>
>>> I agree with Joe that we should simply proceed with doing  
>>> research and collect information on what people actually say,  
>>> whether children or adults, and don't heed the voices saying that  
>>> what we hear is "merely" this or "merely" that, when any  
>>> philosopher major in their first year would tell us how confused  
>>> the whole idea of competence/performance distinction is.
>>>
>>> The best,
>>>
>>> Anat Ninio
>>>
>>>
>>>
>>>
>>>
>>> Joe Stemberger wrote:
>>>> Hello, everyone.
>>>>
>>>> Just to start, I'd like to mention that I've spent all of my  
>>>> career working on speech errors in adults (both spontaneous  
>>>> speech and experimental tasks), and so I accept the notion that  
>>>> there are outputs that are "correct" but that performance  
>>>> factors can lead to "incorrect" outputs. But that doesn't mean  
>>>> that "the" competence-performance distinction gets us anywhere  
>>>> practical.
>>>>
>>>> Defenses of "the" competence-performance distinction are missing  
>>>> two main points:
>>>> (1) that the exact division between what is competence and what  
>>>> is performance, as well as the criteria that distinguish them,  
>>>> are largely unknown after more than 40 years.
>>>> (2) There is no clear way to test competence except through  
>>>> performance.
>>>>
>>>> Consider John Limber's quote:
>>>> "Inferences about linguistic competence in children are  
>>>> typically based on spontaneous speech.
>>>> This poses a problem since we know that other factors are also  
>>>> involved in speech production."
>>>>
>>>> Well, yes, there are performance factors in language production.
>>>> Just as there are performance factors in language perception,  
>>>> language comprehension, grammaticality judgments, etc.
>>>> In the real world, we can observe only performance, and all acts  
>>>> of any sort are "contaminated" by performance.
>>>> And it follows from that that inferences about competence must  
>>>> come from performance, only and always.
>>>>
>>>>
>>>> OK, so now let's get concrete.
>>>> Observe a native speaker of English of any age for long enough,  
>>>> and you will find them saying *SPINNED instead of SPUN.
>>>> Ask them about it, and they'll tell you that it was an error.
>>>> In typical speeded experimental situations, undergraduates  
>>>> produce such errors about 4% of the time in neutral contexts.
>>>> (And in an experiment that I'm just finishing up, where they  
>>>> produce coordinated verbs, the error GRINNED AND *SPINNED (with  
>>>> rhyming regular in the first word) shoots up to about 25% of  
>>>> tokens.)
>>>> Young children produce such outputs from an early age, but every  
>>>> study has shown that such overregularizations are in the  
>>>> minority for most children (and most irregular verbs) from the  
>>>> beginning, and that, like in adult speech, the frequency of the  
>>>> verb is one of the predictors of error rate.
>>>> This is a generally viewed as a performance phenomenon at all ages.
>>>> But yet it is cited all the time as a nice deomstration that  
>>>> patterns are being extracted and generalized.
>>>>
>>>> At the same time, speakers will also occasionally produce,  
>>>> instead of SPUN, *SPAN. In experimental situations, such errors  
>>>> are easy to come by.
>>>> As well as things like *GRUN instead of regular GRINNED. (In  
>>>> that experiment that I'm just finishing up, *GRUN occurs about  
>>>> 2% of the time in neutral contexts,
>>>> rising to about 6% in SPUN AND *GRUN (with a rhyming irregular  
>>>> in the first word).)
>>>> Bybee & Moder 1983 showed that irregular patterns generalize to  
>>>> novel words at much greater rates.
>>>> So irregular patterns also show generalization.
>>>>
>>>> Consider another of John's quotes:
>>>> "Does anyone really doubt that the language one observes is but  
>>>> a subset of the language one might observe under such and such  
>>>> conditions?"
>>>>
>>>> So, if we MUST conclude that competence extracts and  
>>>> encapsulates explicit procedures (such as rules) to occount for  
>>>> generalization, then competence contains such procedures for  
>>>> creating both regular and irregular forms, because both types of  
>>>> patterns overgeneralize, right?
>>>>
>>>> Steve Pinker, Harald Clausen, and others taking a similar  
>>>> approach, have argued that the overgeneralization of the -/ed/  
>>>> pattern involves the use of a rule after failure to access an  
>>>> irregular form, but that overregularization of irregular  
>>>> patterns is a performance phenomenon, based on the way that  
>>>> irregular forms are stored in and accessed from the lexicon.
>>>> Which leads to the conclusion that generalization of patterns  
>>>> can occur for performance reasons, even in approaches that  
>>>> accept "the" competence-performance distinction.
>>>>
>>>> Which leads to this possibility:
>>>> all inflected forms that a speaker has been been exposed to are  
>>>> simply stored in the lexicon.
>>>> All generalization, even of regular patterns, is a performance  
>>>> phenomenon.
>>>> And while it's true that the -/ED/ pattern overgeneralizes more  
>>>> than any other pattern, we could attribute /that/ difference to  
>>>> performance factors, right?
>>>>
>>>> Or even
>>>> The purpose of a grammar is to enumerate the sentences of the  
>>>> language.
>>>> Actual grammars consist of storing every sentence that has ever  
>>>> been observed.
>>>> Generalization is just a performance phenomenon across stored  
>>>> exemplars.
>>>> You want to know in detail how generalization occurs?
>>>> Hey, that's a performance phenomenon. It lies outside the proper  
>>>> bounds of linguistic theory, and so it's not my responsibility  
>>>> to show how it works.
>>>>
>>>> And what I personally would want to see is some formal proof  
>>>> that "the" competence-performance distinction couldn't lead us  
>>>> to that sort of system just as easily as it has led us anywhere  
>>>> else.
>>>>
>>>> >From a practical perspective, it seems to me that, since we  
>>>> have to work with performance data anyway, we want to work with  
>>>> as wide a range of types of data as possible.
>>>> And we want to develop theories that account for all of those  
>>>> data, in detail.
>>>> If there's a distinction between competence and performance,  
>>>> that's fine, but it has to be explicit, and our theories need to  
>>>> explain exactly which phenomena are due to competence, which are  
>>>> due to performance, and why. And if there are aspects of data  
>>>> that aren't accounted for in detail, it means that we should be  
>>>> uncomfortable, because we need to account for it all.
>>>>
>>>> That isn't the way that "the" competence-performance distinction  
>>>> has been used in the past.
>>>> If it had been, that would've been fine, as far as I'm concerned.
>>>> While some theoretical linguists feel it's fine to apply  
>>>> linguistic theory to child language, few are open to using child  
>>>> language to provide the tie-breaker for choosing between two  
>>>> theoretical mechanisms.
>>>>
>>>> In the meantime, I'll happily go on studying performance and  
>>>> working on theories of how human language works, including  
>>>> claims about the fundamental orgainization of the language system.
>>>>
>>>> As should we all.
>>>>
>>>>
>>>> ---Joe Stemberger
>>>> Linguistics
>>>> UBC
>>>>
>>>>
>>>> john limber pravi:
>>>>>
>>>>> On 10/16/07 6:46 AM, "Matthew Saxton" <M.Saxton at ioe.ac.uk>  
>>>>> <mailto:M.Saxton at ioe.ac.uk> wrote:
>>>>>
>>>>>     “No-one has stepped forward to defend the competence- 
>>>>> performance
>>>>>     distinction, or even to offer supportive references.”
>>>>>
>>>>> OK— try this—with references too!
>>>>>
>>>>> The basic idea is so widespread in science that it hardly needs  
>>>>> defense— frictionless bodies, stimulus generalization, latent  
>>>>> learning, stereotype bias.... and linguistic competence all are  
>>>>> more or less scientific concepts designed to variously explain  
>>>>> conditional performance.
>>>>>
>>>>> Does anyone really doubt that the language one observes is but  
>>>>> a subset of the language one might observe under such and such  
>>>>> conditions? And that much of that observed language is  
>>>>> fragmentary and ill-formed? While every case demands its own  
>>>>> explanatory story, to toss out the whole idea of competence or  
>>>>> similar concepts sounds like a lame return to behaviorism.
>>>>>
>>>>> Years ago I spent a lot of time on the development of complex  
>>>>> sentences (Limber, 1973). There was one gap in the thousands of  
>>>>> two to three year old children’s utterances I observed -- a  
>>>>> lack of relative clauses attached to subject NPs.
>>>>>
>>>>> Did this mean these kids didn’t have the potential for those  
>>>>> structures in their behavioral repertoire—their linguistic  
>>>>> competence? Here’s the abstract of my answer (Limber, 1976)— 
>>>>> which curiously in connection with this current discussion,  
>>>>> involves pragmatics.
>>>>>
>>>>> “Inferences about linguistic competence in children are  
>>>>> typically based on spontaneous speech.
>>>>> This poses a problem since we know that other factors are also  
>>>>> involved in speech production.
>>>>> Children who may use complex object and adverbial NPs do not  
>>>>> use complex subject NPs. Is
>>>>> this a competence deficit, a performance problem, or simply a  
>>>>> reflection of pragmatic factors?
>>>>> Evidence presented here suggests that children probably do not  
>>>>> need complex subjects. An
>>>>> extensive use of pronouns in subject but not object position  
>>>>> indicates that pragmatics may
>>>>> account for the distribution of clauses in their speech. A  
>>>>> similar pattern in adult speech indicates there is no warrant  
>>>>> to conclude children's lack of subject clauses reflects  
>>>>> anything more than the nature of spontaneous speech.”
>>>>>
>>>>>
>>>>> In fact, in all my data of several thousand utterances of  
>>>>> children and adults, only TWO subject NPs showed up—one shaky  
>>>>> example from a three-year old and another from an adult. The  
>>>>> probability that a child is exposed to a subject NP is, from my  
>>>>> data, less than 1/1000. Here are the two:
>>>>>
>>>>> Adult: “ Well these buses that I've had today have been really  
>>>>> weird.”
>>>>>
>>>>> Child: “I think that the girl ... that's here ... doesn't ...  
>>>>> she doesn't want me to open it. “
>>>>>
>>>>> I wonder how the Nuevo-Statistical approaches to language  
>>>>> acquisition would handle this?
>>>>>
>>>>>
>>>>>
>>>>> Limber, J. (1973). The genesis of complex sentences. In T.  
>>>>> Moore (Ed.), Cognitive Development and the Acquisition of  
>>>>> Language (pp. 169-186). New York: Academic Press.
>>>>> http://pubpages.unh.edu/~jel/JLimber/ 
>>>>> Genesis_complex_sentences.pdf <http://pubpages.unh.edu/%7Ejel/ 
>>>>> JLimber/Genesis_complex_sentences.pdf>
>>>>>
>>>>> Limber, J. (1976). Unravelling competence, performance, and  
>>>>> pragmatics in the speech of young children. Journal of Child  
>>>>> Language, 3, 309-318.
>>>>> http://pubpages.unh.edu/~jel/JLimber/pragmatics_performance.pdf  
>>>>> <http://pubpages.unh.edu/%7Ejel/JLimber/ 
>>>>> pragmatics_performance.pdf>
>>>>>
>>>>> John Limber
>>>>> University of New Hampshire
>>>>> Durham NH
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I think that the girl ... that's here ...
>>>>> doesn't ... she doesn't want me to open it.
>>>>>
>>>>> Well these
>>>>> buses that I've had today have been really weird.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>
>>
>>
>