Competence vs. Performance: Summary

Wed Oct 17 23:33:45 UTC 2007

I'm sure one can come up with a well-articulated theory of psychology 
that won't scare us all with the dark shadow of behaviorism and still 
won't make an elementary mistake of identifying the rules of chess with 
a player's playing skill.  This is what the competence/performance 
distinction is really about, isn't it?  The pure rules versus the impure 
human behaviour?

Take note that even when a player is making a mistake, he is using -- 
not terribly successfully -- his stored representation of the game's 
rules and his cumulated playing strategies.  Why would you not see this 
as competence, I don't really understand.  Stored competence can be 
partial, full of holes, context-dependent -- in fact, it is just (messy) 
potential for action, nothing more rule-like than that.  I'm sure you'll 
agree to that, at the least.

Anat Ninio

Gary Marcus wrote:
> So far as I know, the only well-articulated theory of psychology in 
> which a competence-performance distinction does not arise is 
> behaviorism, and in that connection I still find Chomsky's 1959 
> arguments to be compelling.  If we are to talk of internal 
> representations -- and I believe we must -- we must have a theory of 
> how those representations relate to behavior; I don't see how to do so 
> coherently without a competence-performance distinction.
>
> Please note, by the way, that a commitment to distinguish competence 
> from performance per se does not commit one to any particular 
> theoretical apparatus; one need not, for example, be a nativist about 
> the rules of chess to think that a competence-performance distinction 
> applies -- and does useful work there  (e.g., in distinguishing 
> "blunders" from more considered miscalculations).
>
> When we discuss the merits of Chomsky's various proposals, it is 
> essential that we judge each of those hypotheses on its own merits.
>
>  -- gfm
>
>
> On Oct 17, 2007, at 6:39 PM, Anat Ninio wrote:
>
>> I disagree strongly, dear Gary, with your untested assumption that 
>> "in psychology, and in linguistics, we are continually left with the 
>> daunting challenge of inferring underlying representations from 
>> surface behavior".  Just imagine for a minute that this rationalistic 
>> perception of psychology and linguistics is in effect incorrect and 
>> misleading -- as well as leading us to wasted decades of research on 
>> transformations, deep structures, and innate parameters.  There are 
>> other ways to look at the organization of reality, and my only hope 
>> is that the next generation of theoreticians of both disciplines will 
>> not repeat the mistakes of the last one.
>>
>> Anat Ninio
>> Professor of Psychology
>> The Hebrew University of Jerusalem
>>
>>
>>
>> Gary Marcus wrote:
>>>
>>> I couldn't disagree more strenuously with this last posting. To 
>>> ascribe both "spinned" and "spun" to competence is to miss the point 
>>> of the distinction, not to undermine it. "Spinned" and "spun" simply 
>>> aren't on a par; people accept the latter as correct and ascribe the 
>>> former to a mistake.
>>>
>>> It makes perfect sense to collect data on both (I did just that in 
>>> 1992 SRCD Monograph on overregularizations, and I have cited 
>>> Stemberger's data in this connection), but no sense whatsoever to 
>>> treat the two as if they are on equal footing. And I say this not as 
>>> a matter of prescriptive linguistics (what the rules of language 
>>> "ought to be", according to some self-appointed grammarian) but as a 
>>> matter of descriptive psycholinguistics, as a characterization of 
>>> what speakers themselves believe.
>>>
>>> When I inadvertently forget to carry a 1 and report that the sum of 
>>> 87 and 24 is 101, we need to able to distinguish my transient error 
>>> from my general understanding of what would constitute a correct sum 
>>> (viz. 111).  A competence-performance distinction gives us a 
>>> theoretical tool with which to make that distinction; to say that 
>>> 101 and 111 are equally good answers ("since I have the 'competence' 
>>> to utter either number") would be nothing more than sophistry.
>>>
>>> The competence-performance distinction does, as others have noted, 
>>> have any number of difficulties, but most are methodological ("how 
>>> can I tell which is which", a fact that will only reveal itself once 
>>> we have properly carved nature at it joints), not theoretical.
>>>
>>> In psychology, and in linguistics, we are continually left with the 
>>> daunting challenge of inferring underlying representations from 
>>> surface behavior; there is no way we can conceivably succeed at that 
>>> task without a firm understanding of the fact that the mapping 
>>> between the two is often indirect: behavior is dictated not only by 
>>> underlying representations but by a host of other factors.  Inasmuch 
>>> as the competence-performance distinction encapsulates that 
>>> fundamental truth, it is a valuable tool that we cannot afford to 
>>> dismiss.
>>>
>>> -- Gary Marcus
>>> Professor of Psychology
>>> New York University
>>>
>>>
>>>
>>> On Oct 17, 2007, at 1:26 PM, Anat Ninio wrote:
>>>
>>>> Hello Everybody,
>>>>
>>>> I have a feeling -- following Joe Stemberger's excellent 
>>>> contributions to this strand -- that we're all a little scared 
>>>> stating the obvious, which is that the notion of competence as a 
>>>> separate entity from performance is a philosophical error, pure and 
>>>> simple. If under time stress people say in 4% of the time "spinned" 
>>>> instead of "spun", then they possess a competence to do so. 
>>>> Competence is -- in plain English -- an a priori stored potential 
>>>> or ability to produce some behaviour, and any other treatment of 
>>>> it, by Chomsky or whoever, is at best a mystification of the obvious.
>>>>
>>>> I agree with Joe that we should simply proceed with doing research 
>>>> and collect information on what people actually say, whether 
>>>> children or adults, and don't heed the voices saying that what we 
>>>> hear is "merely" this or "merely" that, when any philosopher major 
>>>> in their first year would tell us how confused the whole idea of 
>>>> competence/performance distinction is.
>>>>
>>>> The best,
>>>>
>>>> Anat Ninio
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Joe Stemberger wrote:
>>>>> Hello, everyone.
>>>>>
>>>>> Just to start, I'd like to mention that I've spent all of my 
>>>>> career working on speech errors in adults (both spontaneous speech 
>>>>> and experimental tasks), and so I accept the notion that there are 
>>>>> outputs that are "correct" but that performance factors can lead 
>>>>> to "incorrect" outputs. But that doesn't mean that "the" 
>>>>> competence-performance distinction gets us anywhere practical.
>>>>>
>>>>> Defenses of "the" competence-performance distinction are missing 
>>>>> two main points:
>>>>> (1) that the exact division between what is competence and what is 
>>>>> performance, as well as the criteria that distinguish them, are 
>>>>> largely unknown after more than 40 years.
>>>>> (2) There is no clear way to test competence except through 
>>>>> performance.
>>>>>
>>>>> Consider John Limber's quote:
>>>>> "Inferences about linguistic competence in children are typically 
>>>>> based on spontaneous speech.
>>>>> This poses a problem since we know that other factors are also 
>>>>> involved in speech production."
>>>>>
>>>>> Well, yes, there are performance factors in language production.
>>>>> Just as there are performance factors in language perception, 
>>>>> language comprehension, grammaticality judgments, etc.
>>>>> In the real world, we can observe only performance, and all acts 
>>>>> of any sort are "contaminated" by performance.
>>>>> And it follows from that that inferences about competence must 
>>>>> come from performance, only and always.
>>>>>
>>>>>
>>>>> OK, so now let's get concrete.
>>>>> Observe a native speaker of English of any age for long enough, 
>>>>> and you will find them saying *SPINNED instead of SPUN.
>>>>> Ask them about it, and they'll tell you that it was an error.
>>>>> In typical speeded experimental situations, undergraduates produce 
>>>>> such errors about 4% of the time in neutral contexts.
>>>>> (And in an experiment that I'm just finishing up, where they 
>>>>> produce coordinated verbs, the error GRINNED AND *SPINNED (with 
>>>>> rhyming regular in the first word) shoots up to about 25% of tokens.)
>>>>> Young children produce such outputs from an early age, but every 
>>>>> study has shown that such overregularizations are in the minority 
>>>>> for most children (and most irregular verbs) from the beginning, 
>>>>> and that, like in adult speech, the frequency of the verb is one 
>>>>> of the predictors of error rate.
>>>>> This is a generally viewed as a performance phenomenon at all ages.
>>>>> But yet it is cited all the time as a nice deomstration that 
>>>>> patterns are being extracted and generalized.
>>>>>
>>>>> At the same time, speakers will also occasionally produce, instead 
>>>>> of SPUN, *SPAN. In experimental situations, such errors are easy 
>>>>> to come by.
>>>>> As well as things like *GRUN instead of regular GRINNED. (In that 
>>>>> experiment that I'm just finishing up, *GRUN occurs about 2% of 
>>>>> the time in neutral contexts,
>>>>> rising to about 6% in SPUN AND *GRUN (with a rhyming irregular in 
>>>>> the first word).)
>>>>> Bybee & Moder 1983 showed that irregular patterns generalize to 
>>>>> novel words at much greater rates.
>>>>> So irregular patterns also show generalization.
>>>>>
>>>>> Consider another of John's quotes:
>>>>> "Does anyone really doubt that the language one observes is but a 
>>>>> subset of the language one might observe under such and such 
>>>>> conditions?"
>>>>>
>>>>> So, if we MUST conclude that competence extracts and encapsulates 
>>>>> explicit procedures (such as rules) to occount for generalization, 
>>>>> then competence contains such procedures for creating both regular 
>>>>> and irregular forms, because both types of patterns 
>>>>> overgeneralize, right?
>>>>>
>>>>> Steve Pinker, Harald Clausen, and others taking a similar 
>>>>> approach, have argued that the overgeneralization of the -/ed/ 
>>>>> pattern involves the use of a rule after failure to access an 
>>>>> irregular form, but that overregularization of irregular patterns 
>>>>> is a performance phenomenon, based on the way that irregular forms 
>>>>> are stored in and accessed from the lexicon.
>>>>> Which leads to the conclusion that generalization of patterns can 
>>>>> occur for performance reasons, even in approaches that accept 
>>>>> "the" competence-performance distinction.
>>>>>
>>>>> Which leads to this possibility:
>>>>> all inflected forms that a speaker has been been exposed to are 
>>>>> simply stored in the lexicon.
>>>>> All generalization, even of regular patterns, is a performance 
>>>>> phenomenon.
>>>>> And while it's true that the -/ED/ pattern overgeneralizes more 
>>>>> than any other pattern, we could attribute /that/ difference to 
>>>>> performance factors, right?
>>>>>
>>>>> Or even
>>>>> The purpose of a grammar is to enumerate the sentences of the 
>>>>> language.
>>>>> Actual grammars consist of storing every sentence that has ever 
>>>>> been observed.
>>>>> Generalization is just a performance phenomenon across stored 
>>>>> exemplars.
>>>>> You want to know in detail how generalization occurs?
>>>>> Hey, that's a performance phenomenon. It lies outside the proper 
>>>>> bounds of linguistic theory, and so it's not my responsibility to 
>>>>> show how it works.
>>>>>
>>>>> And what I personally would want to see is some formal proof that 
>>>>> "the" competence-performance distinction couldn't lead us to that 
>>>>> sort of system just as easily as it has led us anywhere else.
>>>>>
>>>>> >From a practical perspective, it seems to me that, since we have 
>>>>> to work with performance data anyway, we want to work with as wide 
>>>>> a range of types of data as possible.
>>>>> And we want to develop theories that account for all of those 
>>>>> data, in detail.
>>>>> If there's a distinction between competence and performance, 
>>>>> that's fine, but it has to be explicit, and our theories need to 
>>>>> explain exactly which phenomena are due to competence, which are 
>>>>> due to performance, and why. And if there are aspects of data that 
>>>>> aren't accounted for in detail, it means that we should be 
>>>>> uncomfortable, because we need to account for it all.
>>>>>
>>>>> That isn't the way that "the" competence-performance distinction 
>>>>> has been used in the past.
>>>>> If it had been, that would've been fine, as far as I'm concerned.
>>>>> While some theoretical linguists feel it's fine to apply 
>>>>> linguistic theory to child language, few are open to using child 
>>>>> language to provide the tie-breaker for choosing between two 
>>>>> theoretical mechanisms.
>>>>>
>>>>> In the meantime, I'll happily go on studying performance and 
>>>>> working on theories of how human language works, including claims 
>>>>> about the fundamental orgainization of the language system.
>>>>>
>>>>> As should we all.
>>>>>
>>>>>
>>>>> ---Joe Stemberger
>>>>> Linguistics
>>>>> UBC
>>>>>
>>>>>
>>>>> john limber pravi:
>>>>>>
>>>>>> On 10/16/07 6:46 AM, "Matthew Saxton" <M.Saxton at ioe.ac.uk> 
>>>>>> <mailto:M.Saxton at ioe.ac.uk> wrote:
>>>>>>
>>>>>>     “No-one has stepped forward to defend the competence-performance
>>>>>>     distinction, or even to offer supportive references.”
>>>>>>
>>>>>> OK— try this—with references too!
>>>>>>
>>>>>> The basic idea is so widespread in science that it hardly needs 
>>>>>> defense— frictionless bodies, stimulus generalization, latent 
>>>>>> learning, stereotype bias.... and linguistic competence all are 
>>>>>> more or less scientific concepts designed to variously explain 
>>>>>> conditional performance.
>>>>>>
>>>>>> Does anyone really doubt that the language one observes is but a 
>>>>>> subset of the language one might observe under such and such 
>>>>>> conditions? And that much of that observed language is 
>>>>>> fragmentary and ill-formed? While every case demands its own 
>>>>>> explanatory story, to toss out the whole idea of competence or 
>>>>>> similar concepts sounds like a lame return to behaviorism.
>>>>>>
>>>>>> Years ago I spent a lot of time on the development of complex 
>>>>>> sentences (Limber, 1973). There was one gap in the thousands of 
>>>>>> two to three year old children’s utterances I observed -- a lack 
>>>>>> of relative clauses attached to subject NPs.
>>>>>>
>>>>>> Did this mean these kids didn’t have the potential for those 
>>>>>> structures in their behavioral repertoire—their linguistic 
>>>>>> competence? Here’s the abstract of my answer (Limber, 1976)—which 
>>>>>> curiously in connection with this current discussion, involves 
>>>>>> pragmatics.
>>>>>>
>>>>>> “Inferences about linguistic competence in children are typically 
>>>>>> based on spontaneous speech.
>>>>>> This poses a problem since we know that other factors are also 
>>>>>> involved in speech production.
>>>>>> Children who may use complex object and adverbial NPs do not use 
>>>>>> complex subject NPs. Is
>>>>>> this a competence deficit, a performance problem, or simply a 
>>>>>> reflection of pragmatic factors?
>>>>>> Evidence presented here suggests that children probably do not 
>>>>>> need complex subjects. An
>>>>>> extensive use of pronouns in subject but not object position 
>>>>>> indicates that pragmatics may
>>>>>> account for the distribution of clauses in their speech. A 
>>>>>> similar pattern in adult speech indicates there is no warrant to 
>>>>>> conclude children's lack of subject clauses reflects anything 
>>>>>> more than the nature of spontaneous speech.”
>>>>>>
>>>>>>
>>>>>> In fact, in all my data of several thousand utterances of 
>>>>>> children and adults, only TWO subject NPs showed up—one shaky 
>>>>>> example from a three-year old and another from an adult. The 
>>>>>> probability that a child is exposed to a subject NP is, from my 
>>>>>> data, less than 1/1000. Here are the two:
>>>>>>
>>>>>> Adult: “ Well these buses that I've had today have been really 
>>>>>> weird.”
>>>>>>
>>>>>> Child: “I think that the girl ... that's here ... doesn't ... she 
>>>>>> doesn't want me to open it. “
>>>>>>
>>>>>> I wonder how the Nuevo-Statistical approaches to language 
>>>>>> acquisition would handle this?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Limber, J. (1973). The genesis of complex sentences. In T. Moore 
>>>>>> (Ed.), Cognitive Development and the Acquisition of Language (pp. 
>>>>>> 169-186). New York: Academic Press.
>>>>>> http://pubpages.unh.edu/~jel/JLimber/Genesis_complex_sentences.pdf 
>>>>>> <http://pubpages.unh.edu/%7Ejel/JLimber/Genesis_complex_sentences.pdf> 
>>>>>>
>>>>>>
>>>>>> Limber, J. (1976). Unravelling competence, performance, and 
>>>>>> pragmatics in the speech of young children. Journal of Child 
>>>>>> Language, 3, 309-318.
>>>>>> http://pubpages.unh.edu/~jel/JLimber/pragmatics_performance.pdf 
>>>>>> <http://pubpages.unh.edu/%7Ejel/JLimber/pragmatics_performance.pdf>
>>>>>>
>>>>>> John Limber
>>>>>> University of New Hampshire
>>>>>> Durham NH
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> I think that the girl ... that's here ...
>>>>>> doesn't ... she doesn't want me to open it.
>>>>>>
>>>>>> Well these
>>>>>> buses that I've had today have been really weird.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>
>