Competence vs. Performance: Summary

Wed Oct 17 22:39:24 UTC 2007

I disagree strongly, dear Gary, with your untested assumption that "in 
psychology, and in linguistics, we are continually left with the 
daunting challenge of inferring underlying representations from surface 
behavior".  Just imagine for a minute that this rationalistic perception 
of psychology and linguistics is in effect incorrect and misleading -- 
as well as leading us to wasted decades of research on transformations, 
deep structures, and innate parameters.  There are other ways to look at 
the organization of reality, and my only hope is that the next 
generation of theoreticians of both disciplines will not repeat the 
mistakes of the last one.

Anat Ninio
Professor of Psychology
The Hebrew University of Jerusalem

Gary Marcus wrote:
>
> I couldn't disagree more strenuously with this last posting. To 
> ascribe both "spinned" and "spun" to competence is to miss the point 
> of the distinction, not to undermine it. "Spinned" and "spun" simply 
> aren't on a par; people accept the latter as correct and ascribe the 
> former to a mistake.
>
> It makes perfect sense to collect data on both (I did just that in 
> 1992 SRCD Monograph on overregularizations, and I have cited 
> Stemberger's data in this connection), but no sense whatsoever to 
> treat the two as if they are on equal footing. And I say this not as a 
> matter of prescriptive linguistics (what the rules of language "ought 
> to be", according to some self-appointed grammarian) but as a matter 
> of descriptive psycholinguistics, as a characterization of what 
> speakers themselves believe.
>
> When I inadvertently forget to carry a 1 and report that the sum of 87 
> and 24 is 101, we need to able to distinguish my transient error from 
> my general understanding of what would constitute a correct sum (viz. 
> 111).  A competence-performance distinction gives us a theoretical 
> tool with which to make that distinction; to say that 101 and 111 are 
> equally good answers ("since I have the 'competence' to utter either 
> number") would be nothing more than sophistry.
>
> The competence-performance distinction does, as others have noted, 
> have any number of difficulties, but most are methodological ("how can 
> I tell which is which", a fact that will only reveal itself once we 
> have properly carved nature at it joints), not theoretical.
>
> In psychology, and in linguistics, we are continually left with the 
> daunting challenge of inferring underlying representations from 
> surface behavior; there is no way we can conceivably succeed at that 
> task without a firm understanding of the fact that the mapping between 
> the two is often indirect: behavior is dictated not only by underlying 
> representations but by a host of other factors.  Inasmuch as the 
> competence-performance distinction encapsulates that fundamental 
> truth, it is a valuable tool that we cannot afford to dismiss.
>
> -- Gary Marcus
> Professor of Psychology
> New York University
>
>
>
> On Oct 17, 2007, at 1:26 PM, Anat Ninio wrote:
>
>> Hello Everybody,
>>
>> I have a feeling -- following Joe Stemberger's excellent 
>> contributions to this strand -- that we're all a little scared 
>> stating the obvious, which is that the notion of competence as a 
>> separate entity from performance is a philosophical error, pure and 
>> simple. If under time stress people say in 4% of the time "spinned" 
>> instead of "spun", then they possess a competence to do so. 
>> Competence is -- in plain English -- an a priori stored potential or 
>> ability to produce some behaviour, and any other treatment of it, by 
>> Chomsky or whoever, is at best a mystification of the obvious.
>>
>> I agree with Joe that we should simply proceed with doing research 
>> and collect information on what people actually say, whether children 
>> or adults, and don't heed the voices saying that what we hear is 
>> "merely" this or "merely" that, when any philosopher major in their 
>> first year would tell us how confused the whole idea of 
>> competence/performance distinction is.
>>
>> The best,
>>
>> Anat Ninio
>>
>>
>>
>>
>>
>> Joe Stemberger wrote:
>>> Hello, everyone.
>>>
>>> Just to start, I'd like to mention that I've spent all of my career 
>>> working on speech errors in adults (both spontaneous speech and 
>>> experimental tasks), and so I accept the notion that there are 
>>> outputs that are "correct" but that performance factors can lead to 
>>> "incorrect" outputs. But that doesn't mean that "the" 
>>> competence-performance distinction gets us anywhere practical.
>>>
>>> Defenses of "the" competence-performance distinction are missing two 
>>> main points:
>>> (1) that the exact division between what is competence and what is 
>>> performance, as well as the criteria that distinguish them, are 
>>> largely unknown after more than 40 years.
>>> (2) There is no clear way to test competence except through 
>>> performance.
>>>
>>> Consider John Limber's quote:
>>> "Inferences about linguistic competence in children are typically 
>>> based on spontaneous speech.
>>> This poses a problem since we know that other factors are also 
>>> involved in speech production."
>>>
>>> Well, yes, there are performance factors in language production.
>>> Just as there are performance factors in language perception, 
>>> language comprehension, grammaticality judgments, etc.
>>> In the real world, we can observe only performance, and all acts of 
>>> any sort are "contaminated" by performance.
>>> And it follows from that that inferences about competence must come 
>>> from performance, only and always.
>>>
>>>
>>> OK, so now let's get concrete.
>>> Observe a native speaker of English of any age for long enough, and 
>>> you will find them saying *SPINNED instead of SPUN.
>>> Ask them about it, and they'll tell you that it was an error.
>>> In typical speeded experimental situations, undergraduates produce 
>>> such errors about 4% of the time in neutral contexts.
>>> (And in an experiment that I'm just finishing up, where they produce 
>>> coordinated verbs, the error GRINNED AND *SPINNED (with rhyming 
>>> regular in the first word) shoots up to about 25% of tokens.)
>>> Young children produce such outputs from an early age, but every 
>>> study has shown that such overregularizations are in the minority 
>>> for most children (and most irregular verbs) from the beginning, and 
>>> that, like in adult speech, the frequency of the verb is one of the 
>>> predictors of error rate.
>>> This is a generally viewed as a performance phenomenon at all ages.
>>> But yet it is cited all the time as a nice deomstration that 
>>> patterns are being extracted and generalized.
>>>
>>> At the same time, speakers will also occasionally produce, instead 
>>> of SPUN, *SPAN. In experimental situations, such errors are easy to 
>>> come by.
>>> As well as things like *GRUN instead of regular GRINNED. (In that 
>>> experiment that I'm just finishing up, *GRUN occurs about 2% of the 
>>> time in neutral contexts,
>>> rising to about 6% in SPUN AND *GRUN (with a rhyming irregular in 
>>> the first word).)
>>> Bybee & Moder 1983 showed that irregular patterns generalize to 
>>> novel words at much greater rates.
>>> So irregular patterns also show generalization.
>>>
>>> Consider another of John's quotes:
>>> "Does anyone really doubt that the language one observes is but a 
>>> subset of the language one might observe under such and such 
>>> conditions?"
>>>
>>> So, if we MUST conclude that competence extracts and encapsulates 
>>> explicit procedures (such as rules) to occount for generalization, 
>>> then competence contains such procedures for creating both regular 
>>> and irregular forms, because both types of patterns overgeneralize, 
>>> right?
>>>
>>> Steve Pinker, Harald Clausen, and others taking a similar approach, 
>>> have argued that the overgeneralization of the -/ed/ pattern 
>>> involves the use of a rule after failure to access an irregular 
>>> form, but that overregularization of irregular patterns is a 
>>> performance phenomenon, based on the way that irregular forms are 
>>> stored in and accessed from the lexicon.
>>> Which leads to the conclusion that generalization of patterns can 
>>> occur for performance reasons, even in approaches that accept "the" 
>>> competence-performance distinction.
>>>
>>> Which leads to this possibility:
>>> all inflected forms that a speaker has been been exposed to are 
>>> simply stored in the lexicon.
>>> All generalization, even of regular patterns, is a performance 
>>> phenomenon.
>>> And while it's true that the -/ED/ pattern overgeneralizes more than 
>>> any other pattern, we could attribute /that/ difference to 
>>> performance factors, right?
>>>
>>> Or even
>>> The purpose of a grammar is to enumerate the sentences of the language.
>>> Actual grammars consist of storing every sentence that has ever been 
>>> observed.
>>> Generalization is just a performance phenomenon across stored 
>>> exemplars.
>>> You want to know in detail how generalization occurs?
>>> Hey, that's a performance phenomenon. It lies outside the proper 
>>> bounds of linguistic theory, and so it's not my responsibility to 
>>> show how it works.
>>>
>>> And what I personally would want to see is some formal proof that 
>>> "the" competence-performance distinction couldn't lead us to that 
>>> sort of system just as easily as it has led us anywhere else.
>>>
>>> >From a practical perspective, it seems to me that, since we have to 
>>> work with performance data anyway, we want to work with as wide a 
>>> range of types of data as possible.
>>> And we want to develop theories that account for all of those data, 
>>> in detail.
>>> If there's a distinction between competence and performance, that's 
>>> fine, but it has to be explicit, and our theories need to explain 
>>> exactly which phenomena are due to competence, which are due to 
>>> performance, and why. And if there are aspects of data that aren't 
>>> accounted for in detail, it means that we should be uncomfortable, 
>>> because we need to account for it all.
>>>
>>> That isn't the way that "the" competence-performance distinction has 
>>> been used in the past.
>>> If it had been, that would've been fine, as far as I'm concerned.
>>> While some theoretical linguists feel it's fine to apply linguistic 
>>> theory to child language, few are open to using child language to 
>>> provide the tie-breaker for choosing between two theoretical 
>>> mechanisms.
>>>
>>> In the meantime, I'll happily go on studying performance and working 
>>> on theories of how human language works, including claims about the 
>>> fundamental orgainization of the language system.
>>>
>>> As should we all.
>>>
>>>
>>> ---Joe Stemberger
>>> Linguistics
>>> UBC
>>>
>>>
>>> john limber pravi:
>>>>
>>>> On 10/16/07 6:46 AM, "Matthew Saxton" <M.Saxton at ioe.ac.uk> 
>>>> <mailto:M.Saxton at ioe.ac.uk> wrote:
>>>>
>>>>     “No-one has stepped forward to defend the competence-performance
>>>>     distinction, or even to offer supportive references.”
>>>>
>>>> OK— try this—with references too!
>>>>
>>>> The basic idea is so widespread in science that it hardly needs 
>>>> defense— frictionless bodies, stimulus generalization, latent 
>>>> learning, stereotype bias.... and linguistic competence all are 
>>>> more or less scientific concepts designed to variously explain 
>>>> conditional performance.
>>>>
>>>> Does anyone really doubt that the language one observes is but a 
>>>> subset of the language one might observe under such and such 
>>>> conditions? And that much of that observed language is fragmentary 
>>>> and ill-formed? While every case demands its own explanatory story, 
>>>> to toss out the whole idea of competence or similar concepts sounds 
>>>> like a lame return to behaviorism.
>>>>
>>>> Years ago I spent a lot of time on the development of complex 
>>>> sentences (Limber, 1973). There was one gap in the thousands of two 
>>>> to three year old children’s utterances I observed -- a lack of 
>>>> relative clauses attached to subject NPs.
>>>>
>>>> Did this mean these kids didn’t have the potential for those 
>>>> structures in their behavioral repertoire—their linguistic 
>>>> competence? Here’s the abstract of my answer (Limber, 1976)—which 
>>>> curiously in connection with this current discussion, involves 
>>>> pragmatics.
>>>>
>>>> “Inferences about linguistic competence in children are typically 
>>>> based on spontaneous speech.
>>>> This poses a problem since we know that other factors are also 
>>>> involved in speech production.
>>>> Children who may use complex object and adverbial NPs do not use 
>>>> complex subject NPs. Is
>>>> this a competence deficit, a performance problem, or simply a 
>>>> reflection of pragmatic factors?
>>>> Evidence presented here suggests that children probably do not need 
>>>> complex subjects. An
>>>> extensive use of pronouns in subject but not object position 
>>>> indicates that pragmatics may
>>>> account for the distribution of clauses in their speech. A similar 
>>>> pattern in adult speech indicates there is no warrant to conclude 
>>>> children's lack of subject clauses reflects anything more than the 
>>>> nature of spontaneous speech.”
>>>>
>>>>
>>>> In fact, in all my data of several thousand utterances of children 
>>>> and adults, only TWO subject NPs showed up—one shaky example from a 
>>>> three-year old and another from an adult. The probability that a 
>>>> child is exposed to a subject NP is, from my data, less than 
>>>> 1/1000. Here are the two:
>>>>
>>>> Adult: “ Well these buses that I've had today have been really weird.”
>>>>
>>>> Child: “I think that the girl ... that's here ... doesn't ... she 
>>>> doesn't want me to open it. “
>>>>
>>>> I wonder how the Nuevo-Statistical approaches to language 
>>>> acquisition would handle this?
>>>>
>>>>
>>>>
>>>> Limber, J. (1973). The genesis of complex sentences. In T. Moore 
>>>> (Ed.), Cognitive Development and the Acquisition of Language (pp. 
>>>> 169-186). New York: Academic Press.
>>>> http://pubpages.unh.edu/~jel/JLimber/Genesis_complex_sentences.pdf 
>>>> <http://pubpages.unh.edu/%7Ejel/JLimber/Genesis_complex_sentences.pdf>
>>>>
>>>> Limber, J. (1976). Unravelling competence, performance, and 
>>>> pragmatics in the speech of young children. Journal of Child 
>>>> Language, 3, 309-318.
>>>> http://pubpages.unh.edu/~jel/JLimber/pragmatics_performance.pdf 
>>>> <http://pubpages.unh.edu/%7Ejel/JLimber/pragmatics_performance.pdf>
>>>>
>>>> John Limber
>>>> University of New Hampshire
>>>> Durham NH
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> I think that the girl ... that's here ...
>>>> doesn't ... she doesn't want me to open it.
>>>>
>>>> Well these
>>>> buses that I've had today have been really weird.
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>>
>
>
>