22.4, Diss: Comp Ling/Psycholing: McCrae: 'A Computational Model for the ...'

[linguist at LINGUISTLIST.ORG]

LINGUIST List: Vol-22-4. Tue Jan 04 2011. ISSN: 1068 - 4875.

Subject: 22.4, Diss: Comp Ling/Psycholing: McCrae: 'A Computational Model for the ...'

Moderators: Anthony Aristar, Eastern Michigan U <aristar at linguistlist.org>
            Helen Aristar-Dry, Eastern Michigan U <hdry at linguistlist.org>
 
Reviews: Monica Macaulay, U of Wisconsin-Madison  
Eric Raimy, U of Wisconsin-Madison  
Joseph Salmons, U of Wisconsin-Madison  
Anja Wanner, U of Wisconsin-Madison  
       <reviews at linguistlist.org> 

Homepage: http://linguistlist.org/

The LINGUIST List is funded by Eastern Michigan University, 
and donations from subscribers and publishers.

Editor for this issue: Mfon Udoinyang <mfon at linguistlist.org>
================================================================  

To post to LINGUIST, use our convenient web form at
http://linguistlist.org/LL/posttolinguist.cfm.

===========================Directory==============================  

1)
Date: 29-Dec-2010
From: Patrick McCrae [patrick.mccrae at informatik.uni-hamburg.de]
Subject: A Computational Model for the Influence of Cross-Modal Context upon Syntactic Parsing
 

	
-------------------------Message 1 ---------------------------------- 
Date: Tue, 04 Jan 2011 11:07:27
From: Patrick McCrae [patrick.mccrae at informatik.uni-hamburg.de]
Subject: A Computational Model for the Influence of Cross-Modal Context upon Syntactic Parsing

E-mail this message to a friend:
http://linguistlist.org/issues/emailmessage/verification.cfm?iss=22-4.html&submissionid=3797176&topicid=14&msgnumber=1
  

Institution: Hamburg University 
Program: Department of Informatics 
Dissertation Status: Completed 
Degree Date: 2010 

Author: Patrick McCrae

Dissertation Title: A Computational Model for the Influence of Cross-Modal
Context upon Syntactic Parsing 

Dissertation URL:  http://www.sub.uni-hamburg.de/opus/volltexte/2010/4800/index.html

Linguistic Field(s): Computational Linguistics
                     Psycholinguistics


Dissertation Director(s):
Wolfgang Menzel
Maosong Sun
Christopher Habel

Dissertation Abstract:

The focus of this thesis is to motivate, specify and validate a computational
model for the cross-modal influence of visual scene context upon natural
language understanding and the process of syntactic parsing, in particular. We
argue for a computational model that establishes cross-modal referential links
between words in the linguistic input and entities in a visual scene context.
Cross-modal referential links are assigned on the basis of conceptual
compatibility between the concepts activated in the linguistic modality and the
concepts instantiated in visual context. The proposed model utilises the
thematic relations in the visual scene context to modulate attachments in the
linguistic analysis.

We base our model architecture on the cognitive framework of Conceptual
Semantics by Ray Jackendoff. In our model, we adopt the central tenet of
Conceptual Semantics that all cross-modal interactions of non-linguistic
modalities with language are mediated by Conceptual Structure, a single, uniform
representation of linguistic and non-linguistic semantics. Conceptual Structure
propagates the influence of the non-linguistic modalities into syntactic
representation via a syntax-semantics interface. The purpose of this interface
is to map between the syntactic and the semantic representation by means of
representational correspondence rules.

Our model implements central aspects of the cognitive architecture in Conceptual
Semantics. We encode the semantic information for all entities, be they
linguistic or non-linguistic in nature, on a single level of semantic
representation. In particular, the semantic part of linguistic analysis and
visual scene information are included in this representation. The semantic
preferences arising from visual context constrain the semantic part of
linguistic analysis. The semantic part of linguistic analysis, in turn,
constrains syntactic analysis via the syntax-semantics interface. In this way,
our model achieves a semantically mediated propagation of non-linguistic visual
scene information into syntactic representation. 

In Part III of the thesis we validate our model's context integration behaviour
under a range of experimental conditions. The integration of visual scene
context as a hard constraint on linguistic analysis enforces an absolute
dominance of visual context information over linguistic analysis. As a result,
hard integration may afford a contextualised linguistic analysis that violates
linguistic well-formedness preferences in order to be semantically compatible
with the modelled visual context. Integrating visual context information as a
soft constraint on linguistic analysis permits a balance between conflicting
linguistic and contextual references based on the strength of the individual
preferences. Under soft integration, our model also diagnoses which aspects of
linguistic analysis are in conflict with visual context information. Diagnosis
constitutes an important cognitive capability of natural systems in the situated
cognition.

We further demonstrate our model's robustness to conceptual underspecification
in the contextual representation. Our experiments show that the integration of
conceptually underspecified context representations provides sufficient
information to support the process of syntactic disambiguation. The capability
of processing conceptually underspecified semantic information is a relevant
capability of natural systems when dealing with perceptual uncertainty and
perceptual ambiguity.





-----------------------------------------------------------
LINGUIST List: Vol-22-4	
----------------------------------------------------------