27.1284, Review: Cog Sci; Computational Ling; Psycholing: Sharp, Delmonte (2015)

[The LINGUIST List via LINGUIST]

LINGUIST List: Vol-27-1284. Mon Mar 14 2016. ISSN: 1069 - 4875.

Subject: 27.1284, Review: Cog Sci; Computational Ling; Psycholing: Sharp, Delmonte (2015)

Moderators: linguist at linguistlist.org (Damir Cavar, Malgorzata E. Cavar)
Reviews: reviews at linguistlist.org (Anthony Aristar, Helen Aristar-Dry, Sara Couture)
Homepage: http://linguistlist.org

*****************    LINGUIST List Support    *****************
                   25 years of LINGUIST List!
Please support the LL editors and operation with a donation at:
           http://funddrive.linguistlist.org/donate/

Editor for this issue: Sara  Couture <sara at linguistlist.org>
================================================================


Date: Mon, 14 Mar 2016 13:50:09
From: Laurel Schenkoske [laurelschenkoske at email.arizona.edu]
Subject: Natural Language Processing and Cognitive Science

 
Discuss this message:
http://linguistlist.org/pubs/reviews/get-review.cfm?subid=36074537


Book announced at http://linguistlist.org/issues/26/26-1446.html

EDITOR: Bernadette  Sharp
EDITOR: Rodolfo  Delmonte
TITLE: Natural Language Processing and Cognitive Science
SUBTITLE: Proceedings 2014
PUBLISHER: De Gruyter Mouton
YEAR: 2015

REVIEWER: Laurel A. Schenkoske, University of Arizona

Reviews Editor: Sara Couture

SUMMARY

'Natural Processing and Cognitive Science' is a conference proceedings of the
2014, 11th annual NLPCS Workshop, the aim of which ''was to foster
interactions among researchers and practitioners in ... (NLP) by taking a
Cognitive Science perspective'' (preface). The edited volume contains 25
independent peer-reviewed articles, many written by graduate students, and
very broadly connected by the conference theme. The papers cover a very broad
array within the scope of Natural Language Processing and Cognitive Science.
Most of the articles are early work and pilot studies, as is typical for
proceedings, and present exciting new venues for research in their respective
fields. Research methodologies include experimental and computational studies,
with people and with machines. The keynote speech was delivered by Jared
Bernstein of Stanford University, and is the basis of the first article of the
volume. In this review, each article is discussed separately, in the order it
appears within the volume.

''Benchmarking Automated Text Correction Services'' served as the keynote
address (Bernestein, et. al.). They compared two automated error detection
services over 24 short middle school essays. Both engines had reported
accuracy rates above 90%, yet Bernstein, et. al., found both to have high
error rates: one had better precision (found only real errors, but also missed
several), while the other had better recall (caught more real errors, but
counted many correct items as errors). Human teachers were found to have high
variability, both within each individual teacher and across various teachers.
The authors of the study suggest that a system consisting of a machine with
high recall plus a human checker would make the best fit. Nothing was said
about meaning, style, coherence, and flow, except that these were
intentionally ignored in the study.

''Enhancing Word Sense Disambiguation Using a Hybrid Knowledge-Based
Technique'' discusses the issue of lexical semantic ambiguities – often
problematic for machine-based computations. The authors combine an adapted
version of Lesk's Algorithm (1986) and Jiang & Coranth's Similarity measure
(1997), and determine that their hybrid measure outperforms both methods when
tested alone in the disambiguation of word senses.

''Disambiguating Distributional Neighbors Using a Lexical Substitution
Dataset'' proposes a new method of clustering polysemous targets. They use a
lexical substitution dataset to attribute one correct sense to each target
word, based on the number of similar substitutes. Unlike more traditional
approaches (e.g. wordnets), this method is able to assign value to neighbors
related to both senses, as well as to neither sense.

''An Evolutionary Game Theoretic Approach to Word Sense Disambiguation'' uses
a semi-supervised model and Game Theory to represent words (nodes) as players
who choose a strategy – a semantic class membership. The approach uses data
(information on semantic class) from labeled nodes (words) and extends that
information to unlabeled nodes / words. This works because the players are
always neighbors, and often share a class. Their approach is different from
others in that they do not need to train the system on large corpora, but use
instead small sets of labeled data.

''The Main Challenge of Semi-Automatic Term Extraction Methods'' demonstrates
that the main challenge is the difficulty of determining rank and threshold of
terms. (The threshold may be either a percentage or a number of candidate
words.) After comparing 21 extraction methods of statistical, linguistic, and
hybrid knowledge, using three different corpora, the authors conclude that
best precision, best recall, and best F-measure are dependent on the type of
knowledge.

''News Text Segmentation in Human Perception'' compares the results of human
psycholinguistic tasks against computational experiments, in determining text
predictability in the homogenous genre of news texts. Using a corpus of
Russian news sources, they look at various language levels, from words to
syntagmas to propositions to discourse. They found that the keywords extracted
by humans and those by the program were very different. Unlike the
computational system, human informants identified words and syntagmas relevant
to the context. The authors also found that, overall, more keywords in a
segment correlated with higher syntagmatic weight (i.e., frequent in the
document, but rare in the corpus).
 
''Preliminary Study of TV Caption Presentation Method for Aphasia Sufferers''
describes how a small set of aphasia sufferers preferred a summarized version
of closed captioning over the original version. The authors have developed a
unified interface in which captions from the video provider are 1)
automatically summarized, and 2) passed on to volunteers for manual
summarization.

''Extraction of Concrete Entities and Part-Whole Relations'' proposes a method
for meronymy (part-whole) extraction of concrete objects from a corpus.
Precision rates were somewhat low, but the authors suggest this could be
improved with better filtering of non-relevant words.

In ''Human Association Network and Text Collection,'' the authors base their
model on the fact that semantic information may be present in human
communication, even when lexical information is not. For example, “terrier”
and “animal” are semantically related in memory. They develop an algorithm for
text-driven extraction of direct and indirect associations.

''Using Function Words for Authorship Attribution'' displays a comparison
between two methods of attributing authorship of a text: a style marker, which
relies on sequential rules, versus function-word frequency, which relies on
''bag-of-words assumptions.'' It seems they set out to find better results in
the rule-based method, but find instead that the bag-of-words assumptions have
much higher accuracy, as previous research had also found. 

In ''Recognition of Discursive Verbal Politeness,'' the authors pilot a
program to automatically annotate text to identify ''Politeness Verbal
Phrases,'' including both positive and negative politeness. Results are high,
close to matching the manual annotation.

''Politeness versus Perceived Engagement'' looks at perceived engagement
between speaker and hearer to determine how differently weighted
Face-Threatening Acts are implemented. Such acts, according to Politeness
Theory, include expressions of disapproval and criticism, as well as orders
and requests (Brown & Levinson, 2004). Despite politeness being subjective,
with individual differences, they found that when a speaker wants to continue
an interaction, he or she needs to use more politeness strategies for a less
engaged hearer. The authors plan to apply these findings to research in
human-machine interaction.

''Sentiment, Polarity and Function Analysis in Bibliometrics'' offers a review
of works on both citation analysis (bibliometrics) and sentiment analysis
(SA), and propose a marriage of the two, claiming that simple, quantitative
citation counts do not offer enough insight into well-cited authors or works,
and sometimes even offering false insight. Instead, the authors claim, a
qualitative measure of sentiment for each cited work should be included.

In ''The Detection and Analysis of Bi-Polar Phrases and Polarity Conflicts,''
the authors conduct a sentiment analysis of Noun Phrases (NPs) (e.g., “just
punishment”) and Verb Phrases (VPs) (e.g., “He admires his sick friend”) for
polarity expectations and violations. They insist more fine-grained
distinctions are needed than simply ''positive'' or ''negative,'' and
therefore use a modified Appraisal Theory (Martin & White, 2005). They first
create a multilingual lexicon of adjectives and nouns that are either positive
or negative, and further classified as either ''Appreciation,'' ''Affect,'' or
Judgment.'' They extend the study to bi-polar VPs (where verbs carry an
unexpected polarity for their arguments), and finally combine the two
analyses.

''Automatically Evaluating Atypical Language in Narratives by Children with
Autistic Spectrum Disorder'' looks a set of narratives from children with ASD.
The large dataset allowed the authors to distinguish between general language
deficiencies and those specific to ASD. They were also able to look at much
more distinctive features than just text length. There were some specific
findings in pronoun use and topic coherence, and a more profound finding in
sentiment and references to mental state, with the ASD participants using
fewer than their matched controls.

In ''How to Make Right Decisions Based on Corrupt Information and Poor
Counselors,'' it is unclear how the title relates to the article. The authors
review open-source Question Answering systems, and their inherent problems,
such as time dependency and ambiguity. They provide suggestions for updating
the OpenEphyra (Schlaefer, N., et. al., 2011) open-source QA system,
specifically in its interface with various internet search engines.

''Meta-Learning for Fast Dialog System Habituation to New Learners''
introduces an example of a human assistant learning the behaviors regarding
phone calling and contact information of her boss, and extends that example to
the ''learning,'' or habituation, of a software app. This pilot study uses a
statistical dialog manager to learn its users intent for phone calling
behavior. Crucially, parameter settings are first learned from a sample
population, and are then further refined for an individual user.

''Evaluation of Freely Available Speech Synthesis Voices for Halef'' may be a
difficult article to follow for someone unfamiliar with speech synthesis
research, as some key acronyms are not explained. The authors analyzed several
freely available voices from two competing text-to-speech (TTS) systems -
Festival and Mary - and found the best voices to belong to the Mary system.
They conclude this is the best system for Halef (Help Assistant
Language-Enabled and Free).

In ''You Shall Find the Target Via its Companion Words,'' the authors
developed a Human-Computer-Interaction system for overcoming Tip-of-the-Tongue
(TOT) states (a search problem in speech production). Since speakers
experiencing a TOT access problem typically have some concept of the word –
meaning especially – the authors were able to design a system using those
known associations to help the speaker access the ''misplaced'' word. They
performed a comparative analysis of WordNet, Extended WordNet, the Edinburg
Association Thesaurus, unstructured language corpus, and Rogert's Thesaurus,
in which each yielded both strengths and weaknesses; a combination of systems
is proposed. The system itself consists of a two-step process. First, the user
inputs a semantically related word, and the entire lexicon (some 60,000 words)
is dramatically reduced. Second, the user is presented with a categorical
tree, made up of nodes, or word sets, with possible target words. From here,
the user may either chose his or her target word, or search a particular word
set further.

''Topic Modeling for Entity Linking Using Keyphrase'' is an update and
improvement of an Entity Linking system that the authors completed in 2012 and
2013. Within a Knowledge Base (here, Wikipedia), the system links entries to
references via a multi-step process between Query and Answer. The last step,
candidate ranking and clustering of entries not in the knowledge base, is
their main challenge and focus.

In ''Extraction of Polish Multiword Expressions For the Automatic Extraction
of Polish Multiword Expressions'' (MWE), the authors experiment with
Dictionary Pattern Extraction (DM) and Syntactic Pattern Extraction (SM), and
several combined methods. In their most successful pattern – ''DM for SM
results'' – the output is a list of words with relevant declension
information, with both good precision and good recall.

''Beyond Classical Set'' offers an extension of classical Set Theory for
Natural Language Semantics. The authors put forth a theory of typed sets,
which they call a “natural extension to classical sets” (p. 270). The theory
considers “urelements” (simple and unanalyzable elements of sets) as binary,
each with a core and a type. Their Typed Sets accommodates the concepts of
predication and aggregation, and allows for finer grained analysis of meaning
and natural language processing.

''Exploring the Effects of Root Expansion, Sentence Splitting, and Ontology in
Arabic Answer Selection'' addresses the issue of imperfection in Arabic
Question Answering Systems (as well as all Question Answering Systems). The
authors compare their system (ALQASIM 2.0) to two other Arabic Question
Answering for Machine Reading and Evaluation systems (QA4MRE). These include
their own previous ALQAISM 1.0 (Ezzeldin, et. al., 2013) and IDRAAQ
(Abouenour, et. al., 2012). The 2.0 system proposed here consists of three
main modules: Document Analysis, Question Analysis, and Answer Selection.
Notably different from the other systems is that it focuses on answer
selection and validation. Through their approach to sentence splitting and
root expansion, they are able to achieve much higher accuracy, as well as give
partial credit for unanswered questions that the system could not answer.

In ''Computer Assisted Translation in Ancient Texts,'' the authors developed a
Computer-Assisted Translation (CAT) system specifically for the use with the
ancient Babylonian Talmud, to be translated into Italian. The authors describe
the rich linguistic variety in the text, but also explain that it contains
many quotations and lexically repetitive, formulaic language, leading them to
base the translation system on Translation Memory (TM). Aside from improved
translation speed, their main objective is to form a collaborative environment
for translators and scholars familiar with the Babylonian Talmud.

''A Rational Statistical Parser'' demonstrates the development of a
computational syntactic parser to work as rationally as a human one – finding
the ''optimal path to the solution'' (p. 303). The three objectives for their
model to be rational is that it choose the most probable analyses according to
experience, that it be quick, and that it minimize cognitive costs. Their
model assigns probability and entropy scores to syntactic trees in order to
determine their weight and to find the best path – a successful method,
according to their experimental results.

EVALUATION

The volume is an excellent addition to the fields of Natural Language
Processing and Cognitive Science. NLPCS appears to be a student-friendly
conference and proceedings, helping to open pathways for new and exciting
research into the fields. The articles span a great diversity, and serve as
springboards to potentially very significant work in their respective
sub-fields. The genres also cover a literature review, psycholinguistic and
computational studies, and human-to-human and human-to-computer interactions. 

While many articles are only loosely connected, some cover intimately related
themes (e.g., Word Sense Disambiguation, Sentiment Analysis). The 25 articles
are not organized into sections or arranged by topic. For the most part, they
are arranged in a logical manner, e.g., those articles discussing text linking
programs are grouped together, as are some on lexical semantics. But the
article on Tip-of-the-Tongue states, and another handling formal semantics, do
not appear until much later. Furthermore, there are two separate articles that
discuss Question Answering Systems, which appear at very different points in
the collection. While there can be no clear linear path linking one article to
the next, a better attempt could have been made to keep like articles
together. 

Aside from the organization among the papers, improvements upon some minor
weaknesses would enhance the articles and the book itself. The volume would
have benefitted from some major copy editing. Most of the authors are
non-native English speakers, and spelling and word order errors – even in the
chapter headings – is a point of confusion and slower processing on the part
of the reader. Another possible point of concern is that some tables are
colored, while most are black and white. Did all the authors know they had the
option to submit colored tables and charts? Finally the 25 articles represent
vastly different formatting techniques for in-text citations, further reducing
the coherency of the volume.

The articles themselves are well formulated. Each begins with a general
introduction, including a plan, goal, or rationale of the research, and then
the organization for the rest of the article. They are well thought out and
include solid research methods, described in the text and clearly represented
in charts and tables.

Because of the broad spectrum of topics, it is doubtful that many people will
read this volume cover-to-cover. But many of the articles contained within
will certainly be of interest to students and researchers of NLP, cognitive
sciences, linguistics, psychology, and computer programming. Most of the
papers propose cutting-edge research in their fields, and propose work to
carry the research even further.

REFERENCES

Abouenour, L., Bouzoubaa, K., & Rosso, P. (2012). IDRAAQ: New Arabic Question
Answering System based on query expansion and passage retrieval. In CLEF 2012
Workshop on Question Answering for Machine Reading Evaluation (QA4MRE). Rome,
Italy.

Bernstein, J. (2014, October). Benchmarking Automated Text Correction
Services. Keynote Address presented at the 11th Annual NLPCS Workshop,Venice,
Italy.

Brown, P. & Levinson, S.C. (1978). Politeness: Some universals in language
usage. Cambridge UP: Cambridge, UK.

Ezzeldin, A. M., Kholief, M. H., & El-Sonbaty, Y. (2013). ALQASIM: Arabic
language question answer selection in machines. In Information Access
Evaluation. Mutilinguality, Multimodality, and Visualization (pp. 100-103).
Springer: Berlin, Germany.

Jaing, J. J.  & Coranth, D. W. (1997). Semantic similarity based on corpus
statistics and lexical taxonomy. In C. K. Huang & R. Sproat (Eds.),
Proceedings of the 10th International Conference on Research in Computational
Linguistics (pp. 19-33). Taipei, Taiwan.

Lesk, M. (1986). Automatic sense disambiguation using machine readable
dictionaries: how to tell a pine cone from an ice cream cone. In V. DeBuys
(Ed.), SIGDOC ‘86: Proceedings of the 5th annual international conference on
systems documentation (pp. 24-26). New York, NY: ACM.

Martin, J. R. & White, P. R. (2005). The language of evaluation: Appraisal in
English. London: Palgrave MacMillan.

Schlaefer, N., Nyberg, E., J.C.J.C. & Chu-Carroll, J. (2011). Statistical
source expansion for question answering. (Doctoral dissertation). Technologies
Institute, School of Computer Science, Carnegie Mellon University.


ABOUT THE REVIEWER

Laurel Schenkoske received her MA in German Studies and Linguistics from the
University of Wisconsin- Milwaukee, where she continued further study in the
field of formal Linguistics. She is currently a student in the SLAT (Second
Language Acquisition and Teaching) PhD program at the University of Arizona,
specializing in Processes, with a particular interest in second-language
sentence processing. She has spent many years in the classroom, teaching a
variety of Linguistics, German, and Composition courses.





------------------------------------------------------------------------------

*****************    LINGUIST List Support    *****************
Please support the LL editors and operation with a donation at:
            http://funddrive.linguistlist.org/donate/
 


----------------------------------------------------------
LINGUIST List: Vol-27-1284	
----------------------------------------------------------