30.3795, Review: Pragmatics: Brône, Oben (2018)
The LINGUIST List
linguist at listserv.linguistlist.org
Tue Oct 8 14:44:44 UTC 2019
LINGUIST List: Vol-30-3795. Tue Oct 08 2019. ISSN: 1069 - 4875.
Subject: 30.3795, Review: Pragmatics: Brône, Oben (2018)
Moderator: Malgorzata E. Cavar (linguist at linguistlist.org)
Student Moderator: Jeremy Coburn
Managing Editor: Becca Morris
Team: Helen Aristar-Dry, Everett Green, Sarah Robinson, Peace Han, Nils Hjortnaes, Yiwen Zhang, Julian Dietrich
Jobs: jobs at linguistlist.org | Conferences: callconf at linguistlist.org | Pubs: pubs at linguistlist.org
Homepage: http://linguistlist.org
Please support the LL editors and operation with a donation at:
https://funddrive.linguistlist.org/donate/
Editor for this issue: Jeremy Coburn <jecoburn at linguistlist.org>
================================================================
Date: Tue, 08 Oct 2019 10:44:24
From: Maria-Jose Arrufat-Marques [arrufatm at uji.es]
Subject: Eye-tracking in Interaction
Discuss this message:
http://linguistlist.org/pubs/reviews/get-review.cfm?subid=36518137
Book announced at http://linguistlist.org/issues/30/30-233.html
EDITOR: Geert Brône
EDITOR: Bert Oben
TITLE: Eye-tracking in Interaction
SUBTITLE: Studies on the role of eye gaze in dialogue
SERIES TITLE: Advances in Interaction Studies 10
PUBLISHER: John Benjamins
YEAR: 2018
REVIEWER: Maria-Jose Arrufat-Marques, Universitat Jaume I
SUMMARY
Eye-tracking in interaction is a groundbreaking, field-shaping volume that
enhances and deepens the reader’s knowledge on the pivotal role of eye gaze in
face-to-face interaction, whether human-human or human-computer, from an
interdisciplinary research stance. This volume is a compilation of twelve
chapters and is divided into three parts. Part 1 includes four chapters that
tackle theoretical issues. Part 2 is comprised of three studies which
highlight methodological issues considered in eye-tracking research. Part 3 is
composed of four case studies that emphasize the importance of employing
eye-tracking techniques to explore interactional situations. The first chapter
is an introduction to this topic elaborated by the editors of this volume. The
remaining eleven chapters are different research studies that have a common
denominator: the use of eye tracking tools to collect and analyze data
gathered from interactional situations between humans or human-computer
communicative situations.
Chapter 1, the introduction (pp. 1-18), frames the study of eye gaze within
the broader research branch of non-verbal communication, where gesture, and
hand gesture more specifically, have dominated the field for years. Eye gaze
is introduced as another central semiotic tool for successful face-to-face
communication. This chapter is divided into three main sections. Section One
informs about early research on eye gaze in its two main research lines:
interactional and cognitive. Most importantly, emphasis is made on the
functional nature of eye gaze to organize and regulate communicative events.
The second section explores technological tools used to study eye gaze in
conversation analysis, psychology and psycholinguistics, and human-computer
interaction, thus highlighting the various and distinct functions eye gaze has
in communication. Part Three encompasses the outline of the book, where
emphasis is made on the interrelationship of all three sections. All in all,
Brône and Oben claim that this book puts together research from different
fields of study, an innovative perspective into eye-tracking in interaction,
underlining the interdisciplinary nature of this line of research. Thus, they
attempt to fill a previously unfilled research niche.
Chapter 2, “Eye gaze as a cue for recognizing intention and coordinating joint
action” (pp. 21-46), is authored by Amati and Brennan. They review seven
research areas within eye gaze to draw attention to the ambiguity that looking
at objects or people may have. In other words, securely knowing what the other
interlocutor is looking at may hasten successful communication. But this is
not always the situation. Humans and computers can recognize eye gaze in both
face-to-face and remote interactions. However, reading someone’s gaze may be a
challenging endeavor, since it may be difficult to interpret. Nevertheless,
Amati and Brennan provide instances that exemplify how humans and computers
employ cues to try to understand the other interlocutor, foresee possible
future actions, and communicate by making use of visual cues.
In Chapter 3, “Effects of a speaker’s gaze on language comprehension and
interaction” (pp. 47-66), there is a change of focus to language comprehension
and acquisition in young learners. Knoeferle, Kreysa, and Pickering
demonstrate that eye gaze is an essential component in processing and learning
a language as well as in communicative interactions which require the
activation of recalling skills to retrieve previously memorized linguistic
knowledge. These authors start off their chapter with a discussion of the
three main types of accounts that can be explained by tracking eye gaze, i.e.
deictic, referential, and syntactic-thematic. Knoeferle et al then proceed to
present a few studies to attest for the effect that eye gaze exerts on the
recognition of syntactic structures and thematic role assignment in adult
language learners. After that, these authors explore the effect that eye gaze
and action depiction has on linguistic comprehension and learning: whenever
there is no eye gaze available in interaction, action depiction will likely be
more effective; whereas, whenever interlocutors can follow each other’s gazes,
it will serve as a tool to enhance understanding and memorization of the
content of the communicative event. Building on this ongoing discussion,
Knoeferle et al argue and conclude that the existence or lack of socially
contextualized communicative interaction (i.e., adult-child eye gaze and joint
attention) is decisive in child vocabulary comprehension and ultimate
learning, taking into consideration social and pragmatic issues as well.
In Chapter 4, “Weaving oneself into others: Coordination in conversational
systems” (pp. 67-90), Dale and Spivey provide evidence to assert that
communicative situations are systems in and of themselves, where the
interlocutors and their non-linguistic behaviors are synchronous. Dale and
Spivey establish first a theoretical foundation to understand the mechanics of
communication as a system, describing the three types of systems (i.e.,
mechanical, computational, and complex) and highlighting the interrelated
nature among them. After that, these authors discuss the varied array of data
collection methods available to gather information from a complex dynamical
system. These range from the more traditional, reiterative tasks, where
participants can be investigated repetitively for long periods of times, to
dense sampling methods, where multiple data from cognitive processes are
collected by the second in a laboratory setting. Dale and Spivey then devote a
whole subsection to exemplify the use of eyetracking as a dense-sampling
method to calculate language-related human communicative interactions.
Finally, these authors provide a series of assets and caveats that come from
understanding and analyzing communication as a complex system.
Chapter 5 represents the last chapter of the theoretical background of this
book. In “On the role of gaze for successful and efficient communication” (pp.
91- 106), Staude and Crocker provide evidence on the importance of eye gaze in
human communication. They start their chapter by introducing the Visual World
Paradigm, where language, together with gaze and other semiotic resources, is
used to reach communicative understanding, compared to the more traditional
research where the linguistic sphere was the sole focus of investigation.
Staude and Crocker then present a study they carried out to explore several
issues: 1) the benefits of speaker gaze in human-robot communicative
situations, 2) follow-up studies confirming the benefits, and 3) additional
research. The authors try to explain whether the robot gaze impacts on the
human’ (listener) gaze as well as in the comprehension of the communicative
act. They showcase some follow-up studies investigating the devices that bear
such benefits and confirming the hypotheses in 1) above. Then, they discuss
the benefits of listener gaze in situated communicative interactions
presenting more research as examples. These authors conclude that gaze is
indeed a tool that speakers share and benefit from in communication, and that
it fosters comprehension.
Chapter 6, “Quantifying the interplay of gaze and gesture in deixis using
experimental-simulative approach” (pp. 109-137) opens Part two of the book,
which examines methodological considerations in eye tracking research.
Pfeiffer and Renner provide examples of studies that quantify and depict eye
gaze and gesture in 3D spaces. Then they discuss three studies they carried
out where they investigated participants’ movements by means of motion
capturing (i.e.: gesture only, gaze only, and gesture-gaze combination). Under
an experimental-simulative approach, these authors developed a 3D situation
model, the EyeSee3D to track such gestures, by means of pointing rays and 3D
heat maps among other tools. EyeSee3D allows annotating automatically those
spaces of interest for their research. This tool creates it a 3D situation
model representing the items of interest. After that, annotations are made of
those items participants fixate on by means of the eye tracker. Therefore,
EyeSee3D proved useful to identify and annotate participants’ gaze on specific
items. All in all, this chapter is paramount in this volume because it also
provides evidence regarding the importance of gaze not only in contextualized
communication but also in research focused on spatiality.
In Chapter 7, “Gaze and face-to-face interaction” (pp. 139-167), Bailly,
Mihoub, Wolf, and Elisei show the ways in which gaze patterns show correlation
with the cognitive states of speakers in interaction, such as speaking,
listening, or thinking, together with the roles they may play in such
conversations, for example instruction giver or respondent, and finally the
social relationship among them, e.g., colleague or supervisor. After that,
this information is translated into the accurate and scrupulous creation of
avatars that communicate with humans. Bailly et al argue for the critically
detailed design of statistical models of multimodal human behavior to feed
that information to the avatars in order to foster a more accurate real-life
semblance of human-computer/robot communication. Finally, these authors
discuss some caveats that arise from developing these big data driven
statistical models, as well as research issues that need to be addressed to
improve the fields of machine learning and computer-vision.
Chapter 8, “Automatic analysis of in-the-wild mobile eye-tracking experiments
using object, face and person detection” (pp. 169-194) is the third and last
chapter in Part 2. De Beugher, Brône, and Goedemé start with a theoretical
discussion that justifies their technique and its implementation. Then, they
discuss the design and implementation of a model to annotate data from a
mobile eye-tracker device grounded in computer vision techniques. The
application of this approach to track face-to-face human-human communication
brings some advantages and disadvantages. As for the former, this new
technique permits the faster analysis of the data, but most importantly, it
implies the automatic identification of the human body and specific parts key
to communication, such as head, arms, and torso. After that, the gaze data
recorded is mapped onto these images, insuring better understanding of the
participants’ behavior. Finally, the authors recognize that their technique
creates a data output format that is compatible with other annotation systems,
such as ELAN, which makes the process more effortless. The authors present
some final remarks that highlight the usefulness of their tool and prove how
this chapter fits into this volume. The tool serves to identify objects as
well as faces and people, decreases the time-consuming process of data
analysis, and (based on results from previous studies they carried out) offers
a trustworthy explanation and analysis of gaze and body movement in
interaction.
Chapter 9, “Gaze, addressee selection and turn-taking in three-party
interaction” (pp. 198-231), opens the third and last part of this volume: case
studies. In this chapter, Auer investigates the imperative function of gaze in
conversational situations involving three language users. Grounding their
explanations in conversation analysis, and employing eye tracking technology,
this author claims that eye cues play an important role in selecting 1) the
addressee as well as 2) the following speaker in a conversation. The former
action poses a greater difficulty to the speaker because their gaze can only
be directed to one person at a time when there are several addressees.
Therefore, the speaker may turn their eyes upon the other interlocutors. The
latter action is easier: the speaker will select the next speaker by means of
looking at them in their last turn in the conversation. Auer also discusses
benefits and caveats of using gaze as a tool to select the next speaker in the
interaction. In so doing, the author raises awareness about the complexity of
the study of eye gaze and its application in human-human communication.
In Chapter, 10 “Gaze as a predictor for lexical and gestural alignment” (pp.
233-263), Oben explores the effect that gaze seem to exert on linguistic and
hand gesture selection alignment, where one participant may mirror the
speaker’s lexical choice and gesture actions. The analysis of corpus data from
dyadic conversations in two case studies allows the author to assert that gaze
affects alignment of hand gesture and lexical behavior in different ways. For
example, an addressee is likely later to copy a word used by the speaker if
the speaker uttered the word when looking at the addressee. Hand gesture, by
contrast, will probably not be mirrored if the speaker is looking at the
addressee while making the gesture. Rather, such gesture will be copied more
frequently if the addressee fixates on the gesture while the speaker is
performing such action, pointing at an object for instance. Oben concludes
that this dissimilar behavior can be rooted in the multi-functional nature of
eye gaze in interaction because gaze can be used to attend to visual cues as
well as to display meaning.
In Chapter, 11 “Mobile dual eye-tracking in face-to-face interaction: The case
of deixis and joint attention” (pp. 265-300), Stukenbrock studies the varied
range of behaviors that gaze can display systematically in conversations where
participants point at an identifiable object that they are referring to.
Innovatively, her data were collected from everyday activities, such as
jointly looking for a book at a library. These interactions were recorded
outside of a laboratory by means of mobile eye-tracking technology. More
specifically, the two participants wore a pair of eye-tracking glasses that
recorded their visual behaviors. Data were analyzed from a conversational
analysis and multimodal perspective. Results revealed that eye gaze behavior
in deictic reference are is indispensable for participants to coordinate their
speech, gestures, and orientation and movements of their bodies in a
particular place. This chapter presents a unique depiction of eye gaze in
interactive real-life communicative situations.
Chapter 12, “Display recipiency in an interpreter-mediated dialogue: An
eye-tracking study” (pp. 301-322), is the last example included in the third
section and the last chapter of this book. Vranjes, Bot, Feyaerts, and Brône
observe the role that eye gaze brings upon a multimodal representation of
backchannel responses in an interpreter-mediated conversation. Data were
collected from a naturally occurring communicative situation between three
people at a mental institution, where the patient and the therapist shared no
common language. Therefore, eye gaze in these particular interactions is
paramount, since mutual understanding is contingent upon the interpreter and
other types of communicative means. Data were recorded with participants
wearing head-mounted eye-trackers, which enabled the further analysis of the
data in terms of their conversation comprehension as well as their use of eye
gaze and other bodily gestures. Results expose an uneven use of eye gaze and
backchannel responses among all participants. The inclusion of this chapter is
appealing because this exemplifies a different type of interaction outside a
laboratory where eye tracking was essential to achieve successful
communication. Additionally, and as the researchers point out, the use of
eye-tracking tools allow the further comprehension of the issues underlying
communicative acts that need to be mediated by an interpreter to achieve a
specific goal jointly.
EVALUATION
All chapters fit well into the objectives of the book. They all attest to the
essential role of eye gaze in both human-human and human-robot communicative
situations in a vast array of situations, both in laboratory facilities and in
non-research spaces. The chapters in the theoretical section provide ample
background on the topic, with in-depth discussions of the mechanics of eye
gaze, how it works, and how it can be tracked and analyzed from different
research fields. This first section of the book serves also as food for
thought for further research, which highlights the interdisciplinary nature of
research in eye tracking. All the chapters in Section 2 discuss different
methodological issues that permit the study, analysis, and comprehension of
eye gaze both in non-communicative and communicative situations. These
chapters represent a small compilation of innovative tools that exemplify the
advances this research field has experienced in the last decades. Finally,
chapters in Section 3 represent specific examples where eye tracking has been
fruitful to explain addressee selection, linguistic and hand gesture
selection, and multimodal representation of backchannel responses in
multi-party conversation. These four studies are representative of the vast
possibilities available to track human and robot gaze. Additionally, they
invite the reader to ponder the complexities and consider the design of new
ways to understand eye gaze in communication.
This book may be useful to graduate students pursuing a PhD degree as well as
more experienced scholars interested in eye tracking in general and its
applications to linguistics and cognitive psychology among many other specific
fields. In summary, this volume provides a critical review of the existing
literature, as well as an identification of research niches that are expected
and encouraged to be covered in the future to advance the study of eye gaze in
human- and human-computer interactions.
ABOUT THE REVIEWER
María-José Arrufat-Marqués is a PhD candidate at Universitat Jaume I (Spain).
She holds a BA in English Philology from Universitat Jaume I (Spain), a MA in
English Language Teaching in Multilingual Contexts from Universitat Jaume I
(Spain). And, as a Fulbright scholar, she earned a MA in Applied Second
Language Acquisition from Carnegie Mellon University (USA). María-José’s
research interests include: second language acquisition, interlanguage
pragmatics, pragmatic development, formulaic language, technology-enhanced
language learning and teaching, and language attitudes.
------------------------------------------------------------------------------
*************************** LINGUIST List Support ***************************
The 2019 Fund Drive is under way! Please visit https://funddrive.linguistlist.org
to find out how to donate and check how your university, country or discipline
ranks in the fund drive challenges. Or go directly to the donation site:
https://iufoundation.fundly.com/the-linguist-list-2019
Let's make this a short fund drive!
Please feel free to share the link to our campaign:
https://funddrive.linguistlist.org/donate/
----------------------------------------------------------
LINGUIST List: Vol-30-3795
----------------------------------------------------------
More information about the LINGUIST
mailing list