30.720, Review: Computational Linguistics: Moorkens, Castilho, Gaspari, Doherty (2018)
The LINGUIST List
linguist at listserv.linguistlist.org
Wed Feb 13 20:20:49 UTC 2019
LINGUIST List: Vol-30-720. Wed Feb 13 2019. ISSN: 1069 - 4875.
Subject: 30.720, Review: Computational Linguistics: Moorkens, Castilho, Gaspari, Doherty (2018)
Moderator: Malgorzata E. Cavar (linguist at linguistlist.org)
Student Moderator: Jeremy Coburn
Managing Editor: Becca Morris
Team: Helen Aristar-Dry, Everett Green, Sarah Robinson, Peace Han, Nils Hjortnaes, Yiwen Zhang, Julian Dietrich
Jobs: jobs at linguistlist.org | Conferences: callconf at linguistlist.org | Pubs: pubs at linguistlist.org
Homepage: http://linguistlist.org
Please support the LL editors and operation with a donation at:
https://funddrive.linguistlist.org/donate/
Editor for this issue: Jeremy Coburn <jecoburn at linguistlist.org>
================================================================
Date: Wed, 13 Feb 2019 15:20:32
From: Zhi Huang [jeffzhihuang at gmail.com]
Subject: Translation Quality Assessment
Discuss this message:
http://linguistlist.org/pubs/reviews/get-review.cfm?subid=36452917
Book announced at http://linguistlist.org/issues/29/29-3321.html
EDITOR: Joss Moorkens
EDITOR: Sheila Castilho
EDITOR: Federico Gaspari
EDITOR: Stephen Doherty
TITLE: Translation Quality Assessment
SUBTITLE: From Principles to Practice
SERIES TITLE: Machine Translation: Technologies and Applications
PUBLISHER: Springer
YEAR: 2018
REVIEWER: Zhi Huang, Macquarie University
SUMMARY
“Translation Quality Assessment: From Principles to Practice”, edited by Joss
Moorkens, Sheila Castilho, Federico Gaspari and Stephen Doherty, appears in
Volume 1 of book series “Machine Translation: Technologies and Applications”,
edited by Andy Way. This volume has three parts, each focusing on one aspect
of translation quality assessment. Part I discusses scenarios for translation
quality assessment and contains four papers. Part II explores the development
of applications of translation quality assessment and includes four papers.
Part III examines translation quality assessment in practice and contains
three papers. A combination of the articles in this book tries to link
principles with practice in translation quality assessment, including
descriptions of approaches to translation quality assessment, metrics and
error analysis for translation quality assessment, as well as applications of
machine translation and future perspectives on its quality. The book is
intended for those who are interested in machine translation evaluation,
translation quality assessment technologies, and applications and practice of
translation quality assessment.
Part I: Scenarios for Translation Quality Assessment
Paper 1: Approaches to Human and Machine Translation Quality Assessment
(Sheila Castilho, Stephen Doherty, Federico Gaspari, and Joss Moorkens)
This paper introduces human and machine translation quality assessment and
approaches to the measurement of translation quality in different settings. A
number of automatic machine translation evaluation metrics are discussed in
terms of standards, consistency, social quality, risks, and implications for
education and training. Examples include Word Error Rate used by Nieben et al.
(2000), Translation Error Rate by Snover et al. (2006) and Bilingual
Evaluation Understudy by Papineni et al. (2002). It provides an insight into
the current knowledge and practice of translation quality assessment in
research, industry and education.
Paper 2: Translation Quality, Quality Management and Agency: Principles and
Practice in the European Union Institutions (Joanna Drugan, Ingemar Strandvik,
and Erkka Vuorinen)
This paper explores translation quality management and agency in the setting
of the European Union institutions, particularly European Commission’s
Directorate-General for Translation (DGT). DGT’s quality management model and
practical quality management tools are discussed, followed by challenges in
consistency of approaches and quality. The impact and implications of
translation quality management are also considered, with regard to power,
agency, professionalism, and values.
Paper 3: Crowdsourcing Translation Quality: Novel Approaches in the Language
Industry and Translation Studies (Miguel A. Jimenez-Crespo)
In this paper, the practice of crowdsourcing is introduced to explore its
impact on translation quality. The papaer reviews the consolidation of
process-based approaches to guarantee quality, the expansion of the fitness
for purpose model, and the distribution of responsibility to different agents.
The paper also explores novel practices and workflows to guarantee quality
focusing on contrasting professionals or crowdsourcing participants to assess
the quality of MT output.
Paper 4: On Education and Training in Translation Quality Assessment (Stephen
Doherty, Joss Moorkens, Federico Gaspari, and Sheila Castilho)
This paper acknowledges the lack of education and training in translation
quality assessment and introduces a range of viewpoints and resources for the
provision of education and training in academic settings. It proposes
Translation Quality Assessment models for education and training and provides
a guide to help educators and translators choose the various types of TQA
compatible with their own scenarios. At the end, the paper mentions the
current status of TQA education and training and gives predicts for its
development trend, confirming that the industry will have its own metrics and
models that will drive the industry to change for the better.
Part II: Development of Applications of Translation Quality Assessment
Paper 5: Metrics for Translation Quality Assessment: A Case for Standardising
Error Typologies (Arle Lommel)
This paper starts with an introduction of the Multidimensional Quality Metrics
(MQM) and Dynamic Quality Framework (DQF) projects for translation quality
assessment. An overview of MQM is detailed focusing on its existing approaches
and the overall structure of MQM including hierarchy, dimensions,
specifications, severities and weights, scoring, and holistic vs analytic
evaluation. The DQF error typology is then discussed and integrated with MQM
with concluding remarks on the status and plans for the future, offering a way
to escape the inconsistency and subjectivity that have so far characterised
TQA.
Paper 6: Error Classification and Analysis for Machine Translation Quality
Assessment (Maja Popovic)
This paper explores the different approaches and tasks to analyse errors in
machine translation output. It starts with an introduction to manual error
classification and error typologies, providing a table of error categories,
followed by an overview of error typologies and tasks with detailed analysis.
The challenges for automatic error classification are also discussed and
analysed with an evaluation as well. Other methods for error analysis are also
explored, such as analysis of (un)matched sequences, and checking and
evaluating specific linguistic features.
Paper 7: Quality Expectations of Machine Translation (Andy Way)
This paper addresses whether machine translation can be useful for human
translators, especially as a productivity enhancer. It starts with the
background information on machine translation use and evaluation at present,
and then considers the inherent problems with automatic evaluation metrics and
problems with automatic evaluation use. It also answers the question whether
automatic evaluation corroborates human evaluation. Finally, the paper
discusses how MT is currently applied and what expectations and perceptions of
MT quality are. The author concludes this paper with a prediction of its
continued use as a production tool besides translation memory.
Paper 8: Assessing Quality in Human- and Machine-Generated Subtitles and
Captions (Stephen Doherty and Jan-Louis Kruger)
This paper explores the current and future issues in quality assessment in
human and machine-generated subtitling and captioning. The guiding principles
for quality in audiovisual translation (AVT), such as accuracy, presentation,
timing and error-based metrics, are discussed first, followed by the
implications and insights from AVT research with a focus on the cognitive load
and immersion in subtitled and captioned media. Finally, the paper raises
challenges and possible solutions for stakeholders to consider in order to
encourage dialogue between disciplines so that the quality in AVT can be
enhanced and further developed in such an evolving technological environment.
Part III: Translation Quality Assessment in Practice
Paper 9: Machine Translation Quality Estimation: Applications and Future
Perspectives (Lucia Specia and Kashif Shah)
This paper addresses the importance of predicting the quality of machine
translation output with a review of various practical applications.
Experiments are carried out to show positive results in quality estimation at
sentence level, that is, filtering low quality cases from post-editing,
selecting the best machine translation system under multiple choices,
improving machine translation performance by selecting parallel data and
sampling for human quality assurance. In the conclusion, the author asserts
that the approaches to quality estimation have the potential to make machine
translation more useful to end-users.
Paper 10: Machine Translation and Self-post-editing for Academic Writing
Support: Quality Explorations (Sharon O’Brien, Michel Simard, and Marie-Josee
Goulet)
This paper discusses the possibility of using machine translation and
self-post-editing as a second-language academic writing aid through a number
of quality assessment measures. It also compares participant perceptions,
temporal effort and revisions required. The results show that more
participants prefer to use machine translation again in the future to support
their writing process and participants believe that the quality is maintained.
This suggests a high potential of using machine translation and
self-post-editing as a useful tool when accessing international academic
publishing via the current Lingua Franca of English so that the cognitive
burden of the authors can be reduced.
Paper 11: What Level of Quality Can Neural Machine Translation Attain on
Literary Text? (Antonio Toral and Andy Way)
This paper introduces a new neural approach to machine translation and
assesses the quality attainable for novels by the two most common paradigms to
machine translation, NMT and PBSMT. The authors built the first in-domain
PBSMT and NMT systems for literary text by training them on large volumes of
parallel novels. With a comparison of the output from machine translation
systems and that from published human translations, surprisingly promising
results are shown, especially considering this special text type. It is of
great value for future research in terms of the assessment of the feasibility
of using machine translation to assist with the translation of literary text.
EVALUATION
This book is nicely presented with each chapter contributing an important
aspect of machine translation quality assessment. The focus of this book is on
the product of translation, describing scenarios for translation quality
assessment and developing applications of translation quality assessment. The
three parts of this book cover the practical side of translation quality
assessment in an emerging technological society. At this critical moment for
the translation industry, when fast-paced economy and social development
require more efficient and effective translation products, I believe this book
provides essential knowledge and insights into translation technology and
quality assurance. Its target readers can be all kinds of industry
practitioners as well as enthusiasts, including but not limited to
translators, teachers, students, researchers or business managers. The book
embraces dominant methods from different translation scenarios, and provides a
comprehensive collection of contributions by international experts in
translation quality assessment and human and machine translation evaluation.
The whole book is well structured to form a picture of human and machine
translation quality assessment from principles to practice.
Although it is understandable that this book focuses on machine translation
quality assessment because of the fast development of technology and the trend
of using such a technology in translation, my suggestion would be some more
focus on human translation quality assessment and the comparison of human and
machine translation quality assessment, to enrich the range of topics covered
under this title “Translation Quality Assessment”. Without a balance of
research on both human and machine translation quality assessment, I think the
book may be more suitably called “Machine Translation Quality Assessment”. I
would also be interested to read more articles about the applications of
machine translation quality assessment in practical scenarios including
translation teacher development, translation teaching techniques, use of
machine translation in more settings, and the possibility of using machine
translation on more text types.
REFERENCES
Nieben, Sonja, Franz Jisef Och, George Leusch, and Hermann Ney. 2000. An
evaluation tool for machine translation: fast evaluation for MT research. In:
Proceedings of the second international conference on language resources and
evaluation, Athens, 31 May-2 June 2000. 39-45.
Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John
Makhoul. 2006. A study of translation edit rate with targeted human
annotation. In: Proceedings of the 7th conference of the Association for
Machine Translation in the Americas: “Visions for the future of Machine
Translation”, Cambridge, 8-12 August 2006. 223-231.
Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a
method for automatic evaluation of machine translation. In: Proceedings of the
40th annual meeting on Association for Computational Linguistics,
Philadelphia. 311-318.
ABOUT THE REVIEWER
Zhi Huang is a PhD candidate at Macquarie University focusing on translation
teacher effectiveness and effective teaching qualities. He is an Australian
NAATI certified translator between Chinese and English languages. He has
completed Master of Advanced Translation at Macquarie University and Master of
Education in TESOL at the University of Sydney. His research interests involve
English language teaching, teacher quality, translation theory and pedagogy.
He has published articles at English Language Teaching, T&I Review, and
Journal of Language Teaching and Learning.
------------------------------------------------------------------------------
***************** LINGUIST List Support *****************
Please support the LL editors and operation with a donation at:
The IU Foundation Crowd Funding site:
https://iufoundation.fundly.com/the-linguist-list
The LINGUIST List FundDrive Page:
https://funddrive.linguistlist.org/donate/
----------------------------------------------------------
LINGUIST List: Vol-30-720
----------------------------------------------------------
More information about the LINGUIST
mailing list