31.1429, Books: Statistical Significance Testing for Natural Language Processing: Dror, Peled-Cohen, Shlomov, reichart

[The LINGUIST List]

LINGUIST List: Vol-31-1429. Wed Apr 22 2020. ISSN: 1069 - 4875.

Subject: 31.1429, Books: Statistical Significance Testing for Natural Language Processing: Dror, Peled-Cohen, Shlomov, reichart

Moderator: Malgorzata E. Cavar (linguist at linguistlist.org)
Student Moderator: Jeremy Coburn
Managing Editor: Becca Morris
Team: Helen Aristar-Dry, Everett Green, Sarah Robinson, Lauren Perkins, Nils Hjortnaes, Yiwen Zhang, Joshua Sims
Jobs: jobs at linguistlist.org | Conferences: callconf at linguistlist.org | Pubs: pubs at linguistlist.org

Homepage: http://linguistlist.org

Please support the LL editors and operation with a donation at:
           https://funddrive.linguistlist.org/donate/

Editor for this issue: Jeremy Coburn <jecoburn at linguistlist.org>
================================================================


Date: Wed, 22 Apr 2020 22:03:48
From: Brent Beckley [beckley at morganclaypool.com]
Subject: Statistical Significance Testing for Natural Language Processing: Dror, Peled-Cohen, Shlomov, reichart

 


Title: Statistical Significance Testing for Natural Language Processing 
Series Title: Human Language Technologies  

Publication Year: 2020 
Publisher: Morgan & Claypool Publishers
	   http://www.morganclaypool.com
	

Book URL: https://www.morganclaypoolpublishers.com/catalog_Orig/product_info.php?products_id=1528http:// 


Author: Rotem Dror
Author: Lotem Peled-Cohen
Author: Segev Shlomov
Author: Roi Reichart

Paperback: ISBN:  9781681737959 Pages: 116 Price: U.S. $ 49.95


Abstract:

Data-driven experimental analysis has become the main evaluation tool of
Natural Language Processing (NLP) algorithms. In fact, in the last decade, it
has become rare to see an NLP paper, particularly one that proposes a new
algorithm, that does not include extensive experimental analysis, and the
number of involved tasks, datasets, domains, and languages is constantly
growing. This emphasis on empirical results highlights the role of statistical
significance testing in NLP research: If we, as a community, rely on empirical
evaluation to validate our hypotheses and reveal the correct language
processing mechanisms, we better be sure that our results are not
coincidental.

The goal of this book is to discuss the main aspects of statistical
significance testing in NLP. Our guiding assumption throughout the book is
that the basic question NLP researchers and engineers deal with is whether or
not one algorithm can be considered better than another one. This question
drives the field forward as it allows the constant progress of developing
better technology for language processing challenges. In practice, researchers
and engineers would like to draw the right conclusion from a limited set of
experiments, and this conclusion should hold for other experiments with
datasets they do not have at their disposal or that they cannot perform due to
limited time and resources. The book hence discusses the opportunities and
challenges in using statistical significance testing in NLP, from the point of
view of experimental comparison between two algorithms. We cover topics such
as choosing an appropriate significance test for the major NLP tasks, dealing
with the unique aspects of significance testing for non-convex deep neural
networks, accounting for a large number of comparisons between two NLP
algorithms in a statistically valid manner (multiple hypothesis testing), and,
finally, the unique challenges yielded by the nature of the data and practices
of the field.
 



Linguistic Field(s): General Linguistics


Written In: English  (eng)

See this book announcement on our website: 
http://linguistlist.org/pubs/books/get-book.cfm?BookID=143253




------------------------------------------------------------------------------

***************************    LINGUIST List Support    ***************************
 The 2019 Fund Drive is under way! Please visit https://funddrive.linguistlist.org
  to find out how to donate and check how your university, country or discipline
     ranks in the fund drive challenges. Or go directly to the donation site:
               https://iufoundation.fundly.com/the-linguist-list-2019

                        Let's make this a short fund drive!
                Please feel free to share the link to our campaign:
                    https://funddrive.linguistlist.org/donate/
 


----------------------------------------------------------
LINGUIST List: Vol-31-1429	
----------------------------------------------------------