25.4834, Review: Applied Ling; Computational Ling: Green (2013)

[The LINGUIST List via LINGUIST]

LINGUIST List: Vol-25-4834. Mon Dec 01 2014. ISSN: 1069 - 4875.

Subject: 25.4834, Review: Applied Ling; Computational Ling: Green (2013)

Moderators: Damir Cavar, Indiana U <damir at linguistlist.org>
            Malgorzata E. Cavar, Indiana U <gosia at linguistlist.org>

Reviews: reviews at linguistlist.org
Anthony Aristar <aristar at linguistlist.org>
Helen Aristar-Dry <hdry at linguistlist.org>
Sara Couture, Indiana U <sara at linguistlist.org>

Homepage: http://linguistlist.org

Do you want to donate to LINGUIST without spending an extra penny? Bookmark
the Amazon link for your country below; then use it whenever you buy from
Amazon!

USA: http://www.amazon.com/?_encoding=UTF8&tag=linguistlist-20
Britain: http://www.amazon.co.uk/?_encoding=UTF8&tag=linguistlist-21
Germany: http://www.amazon.de/?_encoding=UTF8&tag=linguistlistd-21
Japan: http://www.amazon.co.jp/?_encoding=UTF8&tag=linguistlist-22
Canada: http://www.amazon.ca/?_encoding=UTF8&tag=linguistlistc-20
France: http://www.amazon.fr/?_encoding=UTF8&tag=linguistlistf-21

For more information on the LINGUIST Amazon store please visit our
FAQ at http://linguistlist.org/amazon-faq.cfm.

Editor for this issue: Sara  Couture <sara at linguistlist.org>
================================================================


Date: Mon, 01 Dec 2014 15:52:44
From: Elizaveta Tarasova [etarasova at ipc.ac.nz, elizaveta.tarasova1 at gmail.com]
Subject: Statistical Analyses for Language Testers

E-mail this message to a friend:
http://linguistlist.org/issues/emailmessage/verification.cfm?iss=25-4834.html&submissionid=31010737&topicid=9&msgnumber=1
 
Discuss this message: 
http://linguistlist.org/pubs/reviews/get-review.cfm?subid=31010737


Book announced at http://linguistlist.org/issues/24/24-1923.html

AUTHOR: Rita  Green
TITLE: Statistical Analyses for Language Testers
PUBLISHER: Palgrave Macmillan
YEAR: 2013

REVIEWER: Elizaveta Tarasova, International Pacific College

Review's Editor: Helen Aristar-Dry

SUMMARY

There is a large number of textbooks currently on the market that aim to
explain the basics of statistical analyses, yet only few of them focus on
readers from the field of language testing. Rita Green’s recent book
“Statistical Analyses for Language Testers” (SALT) sets an ambitious goal of
providing language teachers and test developers with the knowledge and
practical tools that can enhance their understanding of what contributes to
the validity and reliability of a language test. The textbook is a practical
introduction to the most useful statistical analyses for language test
developers and researchers based on the programs IBM SPSS, Winsteps and
Facets. The first of these is one of the most popular statistics software
packages mainly because it is so user-friendly. The latter two programmes (as
persuasively demonstrated by the author) complement SPSS outputs by providing
opportunities for more detailed and complex analysis of data.

The 16 chapters of SALT cover a wide range of topics from the most basic to
quite advanced. The chapters have a very user-friendly outline: each chapter
focuses on a particular type of analysis and begins with an introduction that
explains the value of this particular analysis to the reader. This is followed
by a (very) brief explanation of the key terms and concepts that are used in
the chapter. The description of each analysis is very detailed, with
step-by-step instructions and an exhaustive translation and deciphering of
outputs. Nevertheless, the explanations are not overloaded with information
and only cover the details that are essential for understanding the core
principles of the analysis and the results it yields. Each chapter ends with a
task for further practice of newly gained skills, where the reader is referred
to one of the Appendices (2-13) that contains additional data sets to practice
the analyses, questions for the reader and answers for self-check. The book
has an adequate table of contents, a list of the symbols and most common
abbreviations (which the author calls acronyms) used in the book, a list of
appendices as well as appendices with additional data for further practice, a
reference list, a list of recommended sources for further reading and an
index. The data files used in the text can be found at
http://www.palgrave.com/language/salt/.

The textbook can be logically divided into two parts, with the first one
(Chapters 1-9) discussing SPSS software and the second one (Chapters 10-16)
dealing with Winsteps and Facets. The level of difficulty of the information
presented in each chapter is indicated by symbols (the author uses pictures of
boots), the more boots – the more difficult the chapter is going to be.
Chapters 1 and 2 gently introduce the reader to the Statistical Package for
Social Sciences (SPSS) by explaining what different types of variables there
can be, how to enter the data and variable details as well as how to identify
and deal with errors in the dataset. Chapter 3 concentrates on the
peculiarities of item analysis. It starts with a brief description of some of
the main operational concepts that are used in statistics, e.g. facility
value, measurements of central tendency (mean, mode and median), range of
scores, etc. This is followed by detailed instructions on how to perform the
test and what the obtained results mean. Chapter 4 looks into descriptive
statistics, covering such stages of the analysis as computing a variable,
creating and interpreting a histogram and a bar chart and obtaining
information in a non-graphic form, as well as comparing two distinct groups in
the dataset. Chapter 5 covers the peculiarities of dealing with the data
obtained from test taker feedback questionnaires and comparing the responses
of the test takers with their performance on the test. Chapters 6-9 explore
the concept of correlation in detail. Chapter 6 goes over the ways of
comparing performances of a test taker on two different tests to see if there
is a correlation between the results. Chapter 7 moves from descriptive
statistics to inferential statistics and t-tests in order explain how to make
generalizations about the relationships or differences that can be found
between two variables. Chapter 8 concentrates on the ANOVA (Analysis of
Variance) test, the test that allows for investigating the differences that
are not due to chance between groups of test takers. 

In Chapter 9 the reader is introduced to factor analysis, which gives a
researcher an opportunity to see what factors may underlie the correlations
demonstrated by the data. Chapters 10-13 and 14-16 give the reader an
opportunity to get acquainted with Winsteps and Facets respectively. Chapter
10 explains how to create a control file from an SPSS data file and how then
to create a convergence table. The next three chapters concentrate on the
concept of fit statistics and some of the analyses that Winsteps has to offer,
including item and person statistics and distracter analysis. These allow the
test developer to see whether the data obtained from the test accurately
reflect test takers’ abilities and whether the level of difficulty of the
items comprising a test is suitable for the level of the test taker. Chapter
14 introduces the statistical program Facets and demonstrates how to work with
a file using an SPSS data set.  The final two chapters of the book inform the
reader about the possibilities of the analyses in Facets, a programme that
allows to concentrate on specific variables. The concept of quality control
fit statistics is re-visited, and the explanations of peculiarities of an
inter-rater agreement analysis is discussed.

EVALUATION

It is often remarked that language teachers are usually not very good at
calculations, and the word ‘maths’ may sometimes cause us a bit of anxiety. At
the same time, the requirements of today’s world put a lot of pressure on
language teachers to ensure that when our learners’ skills and abilities are
tested, they correspond to international standards. Developing tests for
learners is part of a teacher’s everyday job; therefore, the knowledge of how
to improve the quality of the tests and how to check whether the tests provide
valid and reliable data on the students’ progress and/or performance is of
utmost importance. SALT by Rita Green provides a concise and focused yet
exhaustive and practical introduction to the use of statistical software
packages that will be appreciated by language teachers and test developers,
especially those who have little or no experience of dealing with statistics.

>From the very first pages, the textbook strikes the reader as being applied
and experiential, with little or no prerequisite knowledge of statistics
required for grasping the concepts discussed by the author. I also found it
appealing that the amount of theoretical input is limited to the degree
necessary for understanding how things work and why they work this way.  The
author measures the amount of theoretical information that needs to be fed to
the reader very carefully, and deliberately avoids the use of statistical
symbols and formulas, so that even complete beginners do not feel intimidated.
The author does not go too deep into the discussion of theoretical
prerequisites rather concentrating on the practical matters. The list of
references will provide those who are interested with enough reading for
further self-study. All this reflects the author’s overall approach to how the
statistical analyses should be treated, i.e. in terms of their use in the
field of language testing. All the data that is used in the textbook comes
from real tests that were developed for real purposes, which makes the text
accessible, relevant  to the intended audience (language teachers and test
developers) and easy to follow for anyone who has ever set out to develop a
language test, even though they may not have training in statistical analyses.

The outline of the book is easy to follow.  Information about the key terms is
presented in the form of bullet points and is complemented by the detailed
analysis of test data, snapshots of pop-up windows and of tables with outputs,
together with a meticulously detailed explanation of what the reader sees on
the screen when they perform this or that type of test. The “learn-by doing”
approach adopted in the book resembles a language drill, where the learner
attempts to produce the target structure in a safe environment for the very
first time under the strict control of the teacher. This gives the reader a
clear understanding of how exactly to perform the tests and how to interpret
the data, and also creates a learning environment where nothing can go
terribly wrong because the instructor has taken care of everything.  

There are only a few minor points for criticism, mainly of a technical nature.
One of these is that the list of recommended further readings contains only
three sources; it would be helpful to have a larger choice. 

Another issue concerns the data files that the author uses in the book, or,
rather the lack of explicit instructions on where to find them. For example,
in Chapter 2 (p. 13) readers are requested to open the data file “Reading test
data with errors”; however, there is no explicit reference to the location of
this file. It took me some time to realize that there may be a supporting web
page and then I had to look for it online. Once the webpage was discovered, I
had no problem downloading the files and working with them, but it would be
helpful to inform the reader about it. 

As I mentioned above, the limited amount of theoretical input and the strong
focus on technical details is very advantageous on the one hand. On the other
hand, it can be viewed as a weakness, since the book may often sound like a
language teaching-focused manual for using the software. It would be good to
hear the author’s voice more often.

These reservations are very minor, and I can strongly recommend the book to
all teachers and test developers who are striving to improve the quality of
their tests. The book is especially useful for self-study and is very
effective in achieving its goal of providing test developers with practical
skills and basic knowledge of SPSS, Winsteps and Facets in their work.


ABOUT THE REVIEWER

Dr. Elizaveta Tarasova is a TESOL co-ordinator/lecturer at IPC, New Zealand.
Her research interests include a wide range of subjects in Theoretical and
Applied Linguistics. Having recently completed her PhD project on regularities
in the formation of English N+N compounds, she is now working on preparing her
research for publication.








----------------------------------------------------------
LINGUIST List: Vol-25-4834	
----------------------------------------------------------