16.3460, Review: Applied Ling/Methodology: Perry (2005)

[LINGUIST List]

LINGUIST List: Vol-16-3460. Sun Dec 04 2005. ISSN: 1068 - 4875.

Subject: 16.3460, Review: Applied Ling/Methodology: Perry (2005)

Moderators: Anthony Aristar, Wayne State U <aristar at linguistlist.org>
            Helen Aristar-Dry, Eastern Michigan U <hdry at linguistlist.org>
 
Reviews (reviews at linguistlist.org) 
        Sheila Dooley, U of Arizona  
        Terry Langendoen, U of Arizona  

Homepage: http://linguistlist.org/

The LINGUIST List is funded by Eastern Michigan University, Wayne
State University, and donations from subscribers and publishers.

Editor for this issue: Lindsay Butler <lindsay at linguistlist.org>
================================================================  

What follows is a review or discussion note contributed to our 
Book Discussion Forum. We expect discussions to be informal and 
interactive; and the author of the book discussed is cordially 
invited to join in. If you are interested in leading a book 
discussion, look for books announced on LINGUIST as "available 
for review." Then contact Sheila Dooley at dooley at linguistlist.org. 

===========================Directory==============================  

1)
Date: 02-Dec-2005
From: Richard Todd < irictodd at kmutt.ac.th >
Subject: Research in Applied Linguistics: Becoming a Discerning Consumer 

	
-------------------------Message 1 ---------------------------------- 
Date: Sun, 04 Dec 2005 19:36:26
From: Richard Todd < irictodd at kmutt.ac.th >
Subject: Research in Applied Linguistics: Becoming a Discerning Consumer 
 

AUTHOR: Perry, Fred L. Jr.
TITLE: Research in Applied Linguistics
SUBTITLE: Becoming a Discerning Consumer
PUBLISHER: Lawrence Erlbaum Associates
YEAR: 2005
Announced at http://linguistlist.org/issues/16/16-2736.html 

Richard Watson Todd, Department of Language Studies, King 
Mongkut's University of Technology Thonburi, Thailand

SUMMARY OF CONTENTS

This book is designed as a text for courses on research methods on 
Masters-level TESOL and applied linguistics programmes. It claims to 
provide ''a solid introduction to the foundations of research methods'', 
but, in contrast to most texts in the same area, it aims to help MA 
students read, understand and criticise research rather than conduct 
their own research. Thus, the purpose of the book is for readers to 
become the ''discerning consumers'' of research of the book's title.

In order to achieve the goal of enabling readers of the book to ''read 
research reports from beginning to end with a level of understanding 
that can be used to address both theoretical and practical issues'', the 
book starts with two introductory chapters examining the nature of 
research and finding research before moving on to the main part of 
the book which proceeds through the different sections of a typical 
research article. There are also three appendices on writing literature 
reviews, statistics and journals which publish applied linguistics 
research, together with a glossary. I will briefly look at each of the 
chapters in turn.

Part 1: Fundamentals for discerning consumers

Chapter 1: Understanding the nature of research
The first chapter starts by justifying the purpose of the book, namely, 
that readers should become ''discerning consumers'' of research. It 
then moves to discussing the nature of research which allows many of 
the key terms concerning research (e.g. construct, hypothesis) to be 
introduced. In common with all of the chapters, in addition to the 
explanatory text, there are a couple of short exercises for readers to 
attempt, a list of the key terms introduced in the chapter (most of 
which can be found in the glossary), and suggestions for further 
reading.

Chapter 2: How to locate research
This chapter gives advice on how to locate and obtain examples of 
primary research. It includes a substantial section on how to use the 
ERIC database, but also considers looking at literature reviews and 
position papers.

Part 2: The major components of published research

Chapter 3: Understanding the framework of a primary research article
In this chapter, the reader is taken briefly through the main 
components of a research article, namely, title, abstract, introduction, 
methodology, results, and discussion/conclusion.

Chapter 4: Understanding where data come from: the sample
This chapter is devoted to the issue of how the research sample is 
selected, and contrasts the information-rich sampling paradigm with 
the representative sampling paradigm. Guidelines are given for 
evaluating the suitability of the sampling strategies used and the 
generalisability of the sampling.

Chapter 5: Understanding research designs
Starting with ways of classifying research designs, this chapter 
introduces three continua for classifying research: basic - applied, 
qualitative - quantitative, and exploratory - confirmatory. In the 
discussion of these continua, three qualitative research paradigms, 
case studies, ethnography and protocol analysis, are briefly discussed 
as illustrations. The chapter then moves on to how 'what' and 'why' 
research questions can be answered through different research 
designs. The chapter finishes with an in-depth discussion of all the 
various threats to internal validity, largely as they relate to conducting 
experimental research.

Chapter 6: Understanding data gathering
In the first half of this chapter, a variety of data collection procedures 
are discussed. The emphasis here is on collecting data from people 
through observation, questionnaires and tests (as opposed to 
collecting texts to analyse), and on the potential pitfalls in data 
collection. The second half of the chapter concerns the reliability and 
validity of data gathering procedures, focusing especially on the 
statistics available for evaluating these issues.

Chapter 7: Understanding research results
Divided into sections on verbal and numerical data, the chapter starts 
with a discussion of 17 ''tactics'' for checking the quality of verbal data. 
The second half of the chapter which concerns numerical data 
focuses on statistics, especially on the meaning of statistical 
significance as it applies to generalising results from a sample to a 
population.

Chapter 8: Discerning discussions and conclusions: completing the 
picture
Much shorter than the preceding chapters, the final chapter gives 7 
guiding questions for evaluating the discussion/conclusion section of a 
research study, and looks at how these can be applied to two example 
articles.

Appendix A: Constructing a literature review
This appendix reviews chapter 2 on locating research, gives 
guidelines on how to summarise a research study, and presents a 
template for writing a research review.

Appendix B: Going to the next level of statistics
The second appendix provides more details of statistics to add to the 
discussion in chapter 7. Particular emphasis is given to the various 
types of ANOVA and controlling for Type II errors.

Appendix C: Journals related to applied linguistics
The final appendix contains a useful, if not comprehensive (e.g. no 
reference to 'Prospect' or 'Journal of English for Academic Purposes'), 
list of research journals in applied linguistics with URLs and purposes 
of the journals.

CRITICAL EVALUATION

Since 'Research in Applied Linguistics' is designed to be used as a 
text for MA courses on research methods, how well does it fulfill this 
purpose? To start with, let us look at what the text does not do. As a 
book devoted to how to read research, it does not really prepare 
readers for conducting their own research, although many of the 
points in the book are applicable. Even within the limits of reading 
research, the book does not give coverage to some paradigms. There 
is nothing on the less formal research approaches, such as action 
research, and a similar dearth of information on text-based research 
paradigms including genre analysis, policy research and corpus 
approaches. Furthermore, although there are short sections on 
qualitative research, the book places a much heavier emphasis on 
quantitative approaches. 'Research in Applied Linguistics', therefore, 
is really a book on how to read and understand serious academic 
research, primarily focusing on quantitative research paradigms.

So, within this narrower field, how does the book stand up? First, the 
book is replete with useful information about how research should be 
conducted and reported, some of which, such as the various threats 
to internal validity, I was not familiar with. This information is generally 
presented clearly and is well illustrated with examples from sample 
research studies.

There is, however, a general feeling that the book is nitpicking about 
research procedures and overlooking the bigger picture. For example, 
the suggestions for summarising research in Appendix A include an 8-
point template. These points include: the hypothesis being tested (in 
addition to the research question); the size, characteristics and 
methods of choosing the sample; and the observational, independent, 
dependent and moderating variables in the study. Surely, experienced 
readers of research focus on much bigger issues than these points 
(maybe, the appropriateness and usefulness of the topic or the 
innovativeness of the methodology) when reading research, and only 
consider the research technicalities when they are sure that the 
research is worth spending time reading.

'Research in Applied Linguistics', then, is a book focusing on the 
details of research. This is an important topic well worth a full-length 
publication like the one under review. Even at the level of details, 
however, there are a few problems. For example, on page 83 a 
supposed example of exploratory research using a quantitative 
research design is given. The author of the book argues that this is 
exploratory research since it did not state a hypothesis. However, the 
research concerned whether bilingualism influences the learning of a 
third language - which clearly implies a hypothesis even if it is not 
explicitly stated. The research is therefore not exploratory and so 
does not illustrate the point the author wishes to make. Another 
problem, on page 130, is that the author states that correlation 
coefficients range from 0 to +1 (not -1 to +1). Also, the discussion of 
the meaning of statistical significance is restricted to whether findings 
from a sample can be generalised to a population. Not all research 
using inferential statistics is trying to generalise, and significance can 
also indicate the probability that the results obtained were not due to 
chance. In a book of this length, however, these problems are minor 
annoyances.

To summarise my views on the book, therefore, despite its broad title, 
it is quite a narrow book. If you are looking for a clear text explicating 
the details of reading and understanding academic quantitative 
research, this book serves that purpose well. I would guess, however, 
that most MA courses on research methods have broader goals than 
this, and so the book may serve best as a useful supplementary text 
for certain portions of a course.

My own view as a tutor on an MA course in research methods, 
however, is perhaps not the most important. The book is designed for 
Masters students to use. In addition to my own critical evaluation, 
therefore, I asked one of my students (Nguyen Quang Tuyen), who is 
currently studying the research methods course at my university, to 
also read and comment on the book. Below is a summary of his review 
of the book.

Firstly, on the positive side, Tuyen says that, ''as an MA student, I 
have 2 major concerns regarding research: how to evaluate and how 
to conduct research. Therefore, I highly value this book because it 
focuses on the first, which is different from most traditional works in 
the field.'' He also states that the websites, lists of journals and 
frequent examples through the text are useful, and that the language 
is suitable for his level as a non-native speaker. Furthermore, he 
found the format of the chapters helpful, since they are organised 
from general to specific, give clear definitions, list key words, illustrate 
points with examples of published research, and contain exercises to 
reinforce learning.

On the downside, he points out that the book binding is not strong 
enough (a point I can concur with since several pages were loose 
when he returned the book to me). For the content, he would have 
preferred more cross-references, especially for where the same 
research study is used as an illustration in different chapters, he found 
the lack of any support or feedback for completing the exercises 
meant that he was uncertain about his answers, and he found some 
points to be unnecessarily over-complicated. Like me, he was left 
wondering why action research was not dealt with in the book. 
Concerning the style, his expectations had been raised in the author's 
introduction where humour is promised, but he was disappointed that 
the ''humour was not as much as expected''.

To conclude Tuyen's review, he states, ''in short, the strengths 
outweigh the limitations, so I wish to combine this book with the 
traditional ones as both the abilities to do and to evaluate research 
are interrelated and support each other.''

It can be seen that there are some similarities between Tuyen's and 
my own reviews, and yet the emphases in the reviews are very 
different. Apart from highlighting the fact that all book reviews are 
primarily subjective, our differences seem to indicate that our goals in 
reading the book (Tuyen was looking for new information, whereas I 
was viewing the book as a potential coursebook), our concerns with 
what we want from the book (Tuyen was pleased to find a book 
focusing on reading research, whereas I would have preferred 
general coverage of most issues in research), and the ways in which 
we read the book (Tuyen worked through all the exercises, whereas I 
skipped them) all had a major impact on our reviews. 

ABOUT THE REVIEWER

Richard Watson Todd has worked at King Mongkut's University of 
Technology Thonburi in Bangkok for over ten years, where his work 
includes teaching a Masters course on research methods. He has a 
PhD from the University of Liverpool and has published widely, 
including books, research papers and newspaper articles.





-----------------------------------------------------------
LINGUIST List: Vol-16-3460