16.840, Review: Applied Ling/Cognitive Science: Karreman (2004)

[LINGUIST List]

LINGUIST List: Vol-16-840. Sat Mar 19 2005. ISSN: 1068 - 4875.

Subject: 16.840, Review: Applied Ling/Cognitive Science: Karreman (2004)

Moderators: Anthony Aristar, Wayne State U <aristar at linguistlist.org>
            Helen Aristar-Dry, Eastern Michigan U <hdry at linguistlist.org>
 
Reviews (reviews at linguistlist.org) 
        Sheila Collberg, U of Arizona  
        Terry Langendoen, U of Arizona  

Homepage: http://linguistlist.org/

The LINGUIST List is funded by Eastern Michigan University, Wayne
State University, and donations from subscribers and publishers.

Editor for this issue: Naomi Ogasawara <naomi at linguistlist.org>
================================================================  

What follows is a review or discussion note contributed to our 
Book Discussion Forum. We expect discussions to be informal and 
interactive; and the author of the book discussed is cordially 
invited to join in. If you are interested in leading a book 
discussion, look for books announced on LINGUIST as "available 
for review." Then contact Sheila Collberg at collberg at linguistlist.org. 

===========================Directory==============================  

1)
Date: 18-Mar-2005
From: Umarani Pappuswamy < umarani+ at pitt.edu >
Subject: Use and Effect of Declarative Information in User Instructions 

	
-------------------------Message 1 ---------------------------------- 
Date: Sat, 19 Mar 2005 15:20:40
From: Umarani Pappuswamy < umarani+ at pitt.edu >
Subject: Use and Effect of Declarative Information in User Instructions 
 

AUTHOR: Karreman, Joyce
TITLE: Use and Effect of Declarative Information in User Instructions
SERIES: Utrecht Studies in Language and Communication 18
PUBLISHER: Rodopi
YEAR: 2004
Announced at http://linguistlist.org/issues/15/15-1633.html


Umarani Pappuswamy, Research Associate, Learning Research and Development 
Center, University of Pittsburgh, USA. 

BOOK'S PURPOSE AND CONTENTS 
 
The book 'Use and effect of declarative information in user instructions' 
is a PhD thesis by Joyce Karreman, promoted by Michaël Steehouder and 
Peter Jan Schellens at the University of Twente, Netherlands. This study 
investigates the effects of different information types in instructions 
for use. 

The work of Ummelen (1997) serves as the starting point for many 
researchers studying reading and learning from texts and Karreman is not 
an exception. According to Ummelen (1997), information in instructive 
texts can be either procedural or declarative. Information types that are 
directly related to the functioning of the product, for example, "actions" 
that must be performed to get a product working, conditions for actions 
and results from these actions are "procedural" whereas all other 
(explanatory) information about the internal workings of the device, 
trouble-shooting tips etc. are "declarative". Previous studies have shown 
that adding declarative information in an instructive text helps to 
improve the task performance (Kieras and Bovair 1984; Smith and Goodman 
1984; Payne 1988) in a number of ways. The types of declarative 
information used in these studies were: information about how a system 
works; metaphorical and information about the interface (p.43). Karreman 
extends research in this area with the goal 'to investigate whether 
particular types of declarative information lead to specific positive 
effects during the process of learning to work with a device' (p.3). 

The study is presented in the form of five chapters. 

Chapter 1 presents an overview of the literature on using instructive 
texts. It describes two general theories on reading text: Van Dijk and 
Kintsch (1983) and ACT* theory of Anderson (1983). Van Dijk and Kintsch 
(1983) distinguish three models of mental representation, viz., surface 
representation, propositional representation and situational model. After 
a careful analysis of these three representations of text comprehension, 
the author concludes that this theory cannot be directly applied for 
describing the process of operating devices by using instructive texts. 
However, the author finds the notion 'situational model' to be more useful 
to the topic of her research. Karreman discusses ACT* theory on the 
description of procedural learning in detail and finds that it is also not 
an appropriate theory for explaining effects of instructive texts. In the 
next three sections, the author provides various other theoretical 
constructs that were essentially outgrowths of the two reading theories 
mentioned above. Kieras and Bovair (1984) assume that instructive texts 
are read as in Van Dijk and Kinstsch (1983). They also provide a 
characterization of how suitable declarative representations are formed 
based on the instructions read (which was not clear in Anderson's ACT* 
theory). Karreman discusses their model of procedure acquisition and some 
experimental results that threw light on the effect of declarative 
information on task performance in depth. Payne (1988, 1991 and 1993) 
criticized Kieras and Bovair (1984)'s model and Karreman finds some of his 
comments to be a valuable contribution to the topic of effects of 
declarative information in texts (p.22). Lastly, the author discusses 
Dixon (1987, 1993)'s model of hierarchical planning framework with the 
central assumption "mental plans have a hierarchical structure, in which 
the task is described in general terms at the top of the hierarchy and 
specified more precisely at lower levels" (Dixon 1993, p.377). 

The author also presents a comparison of all the five theoretical 
constructs in the latter part of the chapter with a reference to the 
analytical data from experiments concerning declarative information in 
instructive texts. The experiments are about the effects of adding 
declarative information to procedural instructions; the effects of 
declarative information compared with the effects of examples or results 
of practicing. The chapter concludes with a table that neatly summarizes 
the results of these experiments.

Chapter 2 describes an experiment which differs from Ummelen (1997) in 
three aspects. Firstly, it tests if Ummelen's results could be extended to 
another new system. Secondly, unlike Ummelen whose experiments focused on 
the effects of declarative information in general, Karreman investigates 
two particular types of declarative information: system and utilization. 
System information is about the internal working of the product whereas 
utilization information is about the reason, circumstances under which a 
particular function of the system can be used. Thirdly, Karreman's 
experiment uses 'reading times' in addition to Ummelen's 'using times' 
(p.47). The methods, designs and results of the investigation are 
discussed in detail. Justifications for selecting Ummelen's 'Click and 
Read' method over the others are also provided. The results of this 
experiment showed that system and utilization information are read during 
the orientation phase but unfortunately they did not have any effect on 
the task performance and knowledge. Hence, Karreman devised a follow-up 
experiment that uses a combination of 'Click and Read' and 'Think Aloud' 
methods. The results of this experiment also leave the question of absence 
of any effect of declarative information on task performance.

Chapter 3 discusses another experiment and a follow-up study both of which 
are designed differently from the studies described in chapter 2. The goal 
here is to investigate the possible effects of cognitive information on 
cognitive load, confidence and the appreciation of the instructions and 
the device. The results demonstrate that declarative information in user 
instructions does not affect the appreciation of the instructions or the 
device but has a negative effect on the cognitive load and the confidence 
of the users. This experiment also showed that declarative information 
does not have clear effects on task performance. Liberal use of figures 
and tables makes the ideas clear and interesting. 

Chapter 4 investigates the effect of system information on the "transfer 
of knowledge". The author presents a brief overview of previous research 
on the theory of 'structure mapping' that discusses the role of analogy 
and similarity in human cognition along with a review of some empirical 
studies on this concept. Karreman prefers to use the terms 'structural 
similarities and superficial similarities' (p.135) for her study to 
investigate if system information about one device will result in a higher 
degree of transfer when learning to operate structurally similar devices. 
She conducts three experiments which are extensions to the series of 
experiments conducted by O'Reilly and Dixon (1999, 2001 and 2002). Each 
experiment is based on previous research on problem solving and 
transferring knowledge to new tasks. The main finding is that order of 
devices affected the degree of transfer (p. 167). Karreman reports the 
observations of these experiments at length. 

Chapter 5, the final chapter, summarizes the results of her experiments, 
conclusions and discussions. It also provides answers for the three 
research questions raised in the previous chapters. The questions are:
1. Are system and utilization information read?
2. Do system and utilization information affect knowledge, cognitive load, 
confidence and appreciation?
3. Do system and utilization information affect task performance?

Her main conclusions are that declarative information has a weak effect on 
task performance and had some unexpected effects on user's confidence in 
performing a task. She outlines some directions for future research in 
this topic.

Lastly, the author presents a neat compilation of the references running 
into seven pages. 

OVERALL COMMENTS 

This book is a good starting point for researchers interested in 
information in instructional texts. The author provides a good 
presentation of various methods and techniques used in investigating the 
use and effect of declarative information. The language is clear and 
effective and the text material is supported by tables and graphics 
wherever needed. The chapters are arranged in such a manner that the 
findings of one chapter serve as a stepping stone for the next one. Most 
of the experiments reported in the chapters 2, 3 and 4 have been also 
previously published by the author and her co-authors Steehouder and Dixon 
(see the book for references). As pointed out by the author herself in 
many places, her findings provoke several interesting hypotheses that need 
further investigation from a cognitive point of view.

Though it was not within the scope of the study, the author could have 
written a short section on potential guidelines as how to write useful and 
effective instructive texts in general and what kind of system and 
utilization information should be included in an instructive text. Such 
guidelines could be of great help to user-manual writers. 

The book has a minor flaw in typesetting which changes the meaning of the 
sentence the author intended to convey: on page 132, in line 22, the 
word "cannot" should have been " can not" ('These users cannot only 
execute tasks that are explicitly formulated ... ; they can also ..."). 
Besides this, there are other minor presentation issues, for example, 
repetition of hypotheses (shared between two experiments) on p. 152 could 
have been avoided by simply providing reference to the list given on p. 
142. The section 4.2 , titled 'First Experiment', goes on to describe 
O'Reilly and Dixon's experiments before she sets hers own hypotheses for 
the desired experiment (p. 138-142). This spoilt the readability of the 
text at least for this reviewer who feels that this could be avoided by 
moving this portion of the text to the subsection on literature review on 
p. 137 and Karreman could have provided justifications for her motive to 
build upon O'Reilly and Dixon's hypotheses. The book also lacks a subject-
index for reference purposes to concepts, techniques and domain-specific 
vocabulary mentioned in the book. 

Overall, this book should appeal to those interested in research on 
learning from texts. It could also serve as a course material for students 
designing experiments on information texts.

REFERENCES

Dixon, P. (1987). The processing of organizational and component step 
information in written directions. Journal of Memory and Language, 26, 24-
35.

Dixon, P., et al. (1993). Effects of sentence form on the construction of 
mental plans from procedural discourse. Canadian Journal of Experimental 
Psychology, 47, 375-400.

Kieras, D. E. and Bovair, S. (1984). The role of a mental model in 
learning to operate a device. Cognitive Science, 8, 255-273.

O'Reilly, T., and Dixon, P. (1999). Procedures are only skin deep: The 
effects of surface content and surface appearance on the transfer of prior 
knowledge in complex device operation. Proceedings of the Cognitive 
Science Society Conference, 21, 486-489, Hillsdale, NJ:Erlbaum.

Payne, S. J. (1988). Metaphorical instruction and the early learning of an 
abbreviated-command computer system. Acta Psychologica, 69, 207-230.

Payne, S. J. (1993). Memory for mental models of spatial descriptions: An 
episodic-construction-trace hypothesis. Memory and Cognition, 21, 591-603.

Smith, E. E. and Goodman, L. (1984). Understanding written instructions: 
The role of an explanatory schema. Cognition and Instruction, 1, 359-396.

Ummelen, N. ( 1997). Procedural und declarative information in software 
manuals. Effects on information use, task performance and knowledge. 
Amsterdam/Atlanta: Rodopi.

Van Dijk , T. and Kintsch, W. (1983). Strategies of discourse 
comprehension. New York: Academic press.

(For other references mentioned in this review, please see Karreman's 
book). 

ABOUT THE REVIEWER 

Dr. Umarani Pappuswamy is a research associate at LRDC, University of 
Pittsburgh, USA. She has a Ph.D. in Linguistics with specialization in 
computational linguistics. Her main areas of research interests are: 
computational semantics, intelligent tutoring systems, machine learning, 
discourse analysis, typology, and corpus linguistics.





-----------------------------------------------------------
LINGUIST List: Vol-16-840