36.448, FYI: Seminar: Swansea University's Language Research Centre seminar: "Developments in LLAMA test validation" 5th Feb @ 12noon GMT

[The LINGUIST List]

LINGUIST List: Vol-36-448. Tue Feb 04 2025. ISSN: 1069 - 4875.

Subject: 36.448, FYI: Seminar: Swansea University's Language Research Centre seminar: "Developments in LLAMA test validation" 5th Feb @ 12noon GMT

Moderator: Steven Moran (linguist at linguistlist.org)
Managing Editor: Justin Fuller
Team: Helen Aristar-Dry, Steven Franks, Joel Jenkins, Daniel Swanson, Erin Steitz
Jobs: jobs at linguistlist.org | Conferences: callconf at linguistlist.org | Pubs: pubs at linguistlist.org

Homepage: http://linguistlist.org

Editor for this issue: Joel Jenkins <joel at linguistlist.org>

================================================================


Date: 04-Feb-2025
From: Vivienne Rogers [v.e.rogers at swansea.ac.uk]
Subject: Seminar: Swansea University's Language Research Centre seminar: "Developments in LLAMA test validation" 5th Feb @ 12noon GMT


We are delighted to announce that we are kicking off our new LRC
seminar series on Wednesday 5th Feb with a talk by Lars Bokander
(Jönköping University, Sweden) working on a project with Swansea’s own
Vivienne Rogers, Brian Rogers (Digital Humanities) and Paul Meara
together with placement students Tesni Galvin and Rhea Ray. The talk
discusses the ongoing validation of the LLAMA tests originally
developed by Paul Meara in 2005.
When: Wed 5th Feb
Time: 12noon-12.50pm GMT
Zoom link:
https://swanseauniversity.zoom.us/j/91087193768?pwd=8HG2TEmAwy8LbMD5xxhvOdORulYgJQ.1
Abstract:
Language learning aptitude has received increasing attention in recent
years. However, in the absence of freely available test batteries,
researchers have turned to the LLAMA tests (Meara, 2005) despite
issues with their lack of reliability and validation (Bokander &
Bylund, 2020). The LLAMA tests have since undergone a number of
significant changes in terms of items and test layout (Rogers, Meara &
Rogers, 2023) but still comprise 4 tests: LLAMA D (sound recognition),
LLAMA B (vocabulary learning), LLAMA E (sound-symbol correspondence)
and LLAMA F (grammatical inferencing). With the new LLAMA v.3 tests in
use by the research community, it is therefore timely to revisit the
reliability and validity concerns highlighted by Bokander & Bylund
(2020) with the new tests. This paper investigates some of these
concerns in three distinct ways.
The first approach used LLAMA data taken from the lognostics website.
The following inclusion criteria resulted in 1370 participants:
All incomplete tests removed,
Duplicate IDs – first complete test used,
Blanks and “anon” IDs removed,
IDs matched across four subtests.
We adopted the methods used in Bokander & Bylund (2020) and carried
out internal consistency and item-analyses on the data. Internal
consistency reliability (alpha, omega) was much improved compared to
the first LLAMA version (Meara, 2005) in all subtests, with
coefficients around .90 and just slightly lower in LLAMA D. Factor
analysis of the test items revealed a two-dimensional structure in
LLAMA D, reflecting the difference between familiar and unfamiliar
sound stimuli. LLAMA B, E, and F displayed a one-dimensional
structure, thus measuring single constructs. Classical item analysis
and Rasch modelling showed that almost all items contributed to good
measurement. The few that did not (four items in LLAMA D) will be
subject to revision.
The second approach involved a test-retest method. 100 participants
(50M, 50F) were recruited through prolific.com. Screeners included: L1
English, fluent in another language, education no higher than
undergraduate degree and five age-groups (n=20) equally divided
between 25-75 (M=49.7, SD=14.42). Participants took the LLAMA tests
(reprogrammed into gorilla.sc) twice with 5 additional items in LLAMA
F, four weeks apart (T1 & T2). Spearman’s correlations with 94
participants showed significant medium correlations (p<.001) between
T1 and T2 (D: rs=.478, z=.520; B: rs=.570, z=.648; E: rs=.577, z=.658;
F: rs=.654, z=.782). Linear regression showed no significant effect of
age on any test.
The third approach looked at test-taking strategies and included
qualitative interviews with 46 participants after taking the LLAMA
tests in person. Interview transcripts were coded using a bottom-up
approach in N.vivo (v.14). Strategies varied by task although story
creation, linking to known things, guessing and rehearsal were common
themes. LLAMA F strategies divided between participants identifying
rules versus meaning. There was no consensus on which task was easiest
or hardest.

Linguistic Field(s): Applied Linguistics
                     Language Acquisition
                     Psycholinguistics




------------------------------------------------------------------------------

********************** LINGUIST List Support ***********************
Please consider donating to the Linguist List to support the student editors:

https://www.paypal.com/donate/?hosted_button_id=87C2AXTVC4PP8

LINGUIST List is supported by the following publishers:

Bloomsbury Publishing http://www.bloomsbury.com/uk/

Cambridge University Press http://www.cambridge.org/linguistics

Cascadilla Press http://www.cascadilla.com/

De Gruyter Mouton https://cloud.newsletter.degruyter.com/mouton

Elsevier Ltd http://www.elsevier.com/linguistics

John Benjamins http://www.benjamins.com/

Language Science Press http://langsci-press.org

Multilingual Matters http://www.multilingual-matters.com/

Netherlands Graduate School of Linguistics / Landelijke (LOT) http://www.lotpublications.nl/

Wiley http://www.wiley.com


----------------------------------------------------------
LINGUIST List: Vol-36-448
----------------------------------------------------------