testing proficiency

Sat Feb 15 22:33:17 UTC 2014

Taapwee oti!  That's it, isn't it?!  Well stated!

Eekoshi.
Heather Souter

On Sat, Feb 15, 2014 at 12:22 PM, Brendan Fairbanks <brendan at tkdkims.com>wrote:

> In my opinion, no one test is going to tell you everything about a
> person's language ability.  With evaluation, you can only get a feel or a
> taste of their overall language ability.  Right when you decide what you're
> going to test them on, you inadvertently leave something else out, or fail
> to see other features of their language ability.  There is so much about
> language that we are not aware of.   If you interview someone, they may
> shine or not shine depending on what you ask them and what the topics are.
> They may shine when talking about hunting, but when talking about sewing,
> they may not.  If you were to ask me about sewing in English, I would fail
> miserably.  I don't know anything about it, but I'm still a native speaker
> of English.  So if someone is asked about sewing and they use a lot of
> meta-language to speak about that topic, does that person rate high? Low?
> Is meta-language counted as an attribute?  Is it even considered?
>
>
>
> An interview format will allow for some features of their language to come
> out, but is the interviewer aware of ALL the features of their language.
> Paper evaluations miss stuff too.   Do you compare their abilities with a
> set of criteria or to the native speaker models of speaking, or both?  Are
> they the same?  If you test functionality, does functionality rate high on
> a criteria list, or does it match the patterns of a native speaker?  Or
> both?  Has a native speaker model been consulted so as to make such a
> comparison?  Has enough native speaker data been assembled to make such a
> comparison?  Sometimes the interviewees may achieve a high functionality in
> the language, but was it native-like?  Were the appropriate discourse
> markers used?  Is the interview or evaluation set up in which discourse
> markers usage may be demonstrated?  Do storytelling-only formats hide other
> features of their language ability?  Was proper intonation used (if any
> exists in the target language)?  Was native-like word order used?  Is
> native-like word order even part of the criteria? In some languages,
> linguistic structure and form is not a big deal.  In some, it is.  In some
> languages, word order is very strict, in others, it is not.  In the
> languages where word order is not strict, do interviewees use native-like
> word order?  Is the interviewer aware of what native-like word order is
> like to even make a comparison?  So no we don't expect interviewees to
> recite a whole paradigm, but that is not the question.  The question might
> be:  Do interviewees have equivalent conjugational abilities?  Do they have
> an equivalent mastery of the system?  Do they have native-like
> pronunciation?   Are their consonants long enough?  Are their vowels short
> and long enough?  In Ojibwe, learners may use English consonants in the
> production of Ojibwe, e.g. ikwe vs. ikkwe/ihkwe.  Is their production
> quality even measured?  In Ojibwe, is the duration of sibilants measured,
> i.e. short /s/ vs. long /ss/?   For Ojibwe learners, do they use the
> so-called "passive" paradigm structures correctly?  Or does their use
> resemble the English passive paradigm?  Or does it approximate the native
> speaker usage, i.e. where they resemble "active" sentences?  This is a
> common error for Ojibwe language learners.  The "common" infelicitous usage
> by even very advanced learners of Ojibwe goes something like this*:
> Imbawaajige ojibwemong* to mean "I dream in Ojibwe", but they are
> inadvertently saying "I dream of them speaking Ojibwe."  Are such errors
> and usage being tested as well?  Is the interviewer even aware of such
> anomalies?  Etc. Etc.
>
>
>
> So, in my opinion, there is so much about language that we are not aware
> of as interviewers that we may inadvertently create evaluations which are
> able to capture some features of speaking ability, but miss others.   The
> evaluation instrument is only as good as the depth of awareness of its
> creators.   I ran into problems when I was hired to evaluate the Sauk
> language program a few years ago using the ACTFL guidelines.  I scrapped it
> and created my own criteria.  It did the job, but I realized that any
> evaluation instrument I created had big holes in it too.  We probably can't
> test everything and even if we set out to do so, we'll capture some things
> and miss others.  And maybe that's okay.
>
>
>
> Write an evaluation up that measures (as much as possible) the outcomes
> that you are looking for and what you think might be measurable over time.
> It's the best we can do as evaluators in my opinion.  The concept of
> evaluation presupposes that we somehow know the difference between a novice
> and a superior speaker (and all the levels in between).  My worry is that
> we may not always appreciate what makes a superior speaker superior if we
> are not even aware of what it is exactly that makes him/her a superior
> speaker/reader in the first place.
>
>
>
> Brendan
>
>
>
>
>
> *From:* ALGONQUIANA [mailto:ALGONQUIANA at LISTSERV.LINGUISTLIST.ORG] *On
> Behalf Of *Monica Macaulay
> *Sent:* Wednesday, November 13, 2013 8:13 AM
> *To:* ALGONQUIANA at LISTSERV.LINGUISTLIST.ORG
> *Subject:* testing proficiency
>
>
>
> Good morning,
>
>
>
> I was talking to some of the people who work on Menominee language
> revitalization last week, and they were talking about how they need some
> sort of tool for testing the proficiency of their teachers.  Since this is
> way out of my area of expertise, I said I'd ask around.  Do any of you know
> anything about this?
>
>
>
> It would seem to me that any such tool would have to be specialized to
> work for Algonquian languages (and more specifically in this case tailored
> for Menominee).  I mean, you would have to know whether or not to test
> things like conjunct order, right?
>
>
>
> And one further point:  such an evaluation would ideally (I would think)
> be tied to a curriculum - another thing they're working on.  But is it
> possible to do any kind of testing without that?
>
>
>
> Thanks for any leads or hints you can give me.
>
>
>
> - Monica
>
>
>
> Monica Macaulay
>
> Department of Linguistics
> University of Wisconsin
> 1168 Van Hise Hall
>
> 1220 Linden Drive
> Madison, WI  53706
> phone (608) 262-2292
>
> fax (608) 265-3193
> http://monicamacaulay.com/
>
>
>
>
> ------------------------------
>    <http://www.avast.com/>
>
> This email is free from viruses and malware because avast! Antivirus<http://www.avast.com/>protection is active.
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/algonquiana/attachments/20140215/0d30780e/attachment.htm>