<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
..MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>In my opinion, no one test is going to tell you everything about a person’s language ability. With evaluation, you can only get a feel or a taste of their overall language ability. Right when you decide what you’re going to test them on, you inadvertently leave something else out, or fail to see other features of their language ability. There is so much about language that we are not aware of. If you interview someone, they may shine or not shine depending on what you ask them and what the topics are. They may shine when talking about hunting, but when talking about sewing, they may not. If you were to ask me about sewing in English, I would fail miserably. I don’t know anything about it, but I’m still a native speaker of English. So if someone is asked about sewing and they use a lot of meta-language to speak about that topic, does that person rate high? Low? Is meta-language counted as an attribute? Is it even considered?<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>An interview format will allow for some features of their language to come out, but is the interviewer aware of ALL the features of their language. Paper evaluations miss stuff too. Do you compare their abilities with a set of criteria or to the native speaker models of speaking, or both? Are they the same? If you test functionality, does functionality rate high on a criteria list, or does it match the patterns of a native speaker? Or both? Has a native speaker model been consulted so as to make such a comparison? Has enough native speaker data been assembled to make such a comparison? Sometimes the interviewees may achieve a high functionality in the language, but was it native-like? Were the appropriate discourse markers used? Is the interview or evaluation set up in which discourse markers usage may be demonstrated? Do storytelling-only formats hide other features of their language ability? Was proper intonation used (if any exists in the target language)? Was native-like word order used? Is native-like word order even part of the criteria? In some languages, linguistic structure and form is not a big deal. In some, it is. In some languages, word order is very strict, in others, it is not. In the languages where word order is not strict, do interviewees use native-like word order? Is the interviewer aware of what native-like word order is like to even make a comparison? So no we don’t expect interviewees to recite a whole paradigm, but that is not the question. The question might be: Do interviewees have equivalent conjugational abilities? Do they have an equivalent mastery of the system? Do they have native-like pronunciation? Are their consonants long enough? Are their vowels short and long enough? In Ojibwe, learners may use English consonants in the production of Ojibwe, e.g. ikwe vs. ikkwe/ihkwe. Is their production quality even measured? In Ojibwe, is the duration of sibilants measured, i.e. short /s/ vs. long /ss/? For Ojibwe learners, do they use the so-called “passive” paradigm structures correctly? Or does their use resemble the English passive paradigm? Or does it approximate the native speaker usage, i.e. where they resemble “active” sentences? This is a common error for Ojibwe language learners. The “common” infelicitous usage by even very advanced learners of Ojibwe goes something like this<i>: Imbawaajige ojibwemong</i> to mean “I dream in Ojibwe”, but they are inadvertently saying “I dream of them speaking Ojibwe.” Are such errors and usage being tested as well? Is the interviewer even aware of such anomalies? Etc. Etc.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>So, in my opinion, there is so much about language that we are not aware of as interviewers that we may inadvertently create evaluations which are able to capture some features of speaking ability, but miss others. The evaluation instrument is only as good as the depth of awareness of its creators. I ran into problems when I was hired to evaluate the Sauk language program a few years ago using the ACTFL guidelines. I scrapped it and created my own criteria. It did the job, but I realized that any evaluation instrument I created had big holes in it too. We probably can’t test everything and even if we set out to do so, we’ll capture some things and miss others. And maybe that’s okay. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Write an evaluation up that measures (as much as possible) the outcomes that you are looking for and what you think might be measurable over time. It’s the best we can do as evaluators in my opinion. The concept of evaluation presupposes that we somehow know the difference between a novice and a superior speaker (and all the levels in between). My worry is that we may not always appreciate what makes a superior speaker superior if we are not even aware of what it is exactly that makes him/her a superior speaker/reader in the first place. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Brendan<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>From:</span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> ALGONQUIANA [mailto:ALGONQUIANA@LISTSERV.LINGUISTLIST.ORG] <b>On Behalf Of </b>Monica Macaulay<br><b>Sent:</b> Wednesday, November 13, 2013 8:13 AM<br><b>To:</b> ALGONQUIANA@LISTSERV.LINGUISTLIST.ORG<br><b>Subject:</b> testing proficiency<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Good morning,<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I was talking to some of the people who work on Menominee language revitalization last week, and they were talking about how they need some sort of tool for testing the proficiency of their teachers. Since this is way out of my area of expertise, I said I'd ask around. Do any of you know anything about this?<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>It would seem to me that any such tool would have to be specialized to work for Algonquian languages (and more specifically in this case tailored for Menominee). I mean, you would have to know whether or not to test things like conjunct order, right?<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>And one further point: such an evaluation would ideally (I would think) be tied to a curriculum - another thing they're working on. But is it possible to do any kind of testing without that?<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Thanks for any leads or hints you can give me.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>- Monica<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal>Monica Macaulay<o:p></o:p></p><div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'>Department of Linguistics<br>University of Wisconsin<br>1168 Van Hise Hall<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'>1220 Linden Drive<br>Madison, WI 53706<br>phone (608) 262-2292<o:p></o:p></span></p></div><div><p class=MsoNormal style='margin-bottom:13.5pt'><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'>fax (608) 265-3193<br><a href="http://monicamacaulay.com/">http://monicamacaulay.com/</a><br><br><o:p></o:p></span></p></div></div></div><p class=MsoNormal><o:p> </o:p></p></div>
<br /><br />
<hr style='border:none; color:#909090; background-color:#B0B0B0; height: 1px; width: 99%;' />
<table style='border-collapse:collapse;border:none;'>
<tr>
<td style='border:none;padding:0px 15px 0px 8px'>
<a href="http://www.avast.com/">
<img border=0 src="http://static.avast.com/emails/avast-mail-stamp.png" />
</a>
</td>
<td>
<p style='color:#3d4d5a; font-family:"Calibri","Verdana","Arial","Helvetica"; font-size:12pt;'>
This email is free from viruses and malware because <a href="http://www.avast.com/">avast! Antivirus</a> protection is active.
</p>
</td>
</tr>
</table>
<br />
</body></html>
<p>