<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
Now I am pushed to reply.<br>
<br>
I am a frequent blind reviewer. I see an amazing amount of rubbish
which ought never to have been submitted to a journal. I follow my
doctoral students closely and help them with their papers, which is
why I get angry when inundated by bad papers which a supervisor
should have stopped. If the Humanities is in any danger, it is
because of a caring-sharing tendency that refuses to demand clearly
structured and thought out work. A bad paper takes much more time to
review than a good one.<br>
<br>
Blind review may not be the answer to all, but it does keep journals
at a decent standard. As a reviewer, I always send detailed
comments, and have seen many bad papers become good ones. Rather
than complaining about reviewers, it might be worth looking at
appauling supervisors who do not fulful their duty in guiding
students to a good publication.<br>
<br>
It is not for me to name and shame, but I can guess the universities
that are the source of some papers, and thus the supervisors who are
letting rubblish through.<br>
<br>
I once got caught out when the writer took an educated guess at who
I was. I never admitted to being a reviewer, but was clearly
expected to rewrite his thesis. Doing a review job for free is one
thing, doing the job of bad supervisors is another.<br>
<br>
I essentially review for two journals and several major conferences,
I am proud of the quality standard of all, and it comes from blind
review and the excellent review editors with whom I work.<br>
<br>
We have all come up against pig ignorant (an insult as pigs are very
pleasant and intelligent animals, and I would rather share a pint
with a pig than many colleagues), but the quality standard is
maintained by the great majority who give their time freely to help
guide others. Oversized conferences encourage bad reviewing as time
is of the essence, and the quaity variable in the extreme. Do not
tarnish all events with the same briush.<br>
<br>
Do not let the blind lead the blind reveiwers astray, and two is
always better than one.<br>
<br>
Geoffrey<br>
<br>
Having eyes, see ye not? and having ears, hear ye not? and do ye not
remember? Mark 8. 18. King James, of course.<br>
<br>
Le 01/10/2011 12:32, Anil Singh a écrit :
<blockquote
cite="mid:CAEoph7F_LiVRmdafjXW_c8=KgrNUeHsyymaLSNPNc+m9P0L2XQ@mail.gmail.com"
type="cite">
<div>As some academicians/researchers I have respect for have
expressed opinions which I believe to be right and which I have
been writing about on my blog and elsewhere (apart from arguing
for them in conversations), I dare to chip in and say my bit.</div>
<div><br>
</div>
<div>When I had entered this area and was thinking of submitting
my first paper (around 2003-2004) and I found out that reviewing
will be blind, I was delighted. I was a nobody (a graduate
student) from a developing country (India, but not even from one
of the IITs and not with a very good pedigree), I thought double
blind reviewing will be definitely more fair for people like me.</div>
<div><br>
</div>
<div>My experience since then has completely disabused me of that
naive idea. While it may not be possible to exactly identify the
author(s) of the paper, one does get enough information (and
meta-information) that is more than enough to trigger all the
prejudices, biases etc. that blind reviewing is supposed to be
an antidote against. This happens in almost all the cases.
Needless to add that there can be exceptions.</div>
<div><br>
</div>
<div>You can, of course, give numerous counter-examples from cases
where no bias or prejudice is likely anyway or is very unlikely.
But those examples are not the ones that matter here.</div>
<div><br>
</div>
<div>As far as I am concerned, if you can just identify the fact
that the author is from India, that alone removes at least half
of the supposed effectiveness of the idea of double blind
reviewing. And if you work on Indian languages and do certain
kind of work, it's a no-brainer.</div>
<div><br>
</div>
<div>Then there can be things like whether the author is just a
student or an established researcher, whether the project is
funded or non-funded, whether the language is that of a native
speaker or not etc. These are the very things that double blind
reviewing is supposed to guard against, but it simply can't. It
just can't and I am sorry that it can't. Theoretically the idea
still appeals to me, but may be like many other theoretically
good things, it is not practically implementable.</div>
<div><br>
</div>
<div>I especially like Yorick's comment about undignified
gymnastics that one is required to perform to hide one's
identity. It even lowers the academic quality of the paper quite
often because you can't add information that is very relevant.
And I am totally in favour of the reviewer taking responsibility
for his comments. I have a corpus of reviews and some of the
comments simply make one embarrassed that academicians (which
one is too) can behave like that -- and that too in writing.</div>
<div><br>
</div>
<div>One of the things that has always left me wondering (to put
it lightly) is the fact that the conduct of academicians during
the actual meetings, i.e., paper presentations, panel
discussions etc. is so exceedingly civilized (for want of a
better word) that I sometimes feel out of place there (coming
from a chaotic third world country and being disordered
personally). But a lot of the same academicians, when they
blind-review a paper, behave like bullies, vigilantes or just
plain hooligans. Fortunately, their number is still a minority.</div>
<div><br>
</div>
<div>Of course, like everyone else, I have received wonderful
(even if very critical) reviews. But that can happen even with
non-blind reviewing. Just read literary supplements of papers
that take literature seriously.</div>
<div><br>
</div>
<div>To conclude, I would just say that if for nothing else, at
least to maintain the basic dignity of the academic community
and of individual academicians, it would be best if we switch to
a reviewing process that does not pretend to be blind and where
reviewers take responsibility for their comments.</div>
<div><br>
</div>
<div>I am agnostic about whether extended abstracts should be
reviewed or full papers. Both seem to have their merits. For a
conference like LREC, extended abstracts do seem better to me,
though I won't fight for that (borrowing a phrase from review
forms).</div>
<div><br>
</div>
<div>I hope am not doing anything wrong by adding this link here:</div>
<div><br>
</div>
<div><a moz-do-not-send="true" href="http://reviewscontd.org/">http://reviewscontd.org/</a></div>
<div><br>
</div>
<div><br>
</div>
<br>
<div class="gmail_quote"> On Sat, Oct 1, 2011 at 6:28 AM, Yassine
Benajiba <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:benajibayassine@gmail.com">benajibayassine@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex; position: static; z-index: auto;"> Hi
everyone,
<div><br>
</div>
<div>I say let's judge the conference by the results. LREC is
an awesome conference constantly improving year after year.
Even though it would be great if somebody from the
organizing committee could join this conversation and tell
us a bit more about the reasons.</div>
<div><br>
</div>
<div>Best,</div>
<div><br>
</div>
<font color="#888888">
<div>--Yassine.</div>
</font>
<div><br>
<div class="gmail_quote">
<div>
<div class="h5">On Fri, Sep 30, 2011 at 11:31 PM, Eric
Ringger <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:ringger@cs.byu.edu" target="_blank">ringger@cs.byu.edu</a>></span>
wrote:<br>
</div>
</div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex; position: static; z-index: auto;">
<div>
<div class="h5">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:
11pt; color: rgb(31, 73, 125);">Thanks to
all for the open discussion.</span></p>
<p class="MsoNormal"><span style="font-size:
11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size:
11pt; color: rgb(31, 73, 125);">Graeme’s
reason (1)(a) – the impact on merit review –
is for me the strongest reason to encourage
LREC to move away from reviewing extended
abstracts and toward reviewing full papers.</span></p>
<p class="MsoNormal"><span style="font-size:
11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size:
11pt; color: rgb(31, 73, 125);">Best,</span></p>
<p class="MsoNormal"><span style="font-size:
11pt; color: rgb(31, 73, 125);">--Eric </span></p>
<p class="MsoNormal"><a moz-do-not-send="true"
name="132bd095fa86333e_132bcb56d38f5ba9__MailEndCompose"><span
style="font-size: 11pt; color: rgb(31, 73,
125);"> </span></a></p>
<div>
<div style="border-width: 1pt medium medium;
border-style: solid none none; border-color:
rgb(181, 196, 223) -moz-use-text-color
-moz-use-text-color; padding: 3pt 0in 0in;">
<p class="MsoNormal"><b><span
style="font-size: 10pt;">From:</span></b><span
style="font-size: 10pt;"> Graeme Hirst
[mailto:<a moz-do-not-send="true"
href="mailto:gh@cs.toronto.edu"
target="_blank">gh@cs.toronto.edu</a>]
<br>
<b>Sent:</b> Friday, September 30, 2011
11:02 AM<br>
<b>To:</b> Yorick Wilks<br>
<b>Cc:</b> Eric Ringger; <a
moz-do-not-send="true"
href="mailto:corpora@uib.no"
target="_blank">corpora@uib.no</a><br>
<b>Subject:</b> Re: [Corpora-List] why
LREC2012 NOT blind-reviewed?</span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">Yorick,</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">(1) Whether a
conference is reviewed by abstract or by
full paper makes an enormous difference:</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal"> (a) to merit, as
perceived by tenure committees, granting
agencies, and others, who count only
fully peer-reviewed papers.</p>
</div>
<div>
<p class="MsoNormal"> (b) to funding for
travel. Right now, one of my colleagues
has the problem that he cannot be funded
to travel to give a paper at LREC
because it isn't a fully-reviewed
conference, so he doesn't even bother
submitting.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">You might say that
these situations aren't desirable, but
they are nonetheless reality right now.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">(2) I wonder how you
are so sure that you almost invariably
identify the author of an anonymous
paper correctly. If the paper is not
ultimately accepted at the conference,
which is 60 to 80% of them at ACL and
COLING conferences, you will never find
out who the authors actually are. I've
certainly guessed wrongly in the past.
And in my own papers, I often throw in
"hidden signals" to deceive the
reviewers.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">(3) I think Eric
Ringger is 100% right about LREC. As
you say, LREC's reputation and quality
have grown, and for that reason it has
to start acting like a grown-up
conference.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Regards,</p>
</div>
<div>
<p class="MsoNormal">Graeme</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:
12pt;">--<br>
:::: Graeme Hirst<br>
:::: University of Toronto * Department
of Computer Science</p>
</div>
<p class="MsoNormal"> </p>
<div>
<div>
<p class="MsoNormal"> On 2011-09-30, at
11:27, Yorick Wilks wrote:</p>
</div>
<p class="MsoNormal"><br>
<br>
</p>
<div>
<p class="MsoNormal">I disagree
strongly. I dont see why all
conferences should be exactly like all
others. Extended abstracts are less of
a burden on busy academics --both as
writers and reviewers----and there is
no evidence they lower the final
quality; COLING used to do this and I
am sorry it changed. The whole
blind-review business is a huge
nonsense: I rarely meet a paper to
review where i cannot identify the
authors from a careful trawl of hidden
signals and the references. Trying to
make a paper genuinely anonymous is
almost impossible if one has a body of
past work and publication to link it
to---the mental gymnastics required
are undignified and best avoided.
LRECs reputation has grown steadily
and it will be the quality of its
papers that sustain it--there is no
evidence at all anonymity would
improve matters in the least. if it
ain't broke........</p>
<div>
<p class="MsoNormal">Yorick Wilks</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal"> </p>
<div>
<div>
<p class="MsoNormal">On 30 Sep
2011, at 16:02, Eric Ringger
wrote:</p>
</div>
<p class="MsoNormal"><br>
<br>
</p>
<div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);">Greetings.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);">LREC
has been operated in this
manner since its inception.
Personally and for the sake
of LREC’s reputation, I
would like to see the
reviewing process for LREC
upgraded to double-blind
review.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);">I
believe that LREC fills a
couple of important niches:
its focus on language
resources and
evaluation/validation is
important and not well
served elsewhere, and it
does a good job of bringing
a large, diverse group
together. (I should add
that it does a good job of
selecting attractive venues
as well!) If implemented
well, I believe that
double-blind review would
not detract from the primary
objectives of the conference
but would refine the quality
of the program and improve
the reputation of the
venue. I have said as much
in private feedback after
past LRECs.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);">I
also think it is time for
LREC to move up from
reviewing extended abstracts
to reviewing full papers.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);">Regards,</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);">--Eric</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 11pt;
color: rgb(31, 73, 125);"> </span></p>
</div>
<div>
<p class="MsoNormal"> <span
style="font-size: 11pt;
color: rgb(31, 73, 125);"> </span></p>
</div>
<div>
<div style="border-width: 1pt
medium medium; border-style:
solid none none; border-color:
rgb(181, 196, 223)
-moz-use-text-color
-moz-use-text-color; padding:
3pt 0in 0in;">
<div>
<p class="MsoNormal"> <b><span
style="font-size:
10pt;">From:</span></b><span><span
style="font-size:
10pt;"> </span></span><span
style="font-size: 10pt;"><a
moz-do-not-send="true"
href="mailto:corpora-bounces@uib.no" target="_blank">corpora-bounces@uib.no</a><span> </span><a
moz-do-not-send="true"
href="mailto:[mailto:corpora-bounces@uib.no]" target="_blank">[mailto:corpora-bounces@uib.no]</a><span> </span><b>On
Behalf Of<span> </span></b>Isabella
Chiari<br>
<b>Sent:</b><span> </span>Friday,
September 30, 2011 8:45
AM<br>
<b>To:</b><span> </span><a
moz-do-not-send="true"
href="mailto:corpora@uib.no" target="_blank">corpora@uib.no</a><br>
<b>Subject:</b><span> </span>[Corpora-List]
why LREC2012 NOT
blind-reviewed?</span></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size: 10.5pt;
color: black;">Dear
Corpora members,</span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <span
style="font-size: 10.5pt;
color: black;">I just
noticed that the LREC2012
call specifies that
submissions are NOT
anonymous and there will
not be blind-reviewing.</span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <span
style="font-size: 10.5pt;
color: black;"> </span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size: 10.5pt;
color: black;">Does anyone
know why? Which is the
policy under this
decision?</span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size: 10.5pt;
color: black;">Best
regards,</span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size: 10.5pt;
color: black;">Isabella
Chiari</span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size: 10.5pt;
color: black;"> </span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
<div class="im">
_______________________________________________<br>
UNSUBSCRIBE from this page: <a moz-do-not-send="true"
href="http://mailman.uib.no/options/corpora"
target="_blank">http://mailman.uib.no/options/corpora</a><br>
Corpora mailing list<br>
<a moz-do-not-send="true" href="mailto:Corpora@uib.no"
target="_blank">Corpora@uib.no</a><br>
<a moz-do-not-send="true"
href="http://mailman.uib.no/listinfo/corpora"
target="_blank">http://mailman.uib.no/listinfo/corpora</a><br>
<br>
</div>
</blockquote>
</div>
<br>
</div>
<br>
_______________________________________________<br>
UNSUBSCRIBE from this page: <a moz-do-not-send="true"
href="http://mailman.uib.no/options/corpora" target="_blank">http://mailman.uib.no/options/corpora</a><br>
Corpora mailing list<br>
<a moz-do-not-send="true" href="mailto:Corpora@uib.no">Corpora@uib.no</a><br>
<a moz-do-not-send="true"
href="http://mailman.uib.no/listinfo/corpora"
target="_blank">http://mailman.uib.no/listinfo/corpora</a><br>
<br>
</blockquote>
</div>
<br>
<pre wrap=""><fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
UNSUBSCRIBE from this page: <a class="moz-txt-link-freetext" href="http://mailman.uib.no/options/corpora">http://mailman.uib.no/options/corpora</a>
Corpora mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Corpora@uib.no">Corpora@uib.no</a>
<a class="moz-txt-link-freetext" href="http://mailman.uib.no/listinfo/corpora">http://mailman.uib.no/listinfo/corpora</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<style type="text/css">
<!--
.Style1 {font-family: Arial, Helvetica, sans-serif}
.Style5 {font-size: 12px}
.Style6 {font-size: 14px}
-->
</style>
<p><span class="Style1"><span class="Style6"> <strong></strong></span></span><br>
</p>
</div>
</body>
</html>