<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=windows-1252" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.6001.19120">
<STYLE></STYLE>
</HEAD>
<BODY bgColor=#ffffff>
<DIV><FONT size=2 face=Arial>Dear colleagues,</FONT></DIV>
<DIV><FONT size=2 face=Arial></FONT> </DIV>
<DIV><FONT size=2 face=Arial>Note sure if this was mentioned before in the
discussion:</FONT></DIV>
<DIV><FONT size=2 face=Arial>Concerning LREC, as many papers deal with language
resources</FONT></DIV>
<DIV><FONT size=2 face=Arial>which are often made available on the web, it makes
sense </FONT><FONT size=2 face=Arial>that </FONT></DIV>
<DIV><FONT size=2 face=Arial>reviewers can also look at these sites. This is not
</FONT><FONT size=2 face=Arial>compatible </FONT></DIV>
<DIV><FONT size=2 face=Arial>with double blind reviewing.</FONT></DIV>
<DIV><FONT size=2 face=Arial></FONT> </DIV>
<DIV><FONT size=2 face=Arial>Concerning bad reviews, I think reviewers usually
have the</FONT></DIV>
<DIV><FONT size=2 face=Arial>best intentions (otherwise they would not
volunteer), but </FONT><FONT size=2 face=Arial>life </FONT></DIV>
<DIV><FONT size=2 face=Arial>can be very harsh with us poor scientists, so
completely</FONT></DIV>
<DIV><FONT size=2 face=Arial>unforeseen problems such as the following may
arise:</FONT></DIV>
<DIV><FONT size=2 face=Arial><BR>1) The reviewer was too optimistic timewise and
finally has to <BR> act in a last minute fashion (90%
probability).</DIV>
<DIV> </DIV>
<DIV>2) The assigned paper simply does not match the reviewer's</DIV>
<DIV> competences or interests (50% probability).</DIV>
<DIV> </DIV>
<DIV>3) The paper is so bad that it does not deserve a detailed review</DIV>
<DIV> (10% probability; some help for newcomers might
nevertheless </DIV>
<DIV> be nice).</DIV>
<DIV><BR>In my experience, bidding is very helpful to obtain better
matches<BR>between papers and reviewers' interests and thus helps to
solve<BR>problem 2). But it is usually based on abstracts which can be
<BR>misleading.</DIV>
<DIV> </DIV>
<DIV>So let me suggest to take this one step further:</DIV>
<DIV> </DIV>
<DIV>Allow reviewers to return papers which they don't like to review</DIV>
<DIV>after looking at them.</DIV>
<DIV> </DIV>
<DIV>This could be realized as follows:</DIV>
<DIV> </DIV>
<DIV>From the pool of all papers reviewers can pick freely the ones that<BR>look
interesting to them (of course based on some pre-classification
<BR>and up to a maximum number). These papers are then assigned </DIV>
<DIV>to the reviewer for a certain time slot (a fraction of the total
reviewing </DIV>
<DIV>period). If a paper has enough assignments, it is locked, i.e. invisible
</DIV>
<DIV>for new reviewers. If a reviewer does not provide a review
within the </DIV>
<DIV>given time slot (alternatively: makes a binding commitment), the </DIV>
<DIV>paper is unlocked, i.e. returned to the pool.</DIV>
<DIV> </DIV>
<DIV>This method would hopefully solve problem 2) above. But it would
also<BR>reduce problem 1): As early reviewers have a better choice, fast
action<BR>is encouraged. Papers which are returned several times would
finally<BR>(towards the end of the reviewing period) have to be dealt with in a
<BR>last minute fashion. But the hope is that many of these papers would
<BR>fall into category 3) anyway (rather than papers on less popular</DIV>
<DIV>topics finding no reviewer).</DIV>
<DIV> </DIV>
<DIV>This procedure would also remove from the programme committee</DIV>
<DIV>(some of) the burden of making appropriate paper assignments, and
</DIV>
<DIV>would encourage or even force reviewers to provide their reviews
at </DIV>
<DIV>an early stage.</DIV>
<DIV> </DIV>
<DIV>But needless to say that I am not sure whether this would work in
</DIV>
<DIV>practice, and whether the benefits would outweigh the extra
effort. </DIV>
<DIV> </DIV>
<DIV>Kind regards,</DIV>
<DIV> </DIV>
<DIV>Reinhard<BR></DIV></FONT>
<DIV><FONT size=2 face=Arial></FONT> </DIV>
<DIV><FONT size=2 face=Arial></FONT> </DIV>
<BLOCKQUOTE
style="BORDER-LEFT: #000000 2px solid; PADDING-LEFT: 5px; PADDING-RIGHT: 0px; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px">
<DIV style="FONT: 10pt arial">----- Original Message ----- </DIV>
<DIV
style="FONT: 10pt arial; BACKGROUND: #e4e4e4; font-color: black"><B>From:</B>
<A title=tpederse@d.umn.edu href="mailto:tpederse@d.umn.edu">Ted Pedersen</A>
</DIV>
<DIV style="FONT: 10pt arial"><B>To:</B> <A title=corpora@uib.no
href="mailto:corpora@uib.no">corpora@uib.no</A> </DIV>
<DIV style="FONT: 10pt arial"><B>Sent:</B> Monday, October 03, 2011 1:39
PM</DIV>
<DIV style="FONT: 10pt arial"><B>Subject:</B> Re: [Corpora-List] why LREC2012
NOT blind-reviewed?</DIV>
<DIV><BR></DIV>
<DIV><BR></DIV>I think one thing we've learned here is that there are some
fairly significant concerns and frustrations with reviewing.
<DIV><BR></DIV>
<DIV>For myself, I review too much, and am probably a little lazy in some
cases as a result. I started to notice that a bit more recently, and so have
started saying "No" with much more frequency, so hopefully I am doing my small
part to correct something in the process. Reviewers who feel they don't have
time to review really need to say "No" more often - the field will survive
without us, and we aren't doing anyone any favors by submitting reviews that
we don't really spend enough time on. LIkewise, I don't think saying "Yes" and
then farming out reviews to graduate students is all that helpful, unless the
senior person is willing to spend some time with the student on reviewing
(until they are sufficiently experienced). I realize more folks doing this
will make it harder to get reviews, but I think a smaller number of better
reviews is in the end more helpful and healthy.</DIV>
<DIV><BR></DIV>
<DIV>
<DIV>I would like to suggest that maybe we ought to ask people who submit
papers to provide the answers to the following two questions (separate from
their papers) in an effort to streamline the process.</DIV>
<DIV><BR></DIV>
<DIV>1) What is the most important idea presented in this submission (in 50
words or less)</DIV>
<DIV><BR></DIV>
<DIV>2) What other paper is most similar to this submission, and how does this
paper improve upon or extend that? (in 100 words or less)</DIV>
<DIV><BR></DIV>
<DIV>If I'm not sufficiently excited by the answers to both 1 and 2, then the
paper can be rejected without further review. A good paper will of course make
1 and 2 fairly clear, but sometimes you have to dig a little, so I'd like to
dispense with the kabuki dance and simply ask authors to answer
these questions at the start, and then we decide as reviewers if we should
read further. </DIV>
<DIV><BR></DIV>
<DIV>The other part of the equation is that most published papers don't end up
having much impact beyond advancing the author's career (Zipf's Law for
Papers? A few papers cited a lot, most not cited much at all). This doesn't
mean they shouldn't be published, and career advancement is a good thing
generally both for the authors and our field, but it can also make for lots
and lots of incremental papers that just aren't all that interesting and
so they aren't cited much, and they tend to have a mind numbing effect on
reviewers and is part of what I think makes reviewing such a chore sometimes.
So, maybe if we make authors self-identify the incremental work versus
the big new ideas then reviewers can have a better idea of what to expect.
Incremental in the area you care most about can be fascinating stuff, so I
don't think incremental is always a bad thing, but there also needs to be a
balance between the incremental and the more novel. As a reviewer I feel like
I spend huge amounts of time on incremental work, and it just gets a little
dull to be honest...</DIV>
<DIV><BR></DIV>
<DIV>Cordially,</DIV>
<DIV>Ted</DIV>
<DIV><BR></DIV>
<DIV>
<DIV class=gmail_quote>On Fri, Sep 30, 2011 at 9:45 AM, Isabella Chiari <SPAN
dir=ltr><<A
href="mailto:isabella.chiari@uniroma1.it">isabella.chiari@uniroma1.it</A>></SPAN>
wrote:<BR>
<BLOCKQUOTE
style="BORDER-LEFT: #ccc 1px solid; MARGIN: 0px 0px 0px 0.8ex; PADDING-LEFT: 1ex"
class=gmail_quote>
<DIV
style="FONT-FAMILY: Calibri, sans-serif; WORD-WRAP: break-word; COLOR: rgb(0,0,0); FONT-SIZE: 14px">
<DIV>Dear Corpora members,</DIV>
<DIV>I just noticed that the LREC2012 call specifies that submissions are
NOT anonymous and there will not be blind-reviewing.</DIV>
<DIV><BR></DIV>
<DIV>Does anyone know why? Which is the policy under this decision?</DIV>
<DIV>Best regards,</DIV>
<DIV>Isabella Chiari</DIV>
<DIV><BR></DIV>
<DIV>
<P><SPAN>Dipartimento di Scienze documentarie, linguistico-filologiche e
geografiche</SPAN></P>
<P><SPAN>Università di Roma “La Sapienza”</SPAN></P>
<P><SPAN>pl.le Aldo Moro, 5, III Piano, Edificio ex Facoltà di Lettere e
Filosofia, 00185 Roma, tel. +30 06 4991 3575</SPAN> </P>
<P><SPAN>E.mail: <A href="mailto:isabella.chiari@uniroma1.it"
target=_blank>isabella.chiari@uniroma1.it</A></SPAN></P>
<P><SPAN>Website: <A href="http://www.alphabit.net"
target=_blank>www.alphabit.net</A></SPAN></P></DIV></DIV><BR>_______________________________________________<BR>UNSUBSCRIBE
from this page: <A href="http://mailman.uib.no/options/corpora"
target=_blank>http://mailman.uib.no/options/corpora</A><BR>Corpora mailing
list<BR><A href="mailto:Corpora@uib.no">Corpora@uib.no</A><BR><A
href="http://mailman.uib.no/listinfo/corpora"
target=_blank>http://mailman.uib.no/listinfo/corpora</A><BR><BR></BLOCKQUOTE></DIV><BR><BR
clear=all>
<DIV><BR></DIV>-- <BR>Ted Pedersen<BR><A
href="http://www.d.umn.edu/~tpederse">http://www.d.umn.edu/~tpederse</A><BR></DIV></DIV>
<P>
<HR>
<P></P>_______________________________________________<BR>UNSUBSCRIBE from
this page: http://mailman.uib.no/options/corpora<BR>Corpora mailing
list<BR>Corpora@uib.no<BR>http://mailman.uib.no/listinfo/corpora<BR></BLOCKQUOTE></BODY></HTML>