I have already mentioned my opinion about the other things, so I would (predictably) comment only about one point (which has two parts: again):<div><br></div><div>- I don't know how you assume the alleged attitude "I'm an Indian, so I will be rejected". I never said that, did I? (And I am almost the only Indian who had been commenting here, till Ramesh's comment). I do have a number of publications (though not very top-notch), even this year, though I haven't been focusing on writing papers for some time. That there can be prejudice is something that is so obvious that I don't think I need to explain it further. The very fact that there is double blind reviewing assumes the possibility of various kinds of prejudices. And, by the way (and just for the record), this sort of prejudice does not always mean racism or some such thing. It could also mean that, in general, for whatever reasons, the reviewer has come to think that the papers from some specific country are not up to the mark. Some (or a lot of) guilt for that could lie for researchers from that country too.</div>
<div><br></div><div>- I also don't understand why you assume that "the assumption that something happened because I was a foreigner" is always wrong. Things do happen because some people are foreigners. Don't they now? Be reasobale.<br>
<br></div><div>You are also wrong in assuming that I am the typical Indian. I am not. So it may well happen that many Indians that you will meet will heartily agree with you. No need to worry about that.</div><div><br></div>
<div>We are all humans. Someone said on this very thread and I agreed.</div><div><br></div><div>And, another by the way, even travelling to Thailand is not so easy.</div><div><br><div class="gmail_quote">On Thu, Oct 6, 2011 at 11:33 PM, Thomas Schoenemann <span dir="ltr"><<a href="mailto:thomas_schoenemann@yahoo.de" target="_blank">thomas_schoenemann@yahoo.de</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="color:rgb(0, 0, 0);background-color:rgb(255, 255, 255);font-family:'times new roman', 'new york', times, serif;font-size:12pt">
<div><span>Hi everyone,</span></div><div><br><span></span></div><div><span>I'm not exactly an established researcher in this field, but I sensed interest in _everyone_'s view, so I will contribute a few points. First, my reviewing activity has been exclusively limited to computer vision so far. That implies that I don't know anything about the system in NLP, so please don't rate me for making wrong assumptions.</span></div>
<div><br><span></span></div><div><span>Now, here are a few points:</span></div><div><br><span></span></div><div><span>- comparing journals and conferences seems inappropriate: many journals primarily get extended versions of conference publications. Here, hiding the authors is not sensible. Also, the information of where the paper was accepted can help greatly to overcome any prejudices the reviewer might have
about the authors.</span></div><div><span>- I very much liked the idea of having a two-round system with feedback from both the authors and the reviewers.</span></div><div><span>- I wouldn't worry too much about things like searching for similar passages (I support fighting plagiarisms, but in case the paper is original all knowledge accidentally gained in the process should be ignored for the review). As we all agreed, reviewers have little time to spend anyway. But on the other hand, I know that some reviewers are intentionally abusing their position: someone once told me he was doing that (I won't tell who, but I no longer talk to him). And I think we should at least try to improve the system against such persons.</span></div>
<div><span>- If we want to go for less anonymity, I think we should start with revealing the area chair of each paper. After all, these established names are least likely to get a serious disadvantage.</span></div><div><span>-
Maybe it would be good if one could complain about a reviewer, with the effect that (maybe after several complaints) this pair of authors and reviewers will not be matched for some time. This could still be double-blind, only some roof organisation (ACL?) would have the list. Of course, for big names it might be rather easy to circumvent the system. One would have to point out serious errors in the reviews to complain.<br>
</span></div><div><span>- The proposal that reviewers are identified upon complaints is something that should at least be considered - in my opinion.</span></div><div><span>- I believe that the area chairs are in a very good position to improve the reviews: by assigning reviewers that are familiar with the problem.<br>
</span></div><div><span>- Like someone else said already, I frequently guess wrongly about who has written the papers I review. </span></div><div><span>- Teaching students to review seems an essential part of a
supervisor's work. But at least in the beginning, the students should not work alone, and be warned against typical errors (narrow scope, etc.).<br></span></div><div><span>- I really don't like this "I'm an Indian, so I will be rejected"-attitude. I have lived abroad several times (and am living abroad now), and I have always tried to avoid the assumption that something happened because I was a foreigner. Now, here's my proposal: I will attend IJCNLP this year (which is not that far from India), and since I don't have many contacts yet, I explicitly welcome all Indians to come and chat with me! Of course the others are welcome, too.</span></div>
<div><span><br></span></div><div><br><span></span></div><div><span>In any case, change always comes slowly, so we should take our time to discuss this - preferably in a larger scope than here. There are quite a few things that NLP is doing better than computer vision. E.g. that references don't count for the page
limitations.</span></div><div><br></div><div>Best regards,</div><div> Thomas (currently University of Pisa, Italy)<br></div><div><br></div><div style="font-family:times new roman, new york, times, serif;font-size:12pt">
<div style="font-family:times new roman, new york, times, serif;font-size:12pt"><font face="Arial" size="2"><hr size="1"><b><span style="font-weight:bold">Von:</span></b> Michal Ptaszynski <<a href="mailto:ptaszynski@media.eng.hokudai.ac.jp" target="_blank">ptaszynski@media.eng.hokudai.ac.jp</a>><br>
<b><span style="font-weight:bold">An:</span></b> "<a href="mailto:corpora@uib.no" target="_blank">corpora@uib.no</a>" <<a href="mailto:corpora@uib.no" target="_blank">corpora@uib.no</a>>; "<a href="mailto:corpora-request@uib.no" target="_blank">corpora-request@uib.no</a>" <<a href="mailto:corpora-request@uib.no" target="_blank">corpora-request@uib.no</a>><br>
<b><span style="font-weight:bold">Cc:</span></b> <a href="mailto:ptaszynski@hgu.jp" target="_blank">ptaszynski@hgu.jp</a><br><b><span style="font-weight:bold">Gesendet:</span></b> 17:30 Donnerstag, 6.Oktober 2011 <br><b><span style="font-weight:bold">Betreff:</span></b> [Corpora-List] why LREC2012 NOT blind-reviewed?<br>
</font><div><div></div><div><br>Ladies and Gents,<br><br>This fantastic reviewing-ralated
topic has been probably the most active<br>one for some time. It looks like most of Corpora members put their 2 cents<br>into the bag. The bag seems now quite heavy and shows somewhat big<br>potential to turn it into something creative.<br>
<br>My question/proposition is,<br><br>How about working out a better reviewing system than the present double<br>blind reviewing?<br><br>It could actually change something for the better, and since most of us<br>took an active part in the discussion we probably have some ideas.<br>
<br>To give a good start let me say my bunch of ideas.<br><br>- Field assessment before reviewing. Something that has been implemented<br>in easychair and several other systems to some extent. A reviewer selects<br>her fields of expertise to give the conference organizers a hint on what<br>
papers he/she could review with the highest confidence.<br>- Make blinding a choice. Some researchers want to be anonymous, and some<br>want to use their
previous research, but don't want to describe their<br>previous works too extensively (saving paper space).<br>- Discussion between authors and reviewers. For conferences there could be<br>too short time span for double rebuttal, but for journals there could be<br>
allowed a longer open discussion. Its sometimes difficult to convince a<br>reviewer only in one rebuttal session.<br>- Using NLP technology to help reviewers and authors. Automatic<br>summarization, credibility verification of infromation contained in a<br>
paper - for the reviewers, and for the other side (or both), opinion<br>mining/sentiment analysis of all other reviews of the paper (comparing<br>reviews) and of all reviews of one reviewer (is he/she an understanding,<br>
tolerant reviewer or an innocent bunny killer?)<br><br>Let me know about your ideas.<br><br>Michal<br><br>--------------------------<br>Od: Anil Singh <<a href="mailto:anil.phdcl@gmail.com" target="_blank">anil.phdcl@gmail.com</a>><br>
Kopia dla: <a href="mailto:corpora@uib.no" target="_blank">corpora@uib.no</a><br>Do: Gemma Boleda <<a href="mailto:gemma.boleda@upf.edu" target="_blank">gemma.boleda@upf.edu</a>><br>Data: Thu, 6 Oct 2011 02:39:41 +0530<br>
Temat: Re: [Corpora-List] why LREC2012 NOT blind-reviewed?<br><br>Not to forget the fact that getting a visa from countries like India to<br>the US or to Europe is a project in itself. And it costs quite something.<br>And if the visa is rejected after all that time, effort and expenditure...<br>
Rejection of visa is not rare at all: for the US (where a substantial<br>proportion of the highest profile conferences are held) the rate is very<br>high, especially for gradutate students.<br><br><br>On Thu, Oct 6, 2011 at 2:09 AM, Anil Singh <<a href="mailto:anil.phdcl@gmail.com" target="_blank">anil.phdcl@gmail.com</a>> wrote:<br>
I mostly agree on this point, except that this is quite a radical<br>proposal and can be implemented only in the long term. We will need a lot<br>of good quality journal to replace conference proceedings. Will be these<br>
be printed or purely online? Will those purely online considered to have<br>the same credibility as those printed (why not?)?<br><br>The cost of attending a conference/workshop is, of course, a major hurdle<br>for all researchers in developing countries. Nowadays even registration<br>
fee is so high that it is hard to afford it (say, in India) just for one<br>author, even if no one travels abroad to attend the event and present the<br>paper. And no solution for this is in sight (never mind the Emerging<br>
Economy, New Economic Powerhouses etc.). We are clearly being told<br>explicity to not try to publish in conferences, but to try for
journals<br>(students by supervisors and administration, teachers/researchers by<br>administrations and funding agencies). Easy to say, but how many journals<br>are out there for the whole of CL/NLP (as compared to the number<br>
conferences and workshops)?<br><br>Still, shifting to journals from conferences (for publication) is<br>something that has to happen in CL/NLP.<br><br>The academic evaluation forms (in India at least) give a much higher<br>
weightage to journal publications than to conference publications, which<br>is a big disadvantage for those working in CL/NLP under the current<br>situation.<br><br>Of course, as even Church (2005) had hinted, there is practical problem<br>
involved. Conference publications mean registrations and registrations<br>mean $$. Where will the money for the journals come from? Who will sponsor<br>them? If commercial publishers publish them, won't other factors come in<br>
which might affect their
cridibility?<br><br>Just some doubts. But generally I support the idea.<br><br><br>On Wed, Oct 5, 2011 at 9:45 PM, Gemma Boleda <<a href="mailto:gemma.boleda@upf.edu" target="_blank">gemma.boleda@upf.edu</a>> wrote:<br>
Dear list members,<br><br> some of the concerns that have been raised in this discussion, such as<br>reviewer load and "incrementality" in papers, could be addressed if the<br>field moved to journal, rather than conference, publishing, and used<br>
conferences for dissemination of ideas (where only abstracts would be<br>reviewed) and journals for actual publication. This would have the<br>positive side-effect of making the citation indices of computational<br>linguistics as a subfield go up, thus making it more visible in the<br>
"scientific market". For a 1.5-page long elaboration of these ideas, see<br><a href="http://www.lsi.upc.edu/%7Egboleda/pubs/gboleda_publishingCL.pdf" target="_blank">http://www.lsi.upc.edu/~gboleda/pubs/gboleda_publishingCL.pdf</a><br>
<br> Best,<br> Gemma Boleda<br><br><br><br> _______________________________________________<br> UNSUBSCRIBE from this page: <a href="http://mailman.uib.no/options/corpora" target="_blank">http://mailman.uib.no/options/corpora</a><br>
Corpora mailing list<br><a href="mailto:Corpora@uib.no" target="_blank">Corpora@uib.no</a><br><a href="http://mailman.uib.no/listinfo/corpora" target="_blank">http://mailman.uib.no/listinfo/corpora</a><br><br>_______________________________________________<br>
UNSUBSCRIBE from this page: <a href="http://mailman.uib.no/options/corpora" target="_blank">http://mailman.uib.no/options/corpora</a><br>Corpora mailing list<br><a href="mailto:Corpora@uib.no" target="_blank">Corpora@uib.no</a><br>
<a href="http://mailman.uib.no/listinfo/corpora" target="_blank">http://mailman.uib.no/listinfo/corpora</a><br><br><br></div></div></div></div></div></div><br>_______________________________________________<br>
UNSUBSCRIBE from this page: <a href="http://mailman.uib.no/options/corpora" target="_blank">http://mailman.uib.no/options/corpora</a><br>
Corpora mailing list<br>
<a href="mailto:Corpora@uib.no" target="_blank">Corpora@uib.no</a><br>
<a href="http://mailman.uib.no/listinfo/corpora" target="_blank">http://mailman.uib.no/listinfo/corpora</a><br>
<br></blockquote></div><br></div>