LFG 2008 - First call for papers (revised)

Wed Nov 7 17:19:28 UTC 2007

Dear all,

in my view, it is the role of the reviewers to judge the quality of 
submissions.
Regarding the shape of the program, it is up to the program committee to 
take
final decisions. It is the PC that oversees the results of the reviewing 
process,
and should be in a position to judge both the quality of the papers, 
based on
the reviewers' work, and the overall diversity of topics and approaches, 
including
different modes of presentation.

So in my view, both quality and diversity can be ensured by the staged 
process
of peer reviewing and the final decisions by the PC. It should be understood
that the PC should respect the reviewers' judgements. But as far as 
diversity
can be improved *while keeping quality standards*, it should be ok if the PC
makes choices that may slightly diverge from a numeric, ranked scale of
reviewing scores - as long as they can be justified.

As long as free submission is possible, such a staged process also serves as
a sanity check: in case the PC feels forced or tempted to rule out 
high-quality
papers to the advantage of low-ranked papers, it becomes evident that
something goes wrong. In that case, there should be reflections and open
discussions regarding the  focus of the conference.
Such discussions would not emerge if there are forced restrictions of 
submission.

In order to make this a transparent process, it should be a rule that 
the PC
communicates their decisions to the involved reviewers and consults them
regarding the final program selections. This way, the reviewers have a 
view of the
overall selections made and have a chance to protest in case they think the
process is not sane.

My experience from the past is that at some stages at least, there was 
no consultation
with the reviewers when the program was set up, which in some cases led to
irritations. I don't know what the current policy is, but I still think 
this is a good way,
anyhow, to help shape and justify final paper selections, and it could 
naturally involve
discussions about justifiable measures to ensure diversity.

Kind regards,
  Anette

Josef van Genabith wrote:
> I agree with Chris' analysis.
>
> The question is how to best achieve a good balance between quality and 
> diversity.
>
> I'd put emphasis on quality for the main session of the conference: an 
> LFG conference which has two good papers (no matter what authors 
> and/or subject areas) is more interesting than the one with one good 
> paper and perhaps another more mediocre paper (which got in due to 
> some restrictions on number of submissions (author/subject area etc.)).
>
> Diversity can (and should) be supported by the other instruments 
> available to us: including poster sessions, workshops, student 
> sessions and panels.
>
> Best regards,
>
> Josef
>
> chris brew wrote:
>> I'm an interested observer, not (yet) a regular submitter to LFG 
>> conferences, but:
>>
>> - By restricting submissions, you run the risk of missing the best 
>> work, if it so happens that
>>   someone is doing more good work than you expect. If you do not 
>> impose restrictions, you run the      risk that fewer research groups 
>> will have accepted papers, so some people who would benefit from
>>  attending the conference do not do so.
>> - If reviewing standards are high enough to ensure that bad work is 
>> rejected and good work accepted, it doesn't matter whether multiple 
>> submissions are allowed or not, since the strategy of gaming the 
>> system by producing multiple submissions of less polished work will fail
>>
>> - if there is evidence that people are trying to game the system in 
>> this way, and that the reviewing process is failing to handle it 
>> appropriately, two possible solutions exist. Either impose 
>> restrictions or tighten the reviewing process. If there is no such 
>> evidence, why bother with the restrictions anyway?
>>
>> - The role of conferences is different in different subfields. At 
>> some conferences the work presented is typically high-quality 
>> finished research, at others the intent is to provide a venue for 
>> work in progress.
>> Often, a publication in one of the former conferences has as large an 
>> effect on the future of the author as does a publication in a good 
>> journal. If this is happening, fairness dictates that the review 
>> process should be entirely focussed on ensuring that the best work 
>> gets in. Most CL conferences are like this. You can argue about 
>> whether the reviewing process is indeed achieving these goals, but 
>> everybody agrees that the attempt should be made. It would be 
>> unfortunate if someone was denied the crucial publication because of 
>> rules that are not quality-related.
>> This is not so critical if the intent is to provide a venue for work 
>> in progress, and not so much rides on acceptance and rejections. In 
>> that case I think restrictions could make sense. Maybe the LFG 
>> community has to decide what the conference is for?
>>
>>
>>
>> On 03/11/2007, *A. B. Bodomo* < abbodomo at yahoo.com.hk 
>> <mailto:abbodomo at yahoo.com.hk>> wrote:
>>
>>     Dear all,
>>     I support this move to have a restriction on the number of papers
>>     we submit at LFG conferences. Indeed, this move should have
>>     been made long ago. For me the LFG conferences are Linguistics
>>     conferences and most Linguistics conferences, like the annual LSA
>>     conference, already have this wise idea of restricting the number
>>     of submissions. I don't think it would be such a good idea to rely
>>     on people restricting themselves the way we expect it. It is
>>     natural for people to want to maximise their chances of getting
>>     accepted by submitting as many as possible. For me, the
>>     restriction should be the standard one at Linguistics conferences:
>>     at most one single authored paper and one joint paper, but I am
>>     fine with this middle of the road solution by our Executive
>>     Committee since I understand Computational Linguistics
>>     conferences do something different from General Linguistics
>>     conferences.
>>          I would like to see more diversity in the topics, areas,
>>     backgrounds, and languages presented at LFG conferences and this
>>     should, of course, not be achieved at the expense of quality,
>>     since all papers still have to pass through "anonymous" peer 
>> reviews.
>>          Best,
>>     Adams
>>
>>
>>     */Martin Forst < mforst at parc.com <mailto:mforst at parc.com>>/* wrote:
>>
>>         Dear Josef, dear all,
>>         > does anybody know the reason for the new restriction on the
>>         number of
>>         > submissions (see below) in the revised call for papers for
>>         LFG 2008?
>>         The Executive Committee introduced this restriction in order
>>         to secure
>>         diversity in the papers.
>>         The rationale behind this goal is that quality is of course
>>         crucial for
>>         a conference program, but breadth is important, too.
>>
>>         Although we (i.e. the Program Committee - lfg08 at easychair.org
>>         <mailto:lfg08 at easychair.org>) do not
>>         expect the restriction to rule out a lot of potential
>>         submissions, we
>>         are interested in knowing what people think about it, in
>>         particular if
>>         they are opposed to it. Please let us know your arguments and,
>>         even more
>>         importantly, the number of additional abstracts you would have
>>         submitted
>>         if the restriction did not exist. The Executive Committee will
>>         consider
>>         these in the decision of whether to keep the restriction for
>>         LFG 2009.
>>
>>         Best regards,
>>
>>         Martin
>>
>>
>>
>>  

-- 
Anette Frank
Computational Linguistics Department
University of Heidelberg
Im Neuenheimer Feld 325
69120 Heidelberg, Germany

http://www.cl.uni-heidelberg.de/~frank
email: frank at cl.uni-heidelberg.de
phone: +49-(0)6221/54-3247
secr: +49-(0)6221/54-3245
fax: +49-(0)6221/54-3242