[Corpora-List] discussion on reproducibility at ACL 2011 business meeting

[Mark Sammons]

>I believe that people who submit a paper should be encouraged
>to publish the programs, when it is practical to do so.  But for
>many kinds of systems, readers can learn more from a well-written
>article than they could by running the programs or plowing through
>the source code.
>
>The question of whether publishing the source code would be useful
>should be decided on a case-by-case basis by the reviewers of a paper.
>
>John Sowa
>

The overhead of requiring researchers to publish code seems inordinate, for the 
reviewers and even the researchers themselves (where it isn't already a part of the
culture of a given field, and therefore factored into things like the number of papers
academics must publish at good conferences to get tenure, etc.) 

Doesn't publication of software and the resources to literally reproduce published
experimental results speak for itself though? If the code is a good resource
and sufficiently well written that someone else can reuse it without huge effort,
it has a reasonable chance of being used by others and cited (though of course
there's no guarantee).  If authors produce code and whatever resources are needed
to reproduce their experimental results, it adds credibility, provided again that
it is not unduly difficult to run the experiments with what they provide.

Might it not be sufficient to simply encourage this practice without formalizing
the requirements, and rely on positive feedback to do the job?

Mark


_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora