<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: Arial; font-size: 12pt; color: #000000'>Thanks for the clarification Mike. What I meant was that we may be able to learn the pros and cons they experienced first hand - without going through what could be a painful experience ourselves.<br><br>I know that some of their referees actually run the code of the author against their own data to verify the results claimed! I guess that's rather extreme. Being from the two-thirds world, what I would really benefit from is free availability of data more than code (though as has been suggested, the availability of the core of the algorithm would greatly reduce the time taken to reproduce its results on our data for instance). This would also allow us to check our algorithms on other people's data and make sure that we haven't for instance 'overtrained' our models to some specific situation.<br><span><br><span name="x"></span><br>Ruvan Weerasinghe<br>University of Colombo School of Computing<br>Colombo 00700,<br>Sri Lanka.<br><br>Web: http://www.ucsc.lk<br>Phone: +94112158953; Fax: +94112587239<span name="x"></span><br></span><br><hr id="zwchr"><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;" mce_style="border-left:2px solid #1010ff;margin-left:5px;padding-left:5px;"><b>From: </b>"Mike Maxwell" <maxwell@umiacs.umd.edu><br><b>To: </b>"John F. Sowa" <sowa@bestweb.net><br><b>Cc: </b>corpora@uib.no<br><b>Sent: </b>Tuesday, July 5, 2011 3:19:56 AM<br><b>Subject: </b>Re: [Corpora-List] discussion on reproducibility at ACL 2011 business meeting<br><br>On 7/4/2011 4:58 PM, John F. Sowa wrote:<br>> On 7/4/2011 4:25 PM, Mark Sammons wrote:<br>>> Might it not be sufficient to simply encourage this practice without<br>>> formalizing the requirements, and rely on positive feedback to do the<br>>> job?<br>><br>> Yes. That's what I was suggesting.<br>><br>> There are many different research practices that have led to important<br>> new discoveries and breakthroughs. Imposing too many conditions and<br>> restrictions on what is considered publishable is more likely to<br>> inhibit rather than promote progress.<br><br>Which of course is what this thread was about in the first place. <br>Quoting Ted (but adding emphasis):<br> > ...in favor of the innovation this year on *allowing*<br> > submissions of data and code along with paper submissions.<br><br>I don't think Ruvan was suggesting making code/data submission a <br>requirement in our field, he was just saying we could look at how the <br>biology/ bio-informatics people (who reportedly do make it a <br>requirement) addressed the issues which had been raised.<br><br>And I didn't say anything about requiring it, either.<br><br>So can we stop beating the dead horse of requiring code/data submission, <br>and get back to the point Ted raised: the concerns over how and whether <br>to encourage reproducible research by means of code and data publication.<br>-- <br> Mike Maxwell<br> maxwell@umiacs.umd.edu<br> "My definition of an interesting universe is<br> one that has the capacity to study itself."<br> --Stephen Eastmond<br><br>_______________________________________________<br>UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora<br>Corpora mailing list<br>Corpora@uib.no<br>http://mailman.uib.no/listinfo/corpora<br></blockquote><br></div></body></html>