[Corpora-List] IQMT v 1.0 release

[Jesus Angel Gimenez Linares]

[apologies for multiple cross-postings]

Dear NLP-researchers,

  We're glad to announce the release of the IQMT Framework for Machine 
<http://www.lsi.upc.edu/%7Enlp/IQMT/>
Translation Evaluation <http://www.lsi.upc.edu/%7Enlp/IQMT/>v 1.0 
<http://www.lsi.upc.edu/%7Enlp/IQMT/>, an open source software package, 
released under
the GNU Lesser <http://www.fsf.org/licenses/lgpl.html> General Public 
License (LGPL) <http://www.fsf.org/licenses/lgpl.html> of the Free 
Software Foundation <http://www.fsf.org>.
This tool is a joint effort by the UNED NLP & IR <http://nlp.uned.es/> 
Research Group <http://nlp.uned.es/> and the
TALP Research Center <http://www.lsi.upc.edu/%7Enlp> NLP group 
<http://www.lsi.upc.edu/%7Enlp>.

    The IQMT <http://www.lsi.upc.edu/%7Enlp/IQMT/> package (Giménez et 
al., IWSLT'2005) 
<http://www.lsi.upc.edu/%7Ejgimenez/PUBS/IWSLT05-gah.pdf> is based on 
the QARLA
Framework (Amigó et al., <http://acl.ldc.upenn.edu/P/P05/P05-1035.pdf> 
ACL'2005 <http://acl.ldc.upenn.edu/P/P05/P05-1035.pdf>). Rather than 
defining a new supermetric
'XXX' our tool follows a 'divide and conquer' strategy. You can define a 
set of
metrics and then combine them into a single measure of MT quality, in a 
robust
and elegant manner, avoiding scaling problems and metric weightings.

    Using IQMT <http://www.lsi.upc.edu/%7Enlp/IQMT/> offers a number of 
advantages over previous MT evaluation packages.
First, individual metrics improve their level of correlation with human 
judgements
when they are applied inside QARLA. Second, it permits to avoid the 
'metric bias'
problem, by allowing you to tune your system on a combination of metrics 
instead
of on a single metric. Third, it allows you to define a set of subtle 
metrics focusing
on partial aspects of MT quality possibly at different linguistic 
levels, and then
combine them into a single measure.

    Several well-known and freely available current MT evaluation tools 
such as
BLEU, GTM, METEOR, NIST, ROUGE, WER and PER, have been incorporated
so far. We plan to incorporate new metrics working at linguistic levels 
other than
lexical overlap in a near future.

    However, the main feature of the IQMT package is that it allows to 
supply
user-defined MT metrics. So, do not hesitate to try your own metrics inside
QARLA!

  Please feel free to download IQMT v1.0 at 
http://www.lsi.upc.edu/~nlp/IQMT/ <http://www.lsi.upc.edu/%7Enlp/IQMT/> .

  Any feedback (comments, suggestions, bug reporting) from the NLP community
members will be highly appreciated.


   IQMT developing team.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20051118/063dc530/attachment.htm>