Info: ACT, a free, open-source MT Evaluation Metric

[Thierry Hamon]

Date: Thu, 21 Mar 2013 19:20:19 +0100
From: Najeh Hajlaoui <najeh.hajlaoui at idiap.ch>
Message-Id: <2129E973-73C1-46DF-B94C-3D1CBDA46F95 at idiap.ch>
X-url: http://www.idiap.ch/software/act

Dear colleagues,

We have just made available a new metric ACT "Accuracy of Connectives
Translation", a MT metric specific for connectives translation. This is
a free, open-source tool under GPL v3 license (available on github). It
was built using Perl, so it runs on any platform:

http://www.idiap.ch/software/act

ACT is a reference-based metric to measure the accuracy of discourse
connective translation. The metric relies on automatic word-level
alignment (using GIZA++, not included in ACT) between a source sentence
and respectively the reference and candidate translations, along with
other heuristics for comparing translations of discourse connectives.
Using a dictionary of equivalents, the translations are scored
automatically, or, for more accuracy, semi-automatically.  The accuracy
of the ACT metric was assessed by human judges on sample data for
English/French, English/Arabic, English/Italian and English/German
translations; the ACT scores are within 2-4% of human scores.

We would like to maintain and further develop the ACT metric porting it
to other language pairs and to other linguistic phenomena (verb tenses,
pronouns, etc) that still pose problems for current SMT systems, so if
your are interested to extend it to other new language, let us know.

Best regards

Najeh Hajlaoui
Idiap Research Institute  |  tel: (41 27) 721 7761
Centre du Parc, CP 592    |  fax: (41 27) 721 7712
CH-1920 Martigny          |  Najeh.Hajlaoui at idiap.ch
Switzerland               |  http://people.idiap.ch/nhajlaoui

-------------------------------------------------------------------------
Message diffuse par la liste Langage Naturel <LN at cines.fr>
Informations, abonnement : http://www.atala.org/article.php3?id_article=48
English version       : 
Archives                 : http://listserv.linguistlist.org/archives/ln.html
                                http://liste.cines.fr/info/ln

La liste LN est parrainee par l'ATALA (Association pour le Traitement
Automatique des Langues)
Information et adhesion  : http://www.atala.org/
-------------------------------------------------------------------------