Automatic Procedures in MT Evaluation - MT Summit XI, Copenhagen, 11 September 2007

[ELDA]

[Apologies for multiple postings]

Automatic Procedures in MT Evaluation
MT Summit XI, Copenhagen, 11 September 2007
Workshop organised by the ELRA Evaluation Committee

Gregor Thurmair, Linguatec
Khalid Choukri, ELDA
Bente Maegaard, University of Copenhagen

The purpose of this workshop is to discuss automatic evaluation 
procedures in MT. Among the discussion points are:

   * What do the scores really measure?
   * What kind of implicit assumptions do they make?
   * What kind initial effort do they require (e.g.: pre-translate test 
corpus)?
   * What kind of resources do they need (e.g.: third party grammars)?
   * Are they biased towards specific MT technologies?
   * What kind of diagnostic support can they give? (where to improve 
the system)
   * What kind of evaluation criteria (e.g. related to the FEMTI 
framework) do they support (adequacy, fluency, ...)

The objective of the workshop is to have a better understanding of the 
strengths and limitations of the respective approaches,
and perhaps make steps towards defining a common methodology for MT 
output evaluation.

Please visit http://mtsummitcph.ku.dk/workshops/ to find more about the 
workshop's programme.