27.918, FYI: New NIST Evaluation: Low Resource HLT Evaluation

[The LINGUIST List via LINGUIST]

LINGUIST List: Vol-27-918. Mon Feb 22 2016. ISSN: 1069 - 4875.

Subject: 27.918, FYI: New NIST Evaluation: Low Resource HLT Evaluation

Moderators: linguist at linguistlist.org (Damir Cavar, Malgorzata E. Cavar)
Reviews: reviews at linguistlist.org (Anthony Aristar, Helen Aristar-Dry, Sara Couture)
Homepage: http://linguistlist.org

*****************    LINGUIST List Support    *****************
                   25 years of LINGUIST List!
Please support the LL editors and operation with a donation at:
           http://funddrive.linguistlist.org/donate/

Editor for this issue: Ashley Parker <ashley at linguistlist.org>
================================================================


Date: Mon, 22 Feb 2016 09:29:32
From: Shudong Huang [shudongh at gmail.com]
Subject: New NIST Evaluation: Low Resource HLT Evaluation

 
NIST is pleased to announce a new evaluation series aimed to advance human
language technologies that provide rapid and effective response to emerging
incidents where the language resources are very limited. This evaluation
series is called Low Resource HLT Evaluation (LoReHLT). LoReHLT16 plans to
offer three evaluation tasks: machine translation (MT), named entity
recognition (NER), and topic labeling (TL). 

Highlights:

- Two confirmed evaluation tasks: MT and NER (Topic Labeling is under
development)
- Two training conditions: constrained (required) and unconstrained
- Surprise language evaluation
- Three evaluation checkpoints to gauge performance based on training
resources given

2016 Schedule: 

Feb 19 - May 2: Registration period
Jun 2 - 8: Dry run period
Jun 29: Encrypted evaluation data released
Jul 6: Surprise language announced and evaluation begins
Jul 13: Checkpoint 1 due
Jul 20: Checkpoint 2 due
Aug 3: Checkpoint 3 due
Aug 28 – 29: Post evaluation workshop in Washington state

For more information about the LoReHLT16 evaluation, please see the evaluation
plan at http://www.nist.gov/itl/iad/mig/lorehlt16.cfm. If you have any
questions, contact us at lorehlt_poc at nist.gov.

Feel free to forward this message to your colleagues who may find the
evaluation of interest.

Best Regards,
NIST LoReHLT team
 



Linguistic Field(s): Computational Linguistics





 



------------------------------------------------------------------------------

*****************    LINGUIST List Support    *****************
Please support the LL editors and operation with a donation at:
            http://funddrive.linguistlist.org/donate/
 


----------------------------------------------------------
LINGUIST List: Vol-27-918	
----------------------------------------------------------