27.1285, FYI: LoReHLT 2016 Evaluations (NIST)

[The LINGUIST List via LINGUIST]

LINGUIST List: Vol-27-1285. Mon Mar 14 2016. ISSN: 1069 - 4875.

Subject: 27.1285, FYI: LoReHLT 2016 Evaluations (NIST)

Moderators: linguist at linguistlist.org (Damir Cavar, Malgorzata E. Cavar)
Reviews: reviews at linguistlist.org (Anthony Aristar, Helen Aristar-Dry, Sara Couture)
Homepage: http://linguistlist.org

*****************    LINGUIST List Support    *****************
                   25 years of LINGUIST List!
Please support the LL editors and operation with a donation at:
           http://funddrive.linguistlist.org/donate/

Editor for this issue: Ashley Parker <ashley at linguistlist.org>
================================================================


Date: Mon, 14 Mar 2016 14:40:10
From: Shudong Huang [shudong.huang at nist.gov]
Subject: LoReHLT 2016 Evaluations (NIST)

 
NIST is pleased to announce a new evaluation series aimed to advance human
language technologies that provide rapid and effective response to emerging
incidents where the language resources are very limited. This evaluation
series is called Low Resource HLT Evaluation (LoReHLT). LoReHLT16 plans to
offer three evaluation tasks: machine translation (MT), named entity
recognition (NER), and situation frames (SF).

Highlights:

- Three evaluation tasks: MT, NER, and SF
- Two training conditions: constrained (required) and unconstrained
- Surprise language evaluation
- Three evaluation checkpoints to gauge performance based on training
resources given

2016 Schedule:

Feb 19 - May 2: Registration period
Jun 2 - 8: Dry run period
Jun 29: Encrypted evaluation data released
Jul 6: Surprise language announced and evaluation begins
Jul 13: Checkpoint 1 due
Jul 20: Checkpoint 2 due
Aug 3: Checkpoint 3 due
Aug 28 – 29: Post evaluation workshop (location TBD)

For more information about the LoReHLT16 evaluation, please see the evaluation
plan at http://www.nist.gov/itl/iad/mig/lorehlt16.cfm. If you have any
questions, contact us at lorehlt_poc at nist.gov.

Feel free to forward this message to your colleagues who may find the
evaluation of interest.

Best Regards,
NIST LoReHLT team
 



Linguistic Field(s): Computational Linguistics





 



------------------------------------------------------------------------------

*****************    LINGUIST List Support    *****************
Please support the LL editors and operation with a donation at:
            http://funddrive.linguistlist.org/donate/
 


----------------------------------------------------------
LINGUIST List: Vol-27-1285	
----------------------------------------------------------