[Corpora-List] Problems in using ACE evaluation scripts

[lixxx345 at umn.edu]

Dear All,

I am using ACE corpus for relational discovery. I read ACE documents and 
tried to run a sample evaluation. But I get the following errors. I don't 
know what wrong here. Would anyone who is familiar with ACE evaluations 
give me helps?

I tried two scripts and used the same ACE standard xml files as inputs, but 
got the same error message.

The command line I used: 

ace05-eval-v14a.pl 
-R ace_phase2\data\ace2_devtest\bnews\ABC19981001.1830.1257.sgm.tmx.rdc.xml 
-T \ace_phase2\data\ace2_devtest\bnews\ABC19981001.1830.1257.sgm.tmx.rdc.xml

The output: 

Entity scoring is LEVEL weighted
 Relation/Event mapping is allowed when ARGUMENTS criterion is satisfied
 Overlap of BOTH of Arg-* relation arguments is required for mapping
 Arguments contribute to scoring when arguments are OVERLAPPING
 Argument averaging is ARITHMETIC
 Scoring mode parameter set is "2005-TERN"

  min acceptable overlap of matching mention heads or names:
       30.0 percent
  min acceptable run of matching characters in mention heads:
        0.0 percent
  max acceptable extent difference for names and mentions to match:
          4 chars for text sources
      0.400 sec for audio sources
      0.400 cm for image sources

  Entity mention values:
      1.000 for type NAM
      0.500 for type NOM
      0.100 for type PRO
  Entity value weights for entity types:
      1.000 for type FAC
      1.000 for type GPE
      1.000 for type LOC
      1.000 for type ORG
      1.000 for type PER
      1.000 for type VEH
      1.000 for type WEA
  Entity value weights for entity classes:
      1.000 for class SPC
      0.000 for class GEN
      0.000 for class NEG
      0.000 for class USP
  Entity value discounts for entity attribute recognition errors:
      0.750 for CLASS errors
      0.900 for SUBTYPE errors
      0.500 for TYPE errors
  Entity mention value discounts for mention attribute recognition errors:
      0.900 for ROLE errors
      0.900 for STYLE errors
      0.900 for TYPE errors
  Entity value (cost) weight for spurious (false alarm) entities: 0.750
  Entity mention value (cost) weight for spurious entity mentions: 0.750
  Entity mention value (cost) discount for incorrect coreference: 0.000

  Relation value weights for relation types:
      1.000 for type ART
      1.000 for type GEN-AFF
      1.000 for type METONYMY
      1.000 for type ORG-AFF
      1.000 for type PART-WHOLE
      1.000 for type PER-SOC
      1.000 for type PHYS
  Relation value weights for relation modalities:
      1.000 for modality Asserted
      1.000 for modality Other
      1.000 for modality Unspecified
  Relation value discounts for relation attribute recognition errors:
      0.750 for MODALITY errors
      0.700 for SUBTYPE errors
      1.000 for TENSE errors
      1.000 for TYPE errors
  Relation value (cost) weight for spurious (false alarm) relations: 0.750
  Relation value (cost) weight for spurious relation arguments: 0.000
  Relation argument value (cost) discount for argument role errors: 0.750

  Event value weights for event types:
      1.000 for type Business
      1.000 for type Conflict
      1.000 for type Contact
      1.000 for type Justice
      1.000 for type Life
      1.000 for type Movement
      1.000 for type Personnel
      1.000 for type Transaction
  Event value weights for event modalities:
      1.000 for modality Asserted
      1.000 for modality Other
  Event value discounts for event attribute recognition errors:
      1.000 for GENERICITY errors
      0.750 for MODALITY errors
      1.000 for POLARITY errors
      0.900 for SUBTYPE errors
      1.000 for TENSE errors
      0.500 for TYPE errors
  Event value (cost) weight for spurious (false alarm) events: 0.750
  Event value (cost) weight for spurious event arguments: 0.500
  Event argument value (cost) discount for argument role errors: 0.750

  Timex2 attribute value weights for timex2 attributes:
      0.000 for type ANCHOR_DIR
      0.000 for type ANCHOR_VAL
      0.000 for type MOD
      0.000 for type SET
      0.500 for type TYPE
      0.000 for type VAL
  Timex2 value (cost) weight for spurious (false alarm) timex2's: 0.750
  Timex2 mention value (cost) weight for spurious timex2 mentions: 0.750
  Timex2 mention value (cost) discount for incorrect coreference: 0.000

  Value value weights for value types:
      1.000 for type Contact-Info
      1.000 for type Crime
      1.000 for type Illness
      1.000 for type Job-Title
      1.000 for type Numeric
      1.000 for type Sentence
  Value value discounts for value attribute recognition errors:
      0.900 for SUBTYPE errors
      0.500 for TYPE errors
  Value value (cost) weight for spurious (false alarm) values: 0.750
  Value mention value (cost) weight for spurious value mentions: 0.750
  Value mention value (cost) discount for incorrect coreference: 0.000

-------------------------------------------------------------------------------

FATAL INPUT ERROR for entity 'ABC19981001.1830.1257-E1'
    in document 'ABC19981001.1830.1257'
    in file 'C:\Documents and 
Settings\lidi00000\SICCD_RD\ace_phase2\data\ace2_devtest\bnews\ABC19981001.1830.1257.sgm.tmx.rdc.xml'
    Missing entity TYPE attribute

--------------------------------------------------------------------------------

Above is the error message.


Thanks,

Dingcheng Li


_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora