<div dir="ltr">I am forwarding this post on request of Mr. Sujit Pal, Director, Search R&D at Healthline Networks Inc., San Francisco, United States, who has carried out the following experiment.<span class="">The post could be accessed </span><span class=""><span class="">directly</span> from his blog (</span><span class=""><span class=""><a target="blank" href="https://www.linkedin.com/redirect?url=http%3A%2F%2Fsujitpal%2Eblogspot%2Ecom%2F2014%2F04%2Ffind-temporalaty-of-sentences-with%2Ehtml&urlhash=OLJJ&_t=tracking_disc" rel="nofollow">http://sujitpal.blogspot.com/2014/04/find-temporalaty-of-sentences-with.html</a></span>). <br>
<br></span><br>In my <a href="http://sujitpal.blogspot.com/2014/04/scala-implementation-of-negex-algorithm.html">previous post</a>, I described a Scala implementation of the <a href="https://code.google.com/p/negex/">Negex algorithm</a>,
that was initially developed to detect negation of phrases in a
sentence using manually built pre- and post- trigger terms. Later on, it
was found that the same algorithm could be used to also detect
temporality and experiencer characteristics of a phrase in a sentence,
using a different set of pre- and post- trigger terms.<br>
<br>
I also recently became aware of the <a href="https://tempowordnet.greyc.fr/">TempoWordNet project</a> from a group at <a href="https://www.greyc.fr/node/35">Normandie University</a>.
The project provides a free lexical resource that contains each synset
of Wordnet marked up with its probability of being past, present, future
or atemporal. <a href="http://www.aclweb.org/anthology/E/E14/E14-4002.pdf">This paper</a> describes the process by which these probabilities were generated. There is <a href="http://dl.acm.org/citation.cfm?id=2579042">another paper</a>
which one of the authors referenced on LinkedIn, but its unfortunately
behind an ACM paywall, and I am no longer a member starting this year,
so could not read it.<br>
<br>
In running the Negex algorithm against the annotated list that comes
with it, I found that the Historical and Hypothetical annotations had
lower accuracies (0.90 and 0.89 respectively) compared to the other two
(Negation and Experiencer). Thinking about it, I realized that the
Historical and Hypothetical annotators are a pair of binary annotators
used to annotate a phrase into one of 3 classes - Historical, Recent and
Not Particular. With this understanding, some small changes in how I
measured the accuracy brought them up to 0.93 and 0.99 respectively. But
I figured that may be possible to <i>also</i> compute temporality using
the TempoWordNet file, similar to how one does classical sentiment
analysis. This post describes that work.<br>
<br>
Each synset in the TempoWordNet file is written as a triple of word, part of speech and synset ID. From this, I build a <a href="http://alias-i.com/lingpipe/docs/api/com/aliasi/dict/ExactDictionaryChunker.html">LingPipe ExactDictionaryChunker</a>
for each word/POS pair for each temporality state. I check to see that I
only capture the first synset ID for the word/POS combination, so
hopefully I capture the probabilities for the most dominant synset (the
first one). I have written earlier about the LingPipe
ExactDictionaryChunker, it implements the Aho-Corasick string matching
algorithm which is very fast and space efficient.<br>
<br>
Each sentence is tokenized into words and POS tagged (using <a href="http://blog.dpdearing.com/2011/06/part-of-speech-pos-tagging-with-opennlp-1-5-0/">OpenNLP's POS tagger</a>.
Each word/POS combination is matched against each of the
ExactDictionaryChunkers and the probability of matching word for each
tense summed across all words. The class for which the sum of
probabilities of individual words is the highest is the class of the
sentence. Since OpenNLP uses the <a href="https://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html">Penn Treebank tags</a>, they need to be translated to <a href="http://wordnet.princeton.edu/man/wndb.5WN.html">Wordnet tags</a> before matching.<br>
<br>
Here is the code for the TemporalAnnotator.<br>
<br>
<table class=""><tbody><tr><td class=""><div class=""><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94</pre></div></td><td class=""><div class=""><pre><span class="">// Source: src/main/scala/com/mycompany/scalcium/transformers/TemporalAnnotator.scala</span>
<span class="">package</span> <span class="">com.mycompany.scalcium.transformers</span>
<span class="">import</span> <span class="">java.io.File</span>
<span class="">import</span> <span class="">java.util.regex.Pattern</span>
<span class="">import</span> <span class="">scala.Array.canBuildFrom</span>
<span class="">import</span> <span class="">scala.collection.JavaConversions.asScalaIterator</span>
<span class="">import</span> <span class="">scala.collection.mutable.ArrayBuffer</span>
<span class="">import</span> <span class="">scala.io.Source</span>
<span class="">import</span> <span class="">com.aliasi.chunk.Chunker</span>
<span class="">import</span> <span class="">com.aliasi.dict.DictionaryEntry</span>
<span class="">import</span> <span class="">com.aliasi.dict.ExactDictionaryChunker</span>
<span class="">import</span> <span class="">com.aliasi.dict.MapDictionary</span>
<span class="">import</span> <span class="">com.aliasi.tokenizer.IndoEuropeanTokenizerFactory</span>
<span class="">import</span> <span class="">com.mycompany.scalcium.utils.Tokenizer</span>
<span class="">class</span> <span class="">TemporalAnnotator</span><span class="">(</span><span class="">val</span> <span class="">tempoWNFile</span><span class="">:</span> <span class="">File</span><span class="">)</span> <span class="">{</span>
<span class="">val</span> <span class="">targets</span> <span class="">=</span> <span class="">List</span><span class="">(</span><span class="">"Past"</span><span class="">,</span> <span class="">"Present"</span><span class="">,</span> <span class="">"Future"</span><span class="">)</span>
<span class="">val</span> <span class="">chunkers</span> <span class="">=</span> <span class="">buildChunkers</span><span class="">(</span><span class="">tempoWNFile</span><span class="">)</span>
<span class="">val</span> <span class="">tokenizer</span> <span class="">=</span> <span class="">Tokenizer</span><span class="">.</span><span class="">getTokenizer</span><span class="">(</span><span class="">"opennlp"</span><span class="">)</span>
<span class="">val</span> <span class="">ptPoss</span> <span class="">=</span> <span class="">List</span><span class="">(</span><span class="">"JJ"</span><span class="">,</span> <span class="">"NN"</span><span class="">,</span> <span class="">"VB"</span><span class="">,</span> <span class="">"RB"</span><span class="">)</span>
<span class="">.</span><span class="">map</span><span class="">(</span><span class="">p</span> <span class="">=></span> <span class="">Pattern</span><span class="">.</span><span class="">compile</span><span class="">(</span><span class="">p</span> <span class="">+</span> <span class="">".*"</span><span class="">))</span>
<span class="">val</span> <span class="">wnPoss</span> <span class="">=</span> <span class="">List</span><span class="">(</span><span class="">"s"</span><span class="">,</span> <span class="">"n"</span><span class="">,</span> <span class="">"v"</span><span class="">,</span> <span class="">"r"</span><span class="">)</span>
<span class="">def</span> <span class="">predict</span><span class="">(</span><span class="">sentence</span><span class="">:</span> <span class="">String</span><span class="">)</span><span class="">:</span> <span class="">String</span> <span class="">=</span> <span class="">{</span>
<span class="">val</span> <span class="">scoreTargetPairs</span> <span class="">=</span> <span class="">chunkers</span><span class="">.</span><span class="">map</span><span class="">(</span><span class="">chunker</span> <span class="">=></span> <span class="">{</span>
<span class="">val</span> <span class="">taggedSentence</span> <span class="">=</span> <span class="">tokenizer</span><span class="">.</span><span class="">posTag</span><span class="">(</span><span class="">sentence</span><span class="">)</span>
<span class="">.</span><span class="">map</span><span class="">(</span><span class="">wtp</span> <span class="">=></span> <span class="">wtp</span><span class="">.</span><span class="">_1</span><span class="">.</span><span class="">replaceAll</span><span class="">(</span><span class="">"\\p{Punct}"</span><span class="">,</span> <span class="">""</span><span class="">)</span> <span class="">+</span>
<span class="">"/"</span> <span class="">+</span> <span class="">wordnetPos</span><span class="">(</span><span class="">wtp</span><span class="">.</span><span class="">_2</span><span class="">))</span>
<span class="">.</span><span class="">mkString</span><span class="">(</span><span class="">" "</span><span class="">)</span>
<span class="">val</span> <span class="">chunking</span> <span class="">=</span> <span class="">chunker</span><span class="">.</span><span class="">chunk</span><span class="">(</span><span class="">taggedSentence</span><span class="">)</span>
<span class="">chunking</span><span class="">.</span><span class="">chunkSet</span><span class="">().</span><span class="">iterator</span><span class="">().</span><span class="">toList</span>
<span class="">.</span><span class="">map</span><span class="">(</span><span class="">chunk</span> <span class="">=></span> <span class="">chunk</span><span class="">.</span><span class="">score</span><span class="">())</span>
<span class="">.</span><span class="">foldLeft</span><span class="">(</span><span class="">0.0D</span><span class="">)(</span><span class="">_</span> <span class="">+</span> <span class="">_</span><span class="">)</span>
<span class="">})</span>
<span class="">.</span><span class="">zipWithIndex</span>
<span class="">.</span><span class="">filter</span><span class="">(</span><span class="">stp</span> <span class="">=></span> <span class="">stp</span><span class="">.</span><span class="">_1</span> <span class="">></span> <span class="">0.0D</span><span class="">)</span>
<span class="">if</span> <span class="">(</span><span class="">scoreTargetPairs</span><span class="">.</span><span class="">isEmpty</span><span class="">)</span> <span class="">"Present"</span>
<span class="">else</span> <span class="">{</span>
<span class="">val</span> <span class="">bestTarget</span> <span class="">=</span> <span class="">scoreTargetPairs</span>
<span class="">.</span><span class="">sortWith</span><span class="">((</span><span class="">a</span><span class="">,</span><span class="">b</span><span class="">)</span> <span class="">=></span> <span class="">a</span><span class="">.</span><span class="">_1</span> <span class="">></span> <span class="">b</span><span class="">.</span><span class="">_1</span><span class="">)</span>
<span class="">.</span><span class="">head</span><span class="">.</span><span class="">_2</span>
<span class="">targets</span><span class="">(</span><span class="">bestTarget</span><span class="">)</span>
<span class="">}</span>
<span class="">}</span>
<span class="">def</span> <span class="">buildChunkers</span><span class="">(</span><span class="">datafile</span><span class="">:</span> <span class="">File</span><span class="">)</span><span class="">:</span> <span class="">List</span><span class="">[</span><span class="">Chunker</span><span class="">]</span> <span class="">=</span> <span class="">{</span>
<span class="">val</span> <span class="">dicts</span> <span class="">=</span> <span class="">ArrayBuffer</span><span class="">[</span><span class="">MapDictionary</span><span class="">[</span><span class="">String</span><span class="">]]()</span>
<span class="">Range</span><span class="">(</span><span class="">0</span><span class="">,</span> <span class="">targets</span><span class="">.</span><span class="">size</span><span class="">).</span><span class="">foreach</span><span class="">(</span><span class="">i</span> <span class="">=></span>
<span class="">dicts</span> <span class="">+=</span> <span class="">new</span> <span class="">MapDictionary</span><span class="">[</span><span class="">String</span><span class="">]())</span>
<span class="">val</span> <span class="">pwps</span> <span class="">=</span> <span class="">scala</span><span class="">.</span><span class="">collection</span><span class="">.</span><span class="">mutable</span><span class="">.</span><span class="">Set</span><span class="">[</span><span class="">String</span><span class="">]()</span>
<span class="">Source</span><span class="">.</span><span class="">fromFile</span><span class="">(</span><span class="">datafile</span><span class="">).</span><span class="">getLines</span><span class="">()</span>
<span class="">.</span><span class="">filter</span><span class="">(</span><span class="">line</span> <span class="">=></span> <span class="">(!(</span><span class="">line</span><span class="">.</span><span class="">isEmpty</span><span class="">()</span> <span class="">||</span> <span class="">line</span><span class="">.</span><span class="">startsWith</span><span class="">(</span><span class="">"#"</span><span class="">))))</span>
<span class="">.</span><span class="">foreach</span><span class="">(</span><span class="">line</span> <span class="">=></span> <span class="">{</span>
<span class="">val</span> <span class="">cols</span> <span class="">=</span> <span class="">line</span><span class="">.</span><span class="">split</span><span class="">(</span><span class="">"\\s{2,}"</span><span class="">)</span>
<span class="">val</span> <span class="">wordPos</span> <span class="">=</span> <span class="">getWordPos</span><span class="">(</span><span class="">cols</span><span class="">(</span><span class="">1</span><span class="">))</span>
<span class="">val</span> <span class="">probs</span> <span class="">=</span> <span class="">cols</span><span class="">.</span><span class="">slice</span><span class="">(</span><span class="">cols</span><span class="">.</span><span class="">size</span> <span class="">-</span> <span class="">4</span><span class="">,</span> <span class="">cols</span><span class="">.</span><span class="">size</span><span class="">)</span>
<span class="">.</span><span class="">map</span><span class="">(</span><span class="">x</span> <span class="">=></span> <span class="">x</span><span class="">.</span><span class="">toDouble</span><span class="">)</span>
<span class="">if</span> <span class="">(!</span> <span class="">pwps</span><span class="">.</span><span class="">contains</span><span class="">(</span><span class="">wordPos</span><span class="">))</span> <span class="">{</span>
<span class="">Range</span><span class="">(</span><span class="">0</span><span class="">,</span> <span class="">targets</span><span class="">.</span><span class="">size</span><span class="">).</span><span class="">foreach</span><span class="">(</span><span class="">i</span> <span class="">=></span>
<span class="">dicts</span><span class="">(</span><span class="">i</span><span class="">).</span><span class="">addEntry</span><span class="">(</span><span class="">new</span> <span class="">DictionaryEntry</span><span class="">[</span><span class="">String</span><span class="">](</span>
<span class="">wordPos</span><span class="">,</span> <span class="">targets</span><span class="">(</span><span class="">i</span><span class="">),</span> <span class="">probs</span><span class="">(</span><span class="">i</span><span class="">))))</span>
<span class="">}</span>
<span class="">pwps</span> <span class="">+=</span> <span class="">wordPos</span>
<span class="">})</span>
<span class="">val</span> <span class="">chunkers</span> <span class="">=</span> <span class="">new</span> <span class="">ArrayBuffer</span><span class="">[</span><span class="">Chunker</span><span class="">]()</span>
<span class="">dicts</span><span class="">.</span><span class="">map</span><span class="">(</span><span class="">dict</span> <span class="">=></span> <span class="">new</span> <span class="">ExactDictionaryChunker</span><span class="">(</span>
<span class="">dict</span><span class="">,</span> <span class="">IndoEuropeanTokenizerFactory</span><span class="">.</span><span class="">INSTANCE</span><span class="">,</span>
<span class="">false</span><span class="">,</span> <span class="">false</span><span class="">))</span>
<span class="">.</span><span class="">toList</span>
<span class="">}</span>
<span class="">def</span> <span class="">getWordPos</span><span class="">(</span><span class="">synset</span><span class="">:</span> <span class="">String</span><span class="">)</span><span class="">:</span> <span class="">String</span> <span class="">=</span> <span class="">{</span>
<span class="">val</span> <span class="">sscols</span> <span class="">=</span> <span class="">synset</span><span class="">.</span><span class="">split</span><span class="">(</span><span class="">"\\."</span><span class="">)</span>
<span class="">val</span> <span class="">words</span> <span class="">=</span> <span class="">sscols</span><span class="">.</span><span class="">slice</span><span class="">(</span><span class="">0</span><span class="">,</span> <span class="">sscols</span><span class="">.</span><span class="">size</span> <span class="">-</span> <span class="">2</span><span class="">)</span>
<span class="">val</span> <span class="">pos</span> <span class="">=</span> <span class="">sscols</span><span class="">.</span><span class="">slice</span><span class="">(</span><span class="">sscols</span><span class="">.</span><span class="">size</span> <span class="">-</span> <span class="">2</span><span class="">,</span> <span class="">sscols</span><span class="">.</span><span class="">size</span> <span class="">-</span> <span class="">1</span><span class="">).</span><span class="">head</span>
<span class="">words</span><span class="">.</span><span class="">mkString</span><span class="">(</span><span class="">""</span><span class="">)</span>
<span class="">.</span><span class="">split</span><span class="">(</span><span class="">"_"</span><span class="">)</span>
<span class="">.</span><span class="">map</span><span class="">(</span><span class="">word</span> <span class="">=></span> <span class="">word</span> <span class="">+</span> <span class="">"/"</span> <span class="">+</span> <span class="">(</span><span class="">if</span> <span class="">(</span><span class="">"s"</span><span class="">.</span><span class="">equals</span><span class="">(</span><span class="">pos</span><span class="">))</span> <span class="">"a"</span> <span class="">else</span> <span class="">pos</span><span class="">))</span>
<span class="">.</span><span class="">mkString</span><span class="">(</span><span class="">" "</span><span class="">)</span>
<span class="">}</span>
<span class="">def</span> <span class="">wordnetPos</span><span class="">(</span><span class="">ptPos</span><span class="">:</span> <span class="">String</span><span class="">)</span><span class="">:</span> <span class="">String</span> <span class="">=</span> <span class="">{</span>
<span class="">val</span> <span class="">matchIdx</span> <span class="">=</span> <span class="">ptPoss</span><span class="">.</span><span class="">map</span><span class="">(</span><span class="">p</span> <span class="">=></span> <span class="">p</span><span class="">.</span><span class="">matcher</span><span class="">(</span><span class="">ptPos</span><span class="">).</span><span class="">matches</span><span class="">())</span>
<span class="">.</span><span class="">zipWithIndex</span>
<span class="">.</span><span class="">filter</span><span class="">(</span><span class="">mip</span> <span class="">=></span> <span class="">mip</span><span class="">.</span><span class="">_1</span><span class="">)</span>
<span class="">.</span><span class="">map</span><span class="">(</span><span class="">mip</span> <span class="">=></span> <span class="">mip</span><span class="">.</span><span class="">_2</span><span class="">)</span>
<span class="">if</span> <span class="">(</span><span class="">matchIdx</span><span class="">.</span><span class="">isEmpty</span><span class="">)</span> <span class="">"o"</span> <span class="">else</span> <span class="">wnPoss</span><span class="">(</span><span class="">matchIdx</span><span class="">.</span><span class="">head</span><span class="">)</span>
<span class="">}</span>
<span class="">}</span>
</pre></div></td></tr>
</tbody></table><br>
As you can see, I just compute the best of Past, Present and Future and
ignore the Atemporal probabilities. I had initially included it as well,
but accuracy scores on the Negex annotated test data was coming out at
0.89. Changing the logic to only look at Past and flag it as Historical
if the sum of past probabilities of matching sentences was greater than 0
got me an even worse accuracy of 0.5. Finally, after a bit of trial and
error, removing the Atemporal chunker resulted in an accuracy of
0.904741, so thats what I stayed with.<br>
<br>
Here is the JUnit test for evaluating the TemporalWordnetAnnotator using
the annotated list of sentences from Negex. Our default is "Recent",
and only when we can confidently say something about the temporality of
the sentence, we will change to either "Historical" or "Recent". Our
annotators will return a score for each of Past, Present and Future. If
the result is "Past" from our annotators, it will be converted to
"Historical" for comparison.<br>
<br>
<table class=""><tbody><tr><td class=""><div class=""><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50</pre></div></td><td class=""><div class=""><pre><span class="">// Source: src/test/scala/com/mycompany/scalcium/transformers/TemporalAnnotatorTest.scala</span>
<span class="">package</span> <span class="">com.mycompany.scalcium.transformers</span>
<span class="">import</span> <span class="">java.io.File</span>
<span class="">import</span> <span class="">scala.io.Source</span>
<span class="">import</span> <span class="">org.junit.Test</span>
<span class="">class</span> <span class="">TemporalAnnotatorTest</span> <span class="">{</span>
<span class="">val</span> <span class="">tann</span> <span class="">=</span> <span class="">new</span> <span class="">TemporalAnnotator</span><span class="">(</span>
<span class="">new</span> <span class="">File</span><span class="">(</span><span class="">"src/main/resources/TempoWnL_1.0.txt"</span><span class="">))</span>
<span class="">val</span> <span class="">input</span> <span class="">=</span> <span class="">new</span> <span class="">File</span><span class="">(</span><span class="">"src/main/resources/negex/test_input.txt"</span><span class="">)</span>
<span class="">@Test</span>
<span class="">def</span> <span class="">evaluate</span><span class="">()</span><span class="">:</span> <span class="">Unit</span> <span class="">=</span> <span class="">{</span>
<span class="">var</span> <span class="">numTested</span> <span class="">=</span> <span class="">0</span>
<span class="">var</span> <span class="">numCorrect</span> <span class="">=</span> <span class="">0</span>
<span class="">Source</span><span class="">.</span><span class="">fromFile</span><span class="">(</span><span class="">input</span><span class="">).</span><span class="">getLines</span><span class="">().</span><span class="">foreach</span><span class="">(</span><span class="">line</span> <span class="">=></span> <span class="">{</span>
<span class="">val</span> <span class="">cols</span> <span class="">=</span> <span class="">line</span><span class="">.</span><span class="">split</span><span class="">(</span><span class="">"\t"</span><span class="">)</span>
<span class="">val</span> <span class="">sentence</span> <span class="">=</span> <span class="">cols</span><span class="">(</span><span class="">3</span><span class="">)</span>
<span class="">val</span> <span class="">actual</span> <span class="">=</span> <span class="">cols</span><span class="">(</span><span class="">5</span><span class="">)</span>
<span class="">if</span> <span class="">((!</span> <span class="">"Not particular"</span><span class="">.</span><span class="">equals</span><span class="">(</span><span class="">actual</span><span class="">)))</span> <span class="">{</span>
<span class="">val</span> <span class="">predicted</span> <span class="">=</span> <span class="">tann</span><span class="">.</span><span class="">predict</span><span class="">(</span><span class="">sentence</span><span class="">)</span>
<span class="">val</span> <span class="">correct</span> <span class="">=</span> <span class="">actual</span><span class="">.</span><span class="">equals</span><span class="">(</span><span class="">translate</span><span class="">(</span><span class="">predicted</span><span class="">))</span>
<span class="">if</span> <span class="">(!</span> <span class="">correct</span><span class="">)</span> <span class="">{</span>
<span class="">Console</span><span class="">.</span><span class="">println</span><span class="">(</span><span class="">"%s|[%s] %s|%s"</span>
<span class="">.</span><span class="">format</span><span class="">(</span><span class="">sentence</span><span class="">,</span> <span class="">(</span><span class="">if</span> <span class="">(</span><span class="">correct</span><span class="">)</span> <span class="">"+"</span> <span class="">else</span> <span class="">"-"</span><span class="">),</span>
<span class="">actual</span><span class="">,</span> <span class="">predicted</span><span class="">))</span>
<span class="">}</span>
<span class="">numCorrect</span> <span class="">+=</span> <span class="">(</span><span class="">if</span> <span class="">(</span><span class="">correct</span><span class="">)</span> <span class="">1</span> <span class="">else</span> <span class="">0</span><span class="">)</span>
<span class="">numTested</span> <span class="">+=</span> <span class="">1</span>
<span class="">}</span>
<span class="">})</span>
<span class="">Console</span><span class="">.</span><span class="">println</span><span class="">(</span><span class="">"Accuracy=%8.6f"</span>
<span class="">.</span><span class="">format</span><span class="">(</span><span class="">numCorrect</span><span class="">.</span><span class="">toDouble</span> <span class="">/</span> <span class="">numTested</span><span class="">.</span><span class="">toDouble</span><span class="">))</span>
<span class="">}</span>
<span class="">/**</span>
<span class=""> * Converts predictions made by TemporalAnnotator to</span>
<span class=""> * predictions that match annotations in our testcase.</span>
<span class=""> */</span>
<span class="">def</span> <span class="">translate</span><span class="">(</span><span class="">pred</span><span class="">:</span> <span class="">String</span><span class="">)</span><span class="">:</span> <span class="">String</span> <span class="">=</span> <span class="">{</span>
<span class="">pred</span> <span class="">match</span> <span class="">{</span>
<span class="">case</span> <span class="">"Past"</span> <span class="">=></span> <span class="">"Historical"</span>
<span class="">case</span> <span class="">_</span> <span class="">=></span> <span class="">"Recent"</span>
<span class="">}</span>
<span class="">}</span>
<span class="">}</span>
</pre></div></td></tr>
</tbody></table><br>
This approach gives us an accuracy of 0.904741, which is not as good as
Negex, but its lack of accuracy is somewhat offset by its ease of use.
You can send the entire sentence to the annotator, no need (as in the
case of Negex) to concept map the sentence before sending so it
identifies the "important" phrases.</div>