[Corpora-List] A question about Label Bias Problem!

[Chen Wenliang]

Dear all,

Recently, I read the paper(CRFs: Probabilistic Models for Segmenting and Lableling Sequenc Data, ICML01).
I have a question, when I demonstrated why Maximum Entropy Markov Models are prone to the Label Bias problem.

Suppose we have a finite set of states S= {s1, s2}, a set of possible observation O.
Time t-1, we have state s'. Then at time t, we compute P(s1|Ot,s') and P(s2|Ot,s') using the exponential form(can be found at paper: MEMM for Information Extraction and Segmentation, ICML00).

We can have that s' is s1. Suppose there are not pair <s1, s2> in the training data, we have P(s2|Ot, s')=e^0/Z=1/Z, that is not zero. So state s' always has two outgoing transitions even if we have the sparse data.
So I think that "State 1 and 4 of Fig. 1 in the paper have only one outgoing transition" will be not true in MEMM. So I confuse whether MEMM have label bias problem or not.

Can you show me an example for why MEMM has label bias problem in Part of Speech Tagging or NER or some NLP applications?


Thanks　　

Chen Wenliang chenwl at mail.neu.edu.cn

Nlplab, Northeastern University, China.

									2005-05-16