Chunyu, your paper is very nice. A key to the success of your approach seems to be the fact that you experimented only on data from Hong Kong government Web sites, where one could expect both a higher density of parallel pages and a greater degree of conformity to URL naming regularities, as compared to the Web sites in general. The language independence of your student's approach is attractive and should make it easy to investigate a broader range of language pairs and types of Web sites, in order to see how much of a difference that makes. Is this a direction you are exploring, or which you plan to explore? (If so, the URL sets at
<a href="http://umiacs.umd.edu/~resnik/strand/">http://umiacs.umd.edu/~resnik/strand/</a> would make it easy to find hosts already known to contain parallel pages for several language pairs.)<br><br>Best regards,<br><br> Philip
<br><br><br><div><span class="gmail_quote">On 10/11/07, <b class="gmail_sendername">Kit Chun Yu</b> <<a href="mailto:ctckit@cityu.edu.hk">ctckit@cityu.edu.hk</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000">
<tt>Dear Yorick, <br>
<br>
An MPhil student of mine is currently working on automatic Web page
pairing for bitext mining via automatic URL pairing pattern discovery.
It relies on no pre-defined pattern or any text content/structure
comparison but only on a best-first search for an optimal set of
patterns/keys within the URL strings from a Web site in terms of their
linking power (= the number of possible Web page pairs they can paired
up). That is, it works on URL strings only (+language identification,
of course). It is so simple that everyone may try, I think. It is so
simple that I've got a piece of comment from a conference reviewer that
it is too simple for publication, although simplicity = beauty, in a
sense. -:) Our experiments show that it can achieve an </tt><tt>F-score
of 96.4% </tt><tt>on web page pairing for HK bilingual Web sites. I am
not sure if this simple technique could help a bit to get an estimation
of the figures you are interested. <br>
<br>
</tt><big><big><span style="font-size: 10pt;" lang="EN-US">Chunyu
Kit and Jessica Y. H. Ng. 2007. <a href="http://personal.cityu.edu.hk/%7Ectckit/papers/Kit-Ng_URLpairing-PID483174.pdf" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">An
intelligent Web agent
to mine bilingual parallel pages via automatic discovery of URL pairing
patterns</a>. To appear in <span style="font-style: italic;"> the
Agents and Data Mining Interaction Workshop </span> (ADMI-07), Silicon
Valley, California,
November 2-5, 2007.</span></big></big><tt> (But please mind a flaw in
the formulation part: the search space should be UxU' (not U),
similarly to TxT' for possible token pairs that we fortunately got it
right -:)<br>
<br>
Best wishes,<br>
Chunyu</tt><br>
<br>
<br>
<br>
Yorick Wilks wrote:
<blockquote cite="http://midE05D3D89-4AF5-4BAF-AECF-0A31D89D511C@dcs.shef.ac.uk" type="cite"><div><span class="e" id="q_1158eb6fd8f47a64_1">Everyone is aware that some languages/cultures (e.g.
Swedish, Finnish) tend to have alternative webpages in English, while
others (e.g. Arabic) are much less likely to.
<div><span>Does anyone have any reliable
figures as to the frequency of appearance of these parallel-corpora
(in English)for different (source) languages? I am interested at the
moment in :<font face="Arial" size="4"><span style="font-size: 16px;"></span></font></span></div>
<div><span><font face="Arial" size="4"><span style="font-size: 16px;">Ja</span></font><font face="Arial" size="4"><span style="font-size: 16px;">panese, Chinese, Korean, Spanish, Portuguese,
French, German, Italian, Arabic</span></font></span></div>
<div><font face="Arial" size="4"><span style="font-size: 16px;"><br>
</span></font></div>
<div> I would be grateful for any help.</div>
<div>Regards</div>
<div>Yorick Wilks</div>
<div><br>
</div>
<div><font face="Arial" size="4"><span style="font-size: 16px;"><br>
</span></font></div>
</span></div><pre><hr size="4" width="90%"><span class="q">
_______________________________________________<br>Corpora mailing list<br><a href="mailto:Corpora@uib.no" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">Corpora@uib.no</a>
<a href="http://mailman.uib.no/listinfo/corpora" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://mailman.uib.no/listinfo/corpora</a>
</span></pre>
</blockquote>
<br>
<br>
<pre cols="72">-- <br>Chunyu Kit, PhD<br>Assistant Professor in Computational Linguistics<br><br>Dept. of Chinese, Translation & Linguistics<br>City University of Hong Kong<br>83 Tat Chee Ave., Kowloon<br><br><a href="mailto:E-mail:ctckit@cityu.edu.hk" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
E-mail:ctckit@cityu.edu.hk</a>
<a href="http://personal.cityu.edu.hk/%7Ectckit/" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://personal.cityu.edu.hk/~ctckit/</a>
Fax: (+852)2788 8706, 2788 8732
Tel: (+852)2788 9310 (O), 9380 1738 (M)
(+86)136 5881 2972 (China Mobile)</pre>
</div>
<br>_______________________________________________<br>Corpora mailing list<br><a onclick="return top.js.OpenExtLink(window,event,this)" href="mailto:Corpora@uib.no">Corpora@uib.no</a><br><a onclick="return top.js.OpenExtLink(window,event,this)" href="http://mailman.uib.no/listinfo/corpora" target="_blank">
http://mailman.uib.no/listinfo/corpora</a><br><br></blockquote></div><br>