Shakir, <br><br>I am pretty sure this is not the right forum to give support for a quickly <br>hacked perl script, and the diagnosis of what went wrong is too much a<br>speculation given the bug report.<br>However, maybe I should have been clearer in the usage instructions:<br>
<br><br>So basically what you need to do is check the input format. Your wordlist seems to<br>
be ok (from what I can see in your output sample). The corpora need to be plain text,<br>
and one text-file each (again, see the example input for inspiration)<br><br><br>I tested the script with a wordlist of nouns extracted from the bnc frequency list,<br>and as corpusA the europarl-corpus (en, with no tags) and as corpusB a collection <br>
of Charles Dickens novels (from Gutenberg).<br><br>Again, both corpora (I was hoping the provided example was sufficiently illustrative) <br>must be plain text files, and size should not be a problem, given the fact that I was able<br>
to process the europarl (28m tokens) -- *AND* Charles Dickens ;-) , and it only takes<br>a couple of seconds, and it produced the desired output.<br><br><br>Perhaps it would have been better to come up with a unix-shell pipe example <br>
so that you can see how to do "stuff" quickly yourself, and provide references <br>so you are not lost, and can educate yourself when you reach the limitations <br>of the unix shell one-liner. The pointers so far given are excellent resources <br>
to get your hands dirty quickly, really without having to learn everything about<br>programming. <br>More helpful resources include the perl man-pages ('man perl' or 'man perlintro')<br>in the unix-shell, hopefully your system administrator has them installed for you.<br>
I can do a bit more documentation on the script, but I suggest we handle that <br>in private communication.<br><br>Now, I hope there won't be much need to continue this thread. <br>Sorry, but vanity is my favourite sin ;-)<br>
<br>Kind regards,<br>Alex<br><br><br><div class="gmail_quote">On Feb 18, 2008 4:56 PM, True Friend <<a href="mailto:true.friend2004@gmail.com">true.friend2004@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi Sir<br>Tried your script but ........ it has some problems. Probably the large size of txt files was the reason. Corpus A was about 1.9 million and corpus B was almost as A. It generated only "0"s for each word. Another thing was probably big size of wordlist (1000 words). A glimpse of the result.<br>
votes 0 0<br> whereas 0 0<br> whereby 0 0<br> wherein 0 0<br> without 0 0<br> witness 0 0<br> witnesses 0 0<br>
wound 0 0<br> writ 0 0<br> written 0 0<br> zila 0 0<br> zina 0 0<br> court 0 0<br>When tried with small wordlist it generated only one word (the last one <i>court</i>) plz see the result. <br>
judge 0 0<br> judgment 0 0<br> land 0 0<br> law 0 0<br> learned 0 0<br> order 0 0<br> ordinance 0 0<br>
person 0 0<br> petition 0 0<br> petitioner 0 0<br> police 0 0<br> record 0 0<br> respondent 0 0<br> section 0 0<br>
suit 0 0<br> trial 0 0<br> court 718 11128<br>A procedure which I could make in my mind was like grab the word find its frequency in Corpus A and then in Corpus B and then print it. I could not understand the code (not a programmer yet :D), anyhows there is something wrong. So can you spare some more time for it?<br>
Thanks a lot for your effort to write this script. <br>Regards<br>M Shakir <br>Pakistan<br><br><div class="gmail_quote"><div class="Ih2E3d">On Feb 18, 2008 5:34 PM, Alexander Schutz <<a href="mailto:goalscoringsuperstarhero@gmail.com" target="_blank">goalscoringsuperstarhero@gmail.com</a>> wrote:<br>
</div><div><div></div><div class="Wj3C7c"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hi Shakir,<br><br>as part of a little exercise I wrote a tiny perl script performing what you asked.<br>
It takes as parameters the wordlist, the corpus_A and the corpus_B (each as text files)<br>and produces as output the respective frequencies in each corpus:<br>
alesch@nbgal141:~$ perl wordlist_corpus_freq.pl wordlist.txt vbush.txt How2DoResearchMIT.txt<br> color 1 0<br> colour 0 0<br> furiously 0 0<br> green 0 0<br>
idea 7 22<br> sleep 0 0<br><br>It does some normalisation on the corpora, like conversion to lower case and<br>punctuation removal.<br><br>Please find it as attachment, including the sample wordlist, to this email.<br>
<br>Hth,<br>Alex<br><br><br><br><div class="gmail_quote"><div><div></div><div>On Feb 18, 2008 10:53 AM, True Friend <<a href="mailto:true.friend2004@gmail.com" target="_blank">true.friend2004@gmail.com</a>> wrote:<br>
</div></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><div>
Hi Folks<br>I need a a programm/script (even of *nix) that can provide frequency of a wordlist from two corpora. Actually I have made this list by comparing two word lists one from general english (specifically from Pakistani Origin) and law english (also of Pakistani origin). I know want to present these keywords with their frequencies in both corpora as a proof that these words are more frequent in law. Keywords are generated by Antconc. <br>
Is there any script/tool that can generate a parallel list of frequencies of each word in both corpora?<br>Regards<br>M Shakir Aziz<br>A Corpus Linguistics Student<br>Pakistan<br clear="all"><font color="#888888"><br>-- <br>
محمد شاکر عزیز
</font><br></div></div><div>_______________________________________________<br>Corpora mailing list<br><a href="mailto:Corpora@uib.no" target="_blank">Corpora@uib.no</a><br><a href="http://mailman.uib.no/listinfo/corpora" target="_blank">http://mailman.uib.no/listinfo/corpora</a><br>
<br></div></blockquote></div><font color="#888888"><br><br clear="all"><br>-- <br>Alexander Schutz,<br>Digital Enterprise Research Institute,<br>Ollscoil na hÉireann, Gaillimh<br>Galway, Ireland
</font></blockquote></div></div></div><font color="#888888"><br><br clear="all"><br>-- <br>محمد شاکر عزیز
</font></blockquote></div><br><br clear="all"><br>-- <br>Alexander Schutz,<br>Digital Enterprise Research Institute,<br>Ollscoil na hÉireann, Gaillimh<br>Galway, Ireland