<font style="font-family: verdana,sans-serif;" face="verdana,sans-serif">Hi All,<br><br>Thanks for confirming that MapReduce is the way to go and the tutorials! I was trying to go through some of the tutorials, but they lack specific details about using a java project. So, I changed my question. Please excuse me if you consider this discussion inappropriate for this list and ignore the rest. I thought this is a problem that most of us would be facing. Java is the most popular language for NLP (<a href="http://nlpers.blogspot.com/2009/03/programming-language-of-choice.html">http://nlpers.blogspot.com/2009/03/programming-language-of-choice.html</a>) and we all need to map to clusters and reduce our processing. Further, Amazon servers is the way to go for many that don't have access to personal HPC clusters.<br>
<br></font><span style="font-family: verdana,sans-serif;">Wondering if someone could help me with precise instructions to use
Amazon MapReduce for the simple java program below? It has one class
that takes an input, has a dictionary and produces an output. (Basically
whatever is in input, if it is present in dictionary) I would use that
as a template for my java application. I need mapreduce I want to
decrease the time taken for a complex application by n-fold.</span><br style="font-family: verdana,sans-serif;">
<br style="font-family: verdana,sans-serif;"><span style="font-family: verdana,sans-serif;">
I'm kind of lost trying to learn different things. It is easier to do it the other way, I guess. Someone, please?</span><br style="font-family: verdana,sans-serif;">
<br style="font-family: verdana,sans-serif;"><span style="font-family: verdana,sans-serif;">
Here is the tested code:</span><br style="font-family: verdana,sans-serif;">
<a style="font-family: verdana,sans-serif;" class="jive-link-external" href="http://dl.dropbox.com/u/6777654/Simple.zip">http://dl.dropbox.com/u/6777654/Simple.zip</a><br style="font-family: verdana,sans-serif;">
<br style="font-family: verdana,sans-serif;"><span style="font-family: verdana,sans-serif;">
I greatly appreciate you spending 5-10 minutes in giving simple instructions that a java programmer with knowledge of MapReduce and familiarity with Amazon servers could use.</span><br style="font-family: verdana,sans-serif;">
<br style="font-family: verdana,sans-serif;"><span style="font-family: verdana,sans-serif;">
Thanks.
</span><br style="font-family: verdana,sans-serif;"><font style="font-family: verdana,sans-serif;" face="verdana,sans-serif"><br clear="all"></font><span style="font-family:verdana,sans-serif">Sincerely,</span><br style="font-family:verdana,sans-serif">
<span style="font-family:verdana,sans-serif">Siddhartha Jonnalagadda, </span><br style="font-family:verdana,sans-serif"><span style="font-family:verdana,sans-serif">Text mining Researcher, Lnx Research, LLC, Orange, CA</span><br style="font-family:verdana,sans-serif">
<a style="font-family:verdana,sans-serif" href="http://sjonnalagadda.wordpress.com" target="_blank">sjonnalagadda.wordpress.com</a><br style="font-family:verdana,sans-serif"><br style="font-family:verdana,sans-serif"><span style="border-collapse:separate;border-spacing:0px;color:rgb(0, 0, 0);font-family:verdana,sans-serif;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div style="word-wrap:break-word">
<span style="border-collapse:separate;border-spacing:0px;color:rgb(0, 0, 0);font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><p style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;line-height:normal;font-size-adjust:none;font-stretch:normal;min-height:14px">
<font size="1"><br></font></p><p style="margin: 0px;"><font style="font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;line-height:normal;font-size-adjust:none;font-stretch:normal" size="1">Confidentiality Notice:</font></p>
<p style="margin: 0px;"><font style="font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;line-height:normal;font-size-adjust:none;font-stretch:normal" size="1">This
e-mail message, including any attachments, is for the sole use of the
intended recipient(s) and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or distribution is
prohibited. If you are not the intended recipient, please contact the
sender by reply e-mail and destroy all copies of the original message.</font></p><br></span></div></span> <br style="font-family:verdana,sans-serif"><br style="font-family: verdana,sans-serif;">
<br style="font-family: verdana,sans-serif;"><br style="font-family: verdana,sans-serif;"><div style="font-family: verdana,sans-serif;" class="gmail_quote">On Sat, May 21, 2011 at 3:14 PM, Siddhartha Jonnalagadda <span dir="ltr"><<a href="mailto:sid.kgp@gmail.com">sid.kgp@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">I have a single threaded java (NLP) application that processes 1000 sentences in 1 hour. I obviously can't wait for 1000 hours to process million sentences. Are there any simple instructions to make my program run in 100 servers at a time? This involves migrating the project workspace into each of them (or create them from a snapshot that contains it) and concatenate the output that each server produces.<br>
<br>Any quick pointers, please? I spent couple of hours browsing through Amazon MapReduce documentation, but that didn't take me as far...<br><br>Since I don't own shares in Amazon, I am open to non-Amazon solutions too.<br>
<br clear="all">Sincerely,<br>Siddhartha Jonnalagadda, <br>
Text mining Researcher, Lnx Research, LLC, Orange, CA<br><a href="http://sjonnalagadda.wordpress.com" target="_blank">sjonnalagadda.wordpress.com</a><br>
<br><span style="border-collapse: separate; border-spacing: 0px; color: rgb(0, 0, 0); font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><div style="word-wrap:break-word">
<span style="border-collapse:separate;border-spacing:0px;color:rgb(0, 0, 0);font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><p style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;line-height:normal;font-size-adjust:none;font-stretch:normal;min-height:14px">
<font size="1"><br></font></p><p style="margin: 0px;"><font style="font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;line-height:normal;font-size-adjust:none;font-stretch:normal" size="1">Confidentiality Notice:</font></p>
<p style="margin: 0px;"><font style="font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;line-height:normal;font-size-adjust:none;font-stretch:normal" size="1">This
e-mail message, including any attachments, is for the sole use of the
intended recipient(s) and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or distribution is
prohibited. If you are not the intended recipient, please contact the
sender by reply e-mail and destroy all copies of the original message.</font></p><br></span></div></span> <br><br>
</blockquote></div><br style="font-family: verdana,sans-serif;">