[Corpora-List] Simple instructions to scale a java application?

[Miles Osborne]

There really is no need to reinvent the wheel.  If you want to easily
scale your task then just use Hadoop.  Installing it is easy.  Look at
the "streaming" interface which will allow you to call your code
directly, without any special libraries etc.

To give you a feel for how easy it is, this would be the command
(assuming your job is called "parser" and you have loaded your data
onto Hadoop already)

hadoop jar /usr/local/share/hadoop/contrib/streaming/hadoop-*-streaming.jar
-mapper parser  -input myData/* -output myDataOut -file parser -
-numReduceTasks 0

and that is it.  not hard as you can see.

(home brew approaches are not robust and that is the real magic behind
map Reduce)

Miles

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora