[BangPypers] Help in understanding Hadoop with python

JAGANADH G jaganadhg at gmail.com
Thu Oct 18 04:57:13 CEST 2012

On Wed, Oct 10, 2012 at 11:41 AM, vijay <vnbang2003 at yahoo.com> wrote:

> Hi ,
>    I am planing to use hadoop for processing millions record text file for
> my project .Am looking for suggestion on hadoop with python programs.
>    If any one has used do let me know.


You have following options in Hadoop + Python
1) Use of Hadoop streaming API where you can write Map and Reduce programs
and stream it
2) Use disco or dumbo projects which has a pure python API for performing
Hadoop Map Reduce
3) Use Pydoop (http://pydoop.sourceforge.net/docs/) which is a Python API
for working with Hadoop and Python

Best regards


More information about the BangPypers mailing list