From phil.nabble at hanjinet.org Thu Mar 3 00:21:28 2011 From: phil.nabble at hanjinet.org (Philippe May) Date: Thu, 3 Mar 2011 04:51:28 +0530 Subject: [BangPypers] Introducing a "new" project Message-ID: <4BE861AD-4DBE-492B-91E8-5FAC019F9353@hanjinet.org> Hi all, I'm a long time reader of the list, rare poster. I'd like to introduce an open source project that i had for quite some time, and might interest those of you who are into networking, mobility, user content, localization, cloud, web, or in crazy ideas in general :) The main idea behind Hanji (that's the name of the project) is: embedding a web server framework in all kinds of devices. So, it sits in a category somewhere between desktop and web applications. It's built around Django and Twisted. Surprisingly (?) they all play quite well together. For the client side, Dojo. There are quite a few concepts involved, so instead of me trying to describe them all here, you might want to check the few links at http://hanji.info/ . Basically i'd be happy to get your feedback, comments, start discussions around it, and of course would warmly welcome participations since there's far too much work for a single person :) . The source is available online, no platform packages available - yet. For the patient ones, the installation documentation is online (http://hanji.info/doc/development/install.html). I could present it in a group meeting, just let me know. Besides, i also have 2 questions in relation to the current progress: - switching to NoSQL: any experience for a distributed database (nodes having a subset of the global database, with optional replication between nodes)? - anyone in the group has experience in python development and packaging on mobile platforms like Android, Symbian, iOS? Thanks, Philippe From anandology at gmail.com Thu Mar 3 04:34:25 2011 From: anandology at gmail.com (Anand Chitipothu) Date: Thu, 3 Mar 2011 09:04:25 +0530 Subject: [BangPypers] Introducing a "new" project In-Reply-To: <4BE861AD-4DBE-492B-91E8-5FAC019F9353@hanjinet.org> References: <4BE861AD-4DBE-492B-91E8-5FAC019F9353@hanjinet.org> Message-ID: [...] > Besides, i also have 2 questions in relation to the current progress: > - switching to NoSQL: any experience for a distributed database (nodes having a subset of the global database, with optional replication between nodes)? If you want to implement partial replication, CouchDB might be a good choice. Here is a case study using CouchDB replication for replicating clinical information from rural areas with unreliable internet connection to a central server. http://www.couchone.com/case-study-dimagi Anand From pythonic at gmail.com Thu Mar 3 05:07:51 2011 From: pythonic at gmail.com (Shekhar Tiwatne) Date: Thu, 03 Mar 2011 09:37:51 +0530 Subject: [BangPypers] Introducing a "new" project In-Reply-To: <4BE861AD-4DBE-492B-91E8-5FAC019F9353@hanjinet.org> References: <4BE861AD-4DBE-492B-91E8-5FAC019F9353@hanjinet.org> Message-ID: <4D6F1417.30504@gmail.com> On Thursday 03 March 2011 04:51 AM, Philippe May wrote: > [..] > The main idea behind Hanji (that's the name of the project) is: Nice name :-). > embedding a web server framework in all kinds of devices. So, it sits in a category somewhere between desktop and web applications. It's built around Django and Twisted. Wish somewhere in docs you will mention, justification of choosing these components. Especially to answer questions like why not flask instead of Django , why Twisted, why not just Twisted. > [..] Best, Shekhar From orsenthil at gmail.com Thu Mar 3 05:49:47 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 3 Mar 2011 12:49:47 +0800 Subject: [BangPypers] Introducing a "new" project In-Reply-To: <4D6F1417.30504@gmail.com> References: <4BE861AD-4DBE-492B-91E8-5FAC019F9353@hanjinet.org> <4D6F1417.30504@gmail.com> Message-ID: <20110303044947.GA2647@kevin> On Thu, Mar 03, 2011 at 09:37:51AM +0530, Shekhar Tiwatne wrote: > Wish somewhere in docs you will mention, justification of choosing > these components. Especially to answer questions like why not flask > instead of Django , why Twisted, why not just Twisted. If you think from the overall perspective of 'what service' the product is going to provide, I don't think it matters. People are interested in value and won't much care about how you provide it. Ofcourse, as developers, sometimes after trying out few things we may switch one buzz-word for another due to certain advantages, but most often we stick to what can be provided easily and the you are (developer) comfortable with or you can easily learn and implement. Tried the demo, it is good. Please don't have many TODO markers in your docs. -- Senthil From delegbede at dudupay.com Thu Mar 3 07:58:27 2011 From: delegbede at dudupay.com (delegbede at dudupay.com) Date: Thu, 3 Mar 2011 06:58:27 +0000 Subject: [BangPypers] Introducing a "new" project In-Reply-To: <4BE861AD-4DBE-492B-91E8-5FAC019F9353@hanjinet.org> References: <4BE861AD-4DBE-492B-91E8-5FAC019F9353@hanjinet.org> Message-ID: <1846158284-1299135521-cardhu_decombobulator_blackberry.rim.net-514312771-@b18.c12.bise7.blackberry> I have done a little of python development on symbian. Let me know what exactly the needs are. Best regards, Sent from my BlackBerry wireless device from MTN -----Original Message----- From: Philippe May Sender: bangpypers-bounces+delegbede=dudupay.com at python.org Date: Thu, 3 Mar 2011 04:51:28 To: Reply-To: Bangalore Python Users Group - India Subject: [BangPypers] Introducing a "new" project Hi all, I'm a long time reader of the list, rare poster. I'd like to introduce an open source project that i had for quite some time, and might interest those of you who are into networking, mobility, user content, localization, cloud, web, or in crazy ideas in general :) The main idea behind Hanji (that's the name of the project) is: embedding a web server framework in all kinds of devices. So, it sits in a category somewhere between desktop and web applications. It's built around Django and Twisted. Surprisingly (?) they all play quite well together. For the client side, Dojo. There are quite a few concepts involved, so instead of me trying to describe them all here, you might want to check the few links at http://hanji.info/ . Basically i'd be happy to get your feedback, comments, start discussions around it, and of course would warmly welcome participations since there's far too much work for a single person :) . The source is available online, no platform packages available - yet. For the patient ones, the installation documentation is online (http://hanji.info/doc/development/install.html). I could present it in a group meeting, just let me know. Besides, i also have 2 questions in relation to the current progress: - switching to NoSQL: any experience for a distributed database (nodes having a subset of the global database, with optional replication between nodes)? - anyone in the group has experience in python development and packaging on mobile platforms like Android, Symbian, iOS? Thanks, Philippe _______________________________________________ BangPypers mailing list BangPypers at python.org http://mail.python.org/mailman/listinfo/bangpypers From python.user.01 at gmail.com Thu Mar 3 08:28:48 2011 From: python.user.01 at gmail.com (Python User) Date: Thu, 3 Mar 2011 12:58:48 +0530 Subject: [BangPypers] Python Script to communicate to Application Servers Message-ID: Hi All, I am new user to python, I am interested in knowing how to use python script to communicate with Application server. All I need is Establish connection with an Application server, Send a Request XML and get the Response XML. Please suggest me what are the python services which I need to use for performing the above task. It would be great if you could also provide me some help links or documentation links where I can get more information on the same. Thanks In Advance From noufal at gmail.com Thu Mar 3 10:21:31 2011 From: noufal at gmail.com (Noufal Ibrahim) Date: Thu, 03 Mar 2011 14:51:31 +0530 Subject: [BangPypers] Python Script to communicate to Application Servers In-Reply-To: (Python User's message of "Thu, 3 Mar 2011 12:58:48 +0530") References: Message-ID: <87aahcimp0.fsf@gmail.com> On Thu, Mar 03 2011, Python User wrote: > Hi All, > > I am new user to python, I am interested in knowing how to use python script > to communicate with Application server. All I need is Establish connection > with an Application server, Send a Request XML and get the Response XML. Probably the urllib, urllib2, elementree modules if you want to go raw or xmlrpclib if you want to use XML-RPC. All of these are in the standard library. > Please suggest me what are the python services which I need to use for > performing the above task. It would be great if you could also provide > me some help links or documentation links where I can get more > information on the same. http://docs.python.org/ but these were just a google away. [...] -- ~noufal http://nibrahim.net.in From phil.nabble at hanjinet.org Thu Mar 3 18:29:34 2011 From: phil.nabble at hanjinet.org (Philippe May) Date: Thu, 3 Mar 2011 22:59:34 +0530 Subject: [BangPypers] Introducing a "new" project Message-ID: <24EE2AA5-A01A-4918-B149-725CC19B501A@hanjinet.org> Hi, Thanks Anand for the link, very pertinent. CouchDB seems a good choice for this case thanks to advanced replication, and is gaining popularity quite fast. Unfortunately i fear porting it to some platforms is a real problem (i don't even hope to compile Erlang code for Symbian...). Another option can be MongoDB, for its performances. It is used by the diaspora project (ruby), which shares some similarities with Hanji. I eventually also thought of the good old ZODB using something like ZEO, but i fear the resource usage is just too bad for mobile platforms. A pure Python solution would be great though. Btw I didn't mention the other main reason for stepping out of relational databases: flexibility of the schema. Hanji's applications can create database structures and should be distributed across the devices, and managing possibly different versions can become a complete hell quite fast. I think a solution like CouchDB will help. Any thoughts on that? Philippe > > If you want to implement partial replication, CouchDB might be a good choice. > > Here is a case study using CouchDB replication for replicating > clinical information from rural areas with unreliable internet > connection to a central server. > > http://www.couchone.com/case-study-dimagi > > Anand > From phil.nabble at hanjinet.org Thu Mar 3 19:12:10 2011 From: phil.nabble at hanjinet.org (Philippe May) Date: Thu, 3 Mar 2011 23:42:10 +0530 Subject: [BangPypers] Introducing a "new" project In-Reply-To: References: Message-ID: <8413DC37-E473-4024-80D4-DAC1A54B0207@hanjinet.org> Hi Shekhar, >> [..] >> The main idea behind Hanji (that's the name of the project) is: > Nice name :-). Thanks, lots of research behind that name ;) . Also, staying in Bangalore the very little of Hindi i knew almost vanished :) . >> embedding a web server framework in all kinds of devices. So, it sits in a category somewhere between desktop and web applications. It's built around Django and Twisted. > Wish somewhere in docs you will mention, justification of choosing these > components. Especially to answer questions like why not flask instead of > Django , why Twisted, why not just Twisted. >> [..] > Best, > Shekhar Ok, i'll develop in the documentation. I started with Django only. When more networking functionalities were added (websockets, peer-to-peer, replication), the threads and request/response model reached its limits and Twisted was the straightforward choice: robustness, portability, flexibility, support for many protocols, all that in a single Python package. New promising projects appeared recently that were not available then (greenlets, Flask, etc), and might be valuable additions if they can fit. Switching completely to them require strong arguments though. As for keeping Django: one main reason was for its nice OO-RDBMS mapper, and with the plans for dropping SQL it could become optional. But it is a good drive in Python community and has many applications we can reuse. Cheers, Philippe From phil.nabble at hanjinet.org Thu Mar 3 19:46:33 2011 From: phil.nabble at hanjinet.org (Philippe May) Date: Fri, 4 Mar 2011 00:16:33 +0530 Subject: [BangPypers] Introducing a "new" project In-Reply-To: References: Message-ID: Hi Senthil, >> >> Wish somewhere in docs you will mention, justification of choosing >> these components. Especially to answer questions like why not flask >> instead of Django , why Twisted, why not just Twisted. > > If you think from the overall perspective of 'what service' the > product is going to provide, I don't think it matters. > > People are interested in value and won't much care about how you > provide it. Ofcourse, as developers, sometimes after trying out few > things we may switch one buzz-word for another due to certain > advantages, but most often we stick to what can be provided easily and > the you are (developer) comfortable with or you can easily learn and > implement. True. For applications not requiring "special" networking capabilities, Hanji is almost the same than the plain Django so many developers will feel comfortable. > Tried the demo, it is good. Please don't have many TODO markers in > your docs. > > -- > Senthil Thanks. Hope to add more apps soon, demonstrating use cases (for example: chat, file sharing, photo albums, music streaming). The "map" applet shows another side project related to mobility and telcos, that might be a good drive for Hanji. Will clean up the doc. Philippe From phil.nabble at hanjinet.org Thu Mar 3 20:15:12 2011 From: phil.nabble at hanjinet.org (Philippe May) Date: Fri, 4 Mar 2011 00:45:12 +0530 Subject: [BangPypers] Introducing a "new" project In-Reply-To: References: Message-ID: <30F1D654-97C4-4C54-BF03-93582483A71B@hanjinet.org> Hi, > I have done a little of python development on symbian. Let me know what exactly the needs are. > Best regards, Ideally: create a complete standard SIS package for regular users, and update it easily. Symbian might have got a big kick recently, it will stay as a big player for few years. I had long time back a look at PyS60 and the packaging process looks not so easy, any help would be warmly welcome. So my questions: - A standard setup.py is good with Symbian, or additional effort is required? - Non pure Python packages: additional effort for getting C cross compiled (eg Twisted)? - Any chance for getting non-C software (CouchDB:Erlang) included in the package or as dependency? - Did you get help from the guys at OVI (eg package signing, general support)? If yes, any pointer to good people there? - Most importantly, an overall estimate of the effort, in man*day? Thanks, Philippe From wah_meng at yahoo.com Tue Mar 8 05:13:50 2011 From: wah_meng at yahoo.com (Wah Meng Wong) Date: Mon, 7 Mar 2011 20:13:50 -0800 (PST) Subject: [BangPypers] ORA-12705: Cannot access NLS data files or invalid environment specified In-Reply-To: References: Message-ID: <229189.9115.qm@web30405.mail.mud.yahoo.com> Hello there, My oracledb module on HP-UX is unable to connect to database when I export environment variable NLS_LANG="SIMPLIFIED CHINESE_CHINA.ZHS16GBK". There is no issue when such environment variable is exported and the SQLPLUS is issued. The DBA determined that this is an application error. Does anyone know this might be related to the some national language library not properly installed on the server? I noticed that there is no $ORACLE_HOME/ocommon folder on my server where Oracle installed is 11g, and I googled that environemnt variable ORA_NLS33 by default is pointing to $ORACLE_HOME/ocommon/nls/admin/data. Or ORA_NLS33 is no longer in use for Oracle 11g? $ python Python 1.5.2 (#9, May 23 2000, 14:18:02) [C] on hp-uxB Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import oracledb >>> a=oracledb.oracledb("gen812ora8/gen812db1ora8 at DB1ORA9") Traceback (innermost last): File "", line 1, in ? OracleDbError: LOGON caused a ORA-12705: Cannot access NLS data files or invalid environment specified >>> $ uname -a HP-UX zmy02hp3 B.11.31 U ia64 3240906838 unlimited-user license $ sqlplus gen812ora8/gen812db1ora8 at DB1ORA9 SQL*Plus: Release 11.1.0.7.0 - Production on ?????? 3?? 8 12:10:24 2011 Copyright (c) 1982, 2008, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production With the Data Mining option With the Data Mining option SQL> Regards, Wah Meng From lawgon at gmail.com Tue Mar 8 12:43:27 2011 From: lawgon at gmail.com (Kenneth Gonsalves) Date: Tue, 08 Mar 2011 17:13:27 +0530 Subject: [BangPypers] unittest failing Message-ID: <1299584607.2368.102.camel@localhost> hi, I am trying to figure out unittests on exceptions. I have a test which should pass - but it is failing. The simplest possible code is here: #!/usr/bin/env python import unittest def testfunc(): if 1: raise TypeError class Testfunc(unittest.TestCase): def testit(self): self.assertRaises(TypeError,testfunc()) if __name__ == '__main__': unittest.main() -- regards Kenneth Gonsalves http://lawgon.livejournal.com/ From orsenthil at gmail.com Tue Mar 8 12:52:00 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Tue, 8 Mar 2011 19:52:00 +0800 Subject: [BangPypers] unittest failing In-Reply-To: <1299584607.2368.102.camel@localhost> References: <1299584607.2368.102.camel@localhost> Message-ID: On Tue, Mar 8, 2011 at 7:43 PM, Kenneth Gonsalves wrote: > ? ? ? ?self.assertRaises(TypeError,testfunc()) Not to call testfunc, but just exercise. self.assertRaises(TypeError,testfunc) unittest framework takes care of calling and evaluating. -- Senthil From noufal at gmail.com Tue Mar 8 12:54:50 2011 From: noufal at gmail.com (Noufal Ibrahim) Date: Tue, 08 Mar 2011 17:24:50 +0530 Subject: [BangPypers] unittest failing In-Reply-To: <1299584607.2368.102.camel@localhost> (Kenneth Gonsalves's message of "Tue, 08 Mar 2011 17:13:27 +0530") References: <1299584607.2368.102.camel@localhost> Message-ID: <8739mxj08l.fsf@gmail.com> On Tue, Mar 08 2011, Kenneth Gonsalves wrote: > hi, > > I am trying to figure out unittests on exceptions. I have a test which > should pass - but it is failing. The simplest possible code is here: > > #!/usr/bin/env python > import unittest > def testfunc(): > if 1: > raise TypeError You should raise raise an instance of TypeError. Not the class itself. > > class Testfunc(unittest.TestCase): > def testit(self): > self.assertRaises(TypeError,testfunc()) You should say self.assertRaises(TypeError, testfunc) i.e. without the (). assertRaises will do the invocation. > if __name__ == '__main__': > unittest.main() I'd recommend that you use a harness like py.test (which I highly recommend) or nose to run your tests rather than do it raw like this. -- From lawgon at gmail.com Tue Mar 8 12:56:48 2011 From: lawgon at gmail.com (Kenneth Gonsalves) Date: Tue, 08 Mar 2011 17:26:48 +0530 Subject: [BangPypers] unittest failing In-Reply-To: References: <1299584607.2368.102.camel@localhost> Message-ID: <1299585408.2368.103.camel@localhost> On Tue, 2011-03-08 at 19:52 +0800, Senthil Kumaran wrote: > On Tue, Mar 8, 2011 at 7:43 PM, Kenneth Gonsalves > wrote: > > self.assertRaises(TypeError,testfunc()) > > Not to call testfunc, but just exercise. > > self.assertRaises(TypeError,testfunc) > > unittest framework takes care of calling and evaluating. thanks - I *knew* that, but I was debugging someone else's code and forgot -- regards Kenneth Gonsalves http://lawgon.livejournal.com/ From navin.kabra at gmail.com Thu Mar 10 04:44:34 2011 From: navin.kabra at gmail.com (Navin Kabra) Date: Thu, 10 Mar 2011 09:14:34 +0530 Subject: [BangPypers] Proposed python module: simpleoauth Message-ID: Hi All, Need your help on this. After getting very confused and frustrated at the state of OAuth libraries in python, I decided that there needs to be a simpler way of doing OAuth in Python. As a proof-of-concept I've implemented the following: https://github.com/ngkabra/simpleoauth The design goals were: - Simple interface for the common cases, at the expense of flexibility - Should work with OAuth v1 or OAuth v2 with the same or similar interface - Should be easy to incorporate into Django At this stage, this code is neither complete nor is it well tested. It is just a proof-of-concept that works for basic cases (I've tried it with facebook and twitter). Help needed: - Feedback on whether this seems useful. Note: this is not targeted towards OAuth ninjas, and is not appropriate for complex OAuth implementations. Just a quick way to access basic OAuth based APIs like facebook, twitter, yammer, delicious. - If it seems useful, then help needed in completing it - with finishing the functionality, packaging, testing etc. Let me know what you think. Thanks. From santrajan at gmail.com Thu Mar 10 06:53:55 2011 From: santrajan at gmail.com (Santosh Rajan) Date: Thu, 10 Mar 2011 11:23:55 +0530 Subject: [BangPypers] Proposed python module: simpleoauth In-Reply-To: References: Message-ID: Good work. The original author of OAuth, Eran Hammer Lahav, had to go and bail out Twitter, back in 2009. The problem with this identity thing is that, there are only two ways to solve it. 1) Federated Identity. 2) Delegated Identity. You are doing Delegated Identity. Which has its its own problems. Federated Identity has the "PKI Authority" problem. In either case you are completely fucked up. Just doesn't work. On Thu, Mar 10, 2011 at 9:14 AM, Navin Kabra wrote: > Hi All, > > Need your help on this. > > After getting very confused and frustrated at the state of OAuth libraries > in python, I decided that there needs to be a simpler way of doing OAuth in > Python. As a proof-of-concept I've implemented the following: > > https://github.com/ngkabra/simpleoauth > > The design goals were: > > > - Simple interface for the common cases, at the expense of flexibility > - Should work with OAuth v1 or OAuth v2 with the same or similar > interface > - Should be easy to incorporate into Django > > > At this stage, this code is neither complete nor is it well tested. It is > just a proof-of-concept that works for basic cases (I've tried it with > facebook and twitter). > > Help needed: > > - Feedback on whether this seems useful. Note: this is not targeted > towards OAuth ninjas, and is not appropriate for complex OAuth > implementations. Just a quick way to access basic OAuth based APIs like > facebook, twitter, yammer, delicious. > - If it seems useful, then help needed in completing it - with finishing > the functionality, packaging, testing etc. > > Let me know what you think. Thanks. > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- http://about.me/santosh.rajan From b.ghose at gmail.com Thu Mar 10 08:02:53 2011 From: b.ghose at gmail.com (Baishampayan Ghose) Date: Thu, 10 Mar 2011 12:32:53 +0530 Subject: [BangPypers] Proposed python module: simpleoauth In-Reply-To: References: Message-ID: > After getting very confused and frustrated at the state of OAuth libraries > in python, I decided that there needs to be a simpler way of doing OAuth in > Python. As a proof-of-concept I've implemented the following: > > https://github.com/ngkabra/simpleoauth Where is the code? simpleoauth/__init__.py seems to have only one line - "/home/navin/d/hacks/python/oauth/simple_oauth.py" Another thing, please sanitize the examples and remove your own API keys, they look real to me :) Regards, BG -- Baishampayan Ghose b.ghose at gmail.com From navin.kabra at gmail.com Thu Mar 10 09:03:51 2011 From: navin.kabra at gmail.com (Navin Kabra) Date: Thu, 10 Mar 2011 13:33:51 +0530 Subject: [BangPypers] Proposed python module: simpleoauth In-Reply-To: References: Message-ID: On Thu, Mar 10, 2011 at 12:32 PM, Baishampayan Ghose wrote: > Where is the code? simpleoauth/__init__.py seems to have only one line > - "/home/navin/d/hacks/python/oauth/simple_oauth.py" > Aargh... sorry. Checked in from the wrong directory. It should be fixed now. (Deleted and re-created the repo, so if you had cloned earlier you'll want to delete it and reclone the whole thing.) Thanks for the heads up BG. navin. From nitin.nitp at gmail.com Mon Mar 14 09:26:53 2011 From: nitin.nitp at gmail.com (Nitin Kumar) Date: Mon, 14 Mar 2011 13:56:53 +0530 Subject: [BangPypers] issues using constructor in unittest Message-ID: Hi All, See below small example. I am trying to have one constructor for a class with unittest inherited. But with any of the permutation combination I am getting the error. Any idea for reason behind this?? import unittest2 class XMLSupport(unittest2.TestCase): def __init__(self): *unittest2.TestCase.__init__(self) #with or without this line we get the error* self.x = 3 def test_3(self): print (self.x) self.x += 1 def test_4(self): print (self.x) Traceback (most recent call last): File "testMain.py", line 212, in main() File "testMain.py", line 164, in main suites=unittest2.defaultTestLoader.discover(stdir, pattern) File "c:\Python26\lib\site-packages\unittest2-0.5.1-py2.6.egg\unittest2\loader .py", line 224, in discover tests = list(self._find_tests(start_dir, pattern)) File "c:\Python26\lib\site-packages\unittest2-0.5.1-py2.6.egg\unittest2\loader .py", line 274, in _find_tests yield self.loadTestsFromModule(module) File "c:\Python26\lib\site-packages\unittest2-0.5.1-py2.6.egg\unittest2\loader .py", line 85, in loadTestsFromModule tests.append(self.loadTestsFromTestCase(obj)) File "c:\Python26\lib\site-packages\unittest2-0.5.1-py2.6.egg\unittest2\loader .py", line 76, in loadTestsFromTestCase loaded_suite = self.suiteClass(map(testCaseClass, testCaseNames)) *TypeError: __init__() takes exactly 1 argument (2 given)* -- Nitin K From navin.kabra at gmail.com Mon Mar 14 09:54:59 2011 From: navin.kabra at gmail.com (Navin Kabra) Date: Mon, 14 Mar 2011 14:24:59 +0530 Subject: [BangPypers] issues using constructor in unittest In-Reply-To: References: Message-ID: On Mon, Mar 14, 2011 at 1:56 PM, Nitin Kumar wrote: > See below small example. I am trying to have one constructor for a class > with unittest inherited. Avoid overriding the constructor of TestCase. That is not recommended. > self.x = 3 Put this is "setUp" instead of the constructor *unittest2.TestCase.__init__(self) > If you must do this, the correct way of doing it is: def __init__(self, *args, **kwargs): super(XMLSupport, self).__init__(*args, **kwargs) self.x = 3 From nitin.nitp at gmail.com Mon Mar 14 10:30:01 2011 From: nitin.nitp at gmail.com (Nitin Kumar) Date: Mon, 14 Mar 2011 15:00:01 +0530 Subject: [BangPypers] issues using constructor in unittest In-Reply-To: References: Message-ID: Hi Navin, I can't use setUp as this will be called every time for each test cases, I thought of using setUpClass decorator but there i can define self. import unittest2 class XMLSupport(unittest2.TestCase): def __init__(self): super(XMLSupport,self).__init__(*args, **kwargs) self.x = 3 def test_3(self): print (self.x) self.x += 1 def test_4(self): print (self.x) I used this way as explained by you, still getting the same error. Thanks Nitin K On Mon, Mar 14, 2011 at 2:24 PM, Navin Kabra wrote: > On Mon, Mar 14, 2011 at 1:56 PM, Nitin Kumar wrote: > > > See below small example. I am trying to have one constructor for a class > > with unittest inherited. > > Avoid overriding the constructor of TestCase. That is not recommended. > > > > > self.x = 3 > > > Put this is "setUp" instead of the constructor > > *unittest2.TestCase.__init__(self) > > > > If you must do this, the correct way of doing it is: > > def __init__(self, *args, **kwargs): > super(XMLSupport, self).__init__(*args, **kwargs) > self.x = 3 > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- Nitin K From orsenthil at gmail.com Mon Mar 14 10:46:45 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Mon, 14 Mar 2011 17:46:45 +0800 Subject: [BangPypers] issues using constructor in unittest In-Reply-To: References: Message-ID: <20110314094645.GC4388@kevin> On Mon, Mar 14, 2011 at 03:00:01PM +0530, Nitin Kumar wrote: > Hi Navin, > I can't use setUp as this will be called every time for each test cases, I > thought of using setUpClass decorator but there i can define self. > > import unittest2 > class XMLSupport(unittest2.TestCase): > def __init__(self): > super(XMLSupport,self).__init__(*args, **kwargs) > self.x = 3 > You don't have constructor for unittest class. Because of the way unittest framework works, (namely creates testobjects for its purposes), creating an object is not recommended. I have not seen any code doing it. For what you are trying to do, you could do something like this: (Follow along and modify as per your requirement) import unittest x = 100 class XMLSupport(unittest.TestCase): def setUp(self): print 'In setup' self.x = 3 def test_3(self): print(self.x) self.x += 1 global x x += 1 def test_4(self): print(self.x) print(x) if __name__ == '__main__': unittest.main() From rajeev.sebastian at gmail.com Mon Mar 14 11:38:17 2011 From: rajeev.sebastian at gmail.com (Rajeev J Sebastian) Date: Mon, 14 Mar 2011 16:08:17 +0530 Subject: [BangPypers] [OT][JOBS] Interview/Test for Python/Django candidates Message-ID: Hi, I'm writing on behalf of Alokin Software Pvt Ltd based at Technopark Trivandrum. We are looking for some candidates with strong skills in Python, Django development on a Linux platform. Preferable additional skills: frontend stuff (HTML/CSS/Javascript), knowledge of system administration tasks (deployment, security, VPS), project management skills. If you have the skills that we're looking for, send an email to hr at alokin.in and attach your resume. Kindly pass this on to others who might be looking for jobs in this area. I'm available by email and phone for your questions. Thanks Regards Rajeev J Sebastian rajeev at alokin.in +91-9846088898 From navin.kabra at gmail.com Mon Mar 14 11:57:50 2011 From: navin.kabra at gmail.com (Navin Kabra) Date: Mon, 14 Mar 2011 16:27:50 +0530 Subject: [BangPypers] issues using constructor in unittest In-Reply-To: References: Message-ID: On Mon, Mar 14, 2011 at 3:00 PM, Nitin Kumar wrote: > I used this way as explained by you, still getting the same error. > What version of python are you using? Here is the code I tried and it works fine with python 2.6.5 and unittest2 0.5.1. #!/usr/bin/env python import unittest2 class TestIt(unittest2.TestCase): def __init__(self, *args, **kwargs): super(TestIt, self).__init__(*args, **kwargs) self.x = 3 def testit(self): self.assertEqual(self.x, 3) self.assertNotEqual(self.x, 2) if __name__ == '__main__': unittest2.main() From noufal at gmail.com Mon Mar 14 12:24:49 2011 From: noufal at gmail.com (Noufal Ibrahim) Date: Mon, 14 Mar 2011 04:24:49 -0700 Subject: [BangPypers] issues using constructor in unittest In-Reply-To: (Nitin Kumar's message of "Mon, 14 Mar 2011 15:00:01 +0530") References: Message-ID: <877hc2ymf2.fsf@gmail.com> On Mon, Mar 14 2011, Nitin Kumar wrote: [...] > I used this way as explained by you, still getting the same error. [...] Why don't you consider using an alternate test runner like nose or py.test? This whole XUnit style unit testing thing is not really necessary in the Python world. You just need functions that pass or fail (using assert) and some simple setup/teardown routines. -- From python.user.01 at gmail.com Mon Mar 14 12:51:07 2011 From: python.user.01 at gmail.com (Python User) Date: Mon, 14 Mar 2011 17:21:07 +0530 Subject: [BangPypers] plist files Message-ID: Hi All, I have a plist file, test.plist, All I want is read the scriptNO tag value 12345 and replace it with 67899. Can any one please help me out for this. The file content is command GetTestScriptData comments Get Test Script opParam scriptNO 12345 columnNames Test case fetchKey 1 fetchData 1 I was using the below code to read the plist file, but I am not able to search the scriptNo and replace it with other value. import plistlib file = ('/Users/me/Desktop/test.plist') plist = plistlib.readPlist(file) print plist Please help me out to solve this. Thanks In Advance. From orsenthil at gmail.com Mon Mar 14 12:56:56 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Mon, 14 Mar 2011 19:56:56 +0800 Subject: [BangPypers] issues using constructor in unittest In-Reply-To: <877hc2ymf2.fsf@gmail.com> References: <877hc2ymf2.fsf@gmail.com> Message-ID: <20110314115655.GA2556@kevin> On Mon, Mar 14, 2011 at 04:24:49AM -0700, Noufal Ibrahim wrote: > Why don't you consider using an alternate test runner like nose or > py.test? When I first started using with Linux, it was quite common to hear "Gentoo is great", "Debian the real distro", you should use that instead of Mandrake/Redhat. I used feel excited as well confused. :) But the best motivation for me was when someone really answered the question or provided a technical reasoning for one thing over the other. > This whole XUnit style unit testing thing is not really necessary in > the Python world. Why? It is very much part of python world. It is part of the stdlib, because it is so easy to use. There are easier frameworks like py.test which do a lot of metaprogramming magic and make the testing fun, but that does not count of removing xUnit style test, which can imbibe in you pattern to think about tests. -- Senthil From nitin.nitp at gmail.com Mon Mar 14 13:03:38 2011 From: nitin.nitp at gmail.com (Nitin Kumar) Date: Mon, 14 Mar 2011 17:33:38 +0530 Subject: [BangPypers] issues using constructor in unittest In-Reply-To: <877hc2ymf2.fsf@gmail.com> References: <877hc2ymf2.fsf@gmail.com> Message-ID: Agree Noufal, But we already have a framework using unittest2 in my product (organization), its working perfectly fine for our requirements. Nitin K On Mon, Mar 14, 2011 at 4:54 PM, Noufal Ibrahim wrote: > On Mon, Mar 14 2011, Nitin Kumar wrote: > > [...] > > > I used this way as explained by you, still getting the same error. > > [...] > > Why don't you consider using an alternate test runner like nose or > py.test? This whole XUnit style unit testing thing is not really > necessary in the Python world. You just need functions that pass or fail > (using assert) and some simple setup/teardown routines. > -- > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- Nitin K From noufal at gmail.com Mon Mar 14 13:32:16 2011 From: noufal at gmail.com (Noufal Ibrahim) Date: Mon, 14 Mar 2011 05:32:16 -0700 Subject: [BangPypers] issues using constructor in unittest In-Reply-To: <20110314115655.GA2556@kevin> (Senthil Kumaran's message of "Mon, 14 Mar 2011 19:56:56 +0800") References: <877hc2ymf2.fsf@gmail.com> <20110314115655.GA2556@kevin> Message-ID: <878vwhopbj.fsf@gmail.com> On Mon, Mar 14 2011, Senthil Kumaran wrote: [...] >> This whole XUnit style unit testing thing is not really necessary in >> the Python world. > > Why? It is very much part of python world. It is part of the stdlib, > because it is so easy to use. There are lots of libraries in the standard library that are not really pythonic and which are mirrors of a corresponding C library. > There are easier frameworks like py.test which do a lot of > metaprogramming magic and make the testing fun, but that does not > count of removing xUnit style test, which can imbibe in you pattern to > think about tests. It's probably just my personal preference but I'd like a few functions like. def test_foo(): x = do_foo() assert x == "some true value" in a file which I could just run rather than subclassing and doing all that which Xunit demands. Python is about less boilerplate and more simplicity isn't it? unittest2 is getting better but I think Xunit is a more a child of Smalltalk and Java than Python. Options are always a good thing though and YMMV. -- From orsenthil at gmail.com Mon Mar 14 13:55:51 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Mon, 14 Mar 2011 20:55:51 +0800 Subject: [BangPypers] issues using constructor in unittest In-Reply-To: <878vwhopbj.fsf@gmail.com> References: <877hc2ymf2.fsf@gmail.com> <20110314115655.GA2556@kevin> <878vwhopbj.fsf@gmail.com> Message-ID: <20110314125551.GB2556@kevin> On Mon, Mar 14, 2011 at 05:32:16AM -0700, Noufal Ibrahim wrote: > It's probably just my personal preference but I'd like a few > functions like. > Yeah, that's perfectly fine and +1 to that. -- Senthil From vinayshastry at gmail.com Mon Mar 14 20:51:05 2011 From: vinayshastry at gmail.com (Vinay Shastry) Date: Tue, 15 Mar 2011 01:21:05 +0530 Subject: [BangPypers] plist files In-Reply-To: References: Message-ID: On 14 March 2011 17:21, Python User wrote: > Hi All, > > I have a plist file, test.plist, All I want is read the scriptNO tag value > 12345 and replace it with 67899. Can any one please help me out for > this. The file content is > > > http://www.test.com/DTDs/PropertyList-1.0.dtd"> > > > > command > GetTestScriptData > comments > Get Test Script > opParam > > scriptNO > 12345 > columnNames > Test case > fetchKey > 1 > fetchData > 1 > > > > > > I was using the below code to read the plist file, but I am not able ?to > search the scriptNo and replace it with other value. > > import plistlib > file = ('/Users/me/Desktop/test.plist') > plist = plistlib.readPlist(file) > print plist > > Please help me out to solve this. Thanks In Advance. The answer is in your plist format itself. Everything is enclosed in an array - which becomes a list in py. How about something like this: for key in plist: if 'opParam' in key and 'scriptNO' in key['opParam']: key['opParam']['scriptNO'] = 67899 plistlib.writePlist(plist, "mynewfile.plist") Also, is there a reason why file is a tuple containing filename, and not just a string? -- Vinay S Shastry From vinayshastry at gmail.com Mon Mar 14 20:57:02 2011 From: vinayshastry at gmail.com (Vinay Shastry) Date: Tue, 15 Mar 2011 01:27:02 +0530 Subject: [BangPypers] plist files In-Reply-To: References: Message-ID: On 15 March 2011 01:21, Vinay Shastry wrote: > On 14 March 2011 17:21, Python User wrote: >> Hi All, >> >> I have a plist file, test.plist, All I want is read the scriptNO tag value >> 12345 and replace it with 67899. Can any one please help me out for >> this. The file content is >> >> >> > http://www.test.com/DTDs/PropertyList-1.0.dtd"> >> >> >> >> command >> GetTestScriptData >> comments >> Get Test Script >> opParam >> >> scriptNO >> 12345 >> columnNames >> Test case >> fetchKey >> 1 >> fetchData >> 1 >> >> >> >> >> >> I was using the below code to read the plist file, but I am not able ?to >> search the scriptNo and replace it with other value. >> >> import plistlib >> file = ('/Users/me/Desktop/test.plist') >> plist = plistlib.readPlist(file) >> print plist >> >> Please help me out to solve this. Thanks In Advance. > > The answer is in your plist format itself. Everything is enclosed in > an array - which becomes a list in py. > > How about something like this: > > for key in plist: > ? ?if 'opParam' in key and 'scriptNO' in key['opParam']: > ? ? ? ?key['opParam']['scriptNO'] = 67899 > > plistlib.writePlist(plist, "mynewfile.plist") > > > Also, is there a reason why file is a tuple containing filename, and > not just a string? > Oh, and your DTD is wrong. -- Vinay S Shastry From python.user.01 at gmail.com Tue Mar 15 14:48:26 2011 From: python.user.01 at gmail.com (Python User) Date: Tue, 15 Mar 2011 19:18:26 +0530 Subject: [BangPypers] plist files In-Reply-To: References: Message-ID: Hi All Again I am struck with an Issue if the plist file has multiple dictionary in it then I am not able to get the value of particular key element for example, In the below plist file I wanted to get the value of stepID & COMPONENTNAME key. I was trying like this for key in plist: if 'results' in key: stepIDValue = key['results']['Steps']['stepID'] I am getting error "string indices must be integers, not str". Please help me out PLIST FILE :::: commandStatus success comments Get Test Script results Keywords No keywords. Steps activeFlag 0 expectedResult expectedTime 0000:00:00 instructions Launch stepData Test APP stepID 7663365 stepNumber 1 stepPriority 5 summary title Launching SDM Application columns ACTIVE_FLAG 1 BUSINESS_CATEGORY CHANGE_MANAGEMENT_ID COMPONENTNAME Software Client COMPREVVERS Admin CREATION_DATE 15-Mar-2011 04:52:01 LAST_UPDATE_DATE 15-Mar-2011 06:06:52 PRIORITY 5 REGRESSION STATUS_ID 1 TEST_LEVEL TEST_REGION TEST_SCRIPT_TITLE Software_Auto TRACK_NAME On Tue, Mar 15, 2011 at 11:03 AM, Python User wrote: > Hi Vinay, > > Thanks a Lot it worked for me :) > > Thanks Once Again.... > > > On Tue, Mar 15, 2011 at 1:27 AM, Vinay Shastry wrote: > >> On 15 March 2011 01:21, Vinay Shastry wrote: >> > On 14 March 2011 17:21, Python User wrote: >> >> Hi All, >> >> >> >> I have a plist file, test.plist, All I want is read the scriptNO tag >> value >> >> 12345 and replace it with 67899. Can any one please help me out for >> >> this. The file content is >> >> >> >> >> >> > >> http://www.test.com/DTDs/PropertyList-1.0.dtd"> >> >> >> >> >> >> >> >> command >> >> GetTestScriptData >> >> comments >> >> Get Test Script >> >> opParam >> >> >> >> scriptNO >> >> 12345 >> >> columnNames >> >> Test case >> >> fetchKey >> >> 1 >> >> fetchData >> >> 1 >> >> >> >> >> >> >> >> >> >> >> >> I was using the below code to read the plist file, but I am not able >> to >> >> search the scriptNo and replace it with other value. >> >> >> >> import plistlib >> >> file = ('/Users/me/Desktop/test.plist') >> >> plist = plistlib.readPlist(file) >> >> print plist >> >> >> >> Please help me out to solve this. Thanks In Advance. >> > >> > The answer is in your plist format itself. Everything is enclosed in >> > an array - which becomes a list in py. >> > >> > How about something like this: >> > >> > for key in plist: >> > if 'opParam' in key and 'scriptNO' in key['opParam']: >> > key['opParam']['scriptNO'] = 67899 >> > >> > plistlib.writePlist(plist, "mynewfile.plist") >> > >> > >> > Also, is there a reason why file is a tuple containing filename, and >> > not just a string? >> > >> >> Oh, and your DTD is wrong. >> >> -- >> Vinay S Shastry >> _______________________________________________ >> BangPypers mailing list >> BangPypers at python.org >> http://mail.python.org/mailman/listinfo/bangpypers >> > > From venkat83 at gmail.com Tue Mar 15 18:53:31 2011 From: venkat83 at gmail.com (Venkatraman S) Date: Tue, 15 Mar 2011 23:23:31 +0530 Subject: [BangPypers] [OT] Django Sprint Weekend Message-ID: Hi, Was wondering whether anyone who has intermediate to advanced level of Django knowledge would be interested in a paid code sprint over the weekend to refactor an open source project. I am looking for 2-3 developers; please do not reply if you are not sufficiently conversation with Django+Python. The existing code is a little prolix and needs to be broken down into chunks and dead code needs to be removed. I am not sure, but think i can pay a nominal amount or reimburse for the food/refreshments along with a complimentary gift :) Please contact me off-list. -Venkat PS:I havent tagged this commercial, as i think is under OSS arena. From santhosh.edukulla at gmail.com Wed Mar 16 18:57:24 2011 From: santhosh.edukulla at gmail.com (Santhosh Edukulla) Date: Wed, 16 Mar 2011 23:27:24 +0530 Subject: [BangPypers] Regular Expressions Message-ID: Hi All, I have a list of regular expressions under a file(EX: inp.txt). Then, I have another file (EX: search.txt).The task is that i have to read the list of regular expressions from the inp.txt and then search for the same in "search.txt" and output the matched string. Iam using python *"re" *module to do this. This works except for few regular expressions below. *a = '(/|\\)cmd\.com$'* 1. When i read the regular expression above from the file to a variable *(EX: a)*. as below EX: *a=open('inp.txt','r').readlines()[0]* Iam getting a different regular expresson as *'(/|\\\\)cmd\.com$'* 2. Instead when i use *a.encode('string-escape')*, iam getting the output for a as * '(/|\)cmd\.com$' *So when used like this, re.compile() statement gives an error Any idea to solve this. Thx, Santhosh Thx, Santhosh From lawgon at gmail.com Thu Mar 17 00:25:26 2011 From: lawgon at gmail.com (Kenneth Gonsalves) Date: Thu, 17 Mar 2011 04:55:26 +0530 Subject: [BangPypers] Regular Expressions In-Reply-To: References: Message-ID: <1300317926.2368.450.camel@localhost> On Wed, 2011-03-16 at 23:27 +0530, Santhosh Edukulla wrote: > 1. > When i read the regular expression above from the file to a variable > *(EX: > a)*. as below > EX: > *a=open('inp.txt','r').readlines()[0]* > > Iam getting a different regular expresson as *'(/|\\\\)cmd\.com$'* > > 2. Instead when i use *a.encode('string-escape')*, iam getting the > output > for a as * '(/|\)cmd\.com$' *So when used like this, re.compile() > statement > gives an error are the '*' characters in your mail part of the re? If not please reformat your mail without the '*' characters -- regards Kenneth Gonsalves http://lawgon.livejournal.com/ From orsenthil at gmail.com Thu Mar 17 03:36:48 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 17 Mar 2011 10:36:48 +0800 Subject: [BangPypers] Regular Expressions In-Reply-To: References: Message-ID: <20110317023648.GB3754@kevin> On Wed, Mar 16, 2011 at 11:27:24PM +0530, Santhosh Edukulla wrote: > a = '(/|\\)cmd\.com$' > 1. > When i read the regular expression above from the file to a variable *(EX: > a)*. as below > EX: > *a=open('inp.txt','r').readlines()[0]* > > Iam getting a different regular expresson as *'(/|\\\\)cmd\.com$'* I think, you were trying this in the interpreter and you were using the interpreter to check the regex output. I got tripped by it too, and had to take help from IRC. Look at this session: >>> f = open('inp.txt') >>> c = f.read() >>> c '(/|\\\\)cmd\\.com$\n' >>> # That is because it is doing a repr(c) >>> # But do a print, you will see >>> print c (/|\\)cmd\.com$ Does this help? -- Senthil From gora at mimirtech.com Thu Mar 17 07:06:58 2011 From: gora at mimirtech.com (Gora Mohanty) Date: Thu, 17 Mar 2011 11:36:58 +0530 Subject: [BangPypers] Regular Expressions In-Reply-To: References: Message-ID: On Wed, Mar 16, 2011 at 11:27 PM, Santhosh Edukulla wrote: > Hi All, > > I have a list of regular expressions under a file(EX: inp.txt). Then, I have > another file (EX: search.txt).The task is that i have to read the list of > regular expressions from the inp.txt and then search for the same in > "search.txt" and output the matched string. > > Iam using python *"re" *module to do this. > > This works except for few regular expressions below. > > *a = '(/|\\)cmd\.com$'* > > 1. > When i read the regular expression above from the file to a variable *(EX: > a)*. as below > EX: > *a=open('inp.txt','r').readlines()[0]* There seem to be several issues here: * The above will also give you an end-of-line character at the end of 'a'. You will need to remove that, e.g., one way is: import os a = open( 'inp.txt','r' ).readlines()[0] a = a.rstrip( os.linesep ) * If setting 'a' directly in the interpreter, the best way is to use a raw string: a = r'(/|\\)cmd\.com$' Then, both of re.match( a, '\cmd.com' ) re.match( a, '/cmd.com' ) give a match. * If reading from a file, with read()/readlines(), things should just work, i.e., if the file starts with the line: (/|\\)cmd\.com$ then, a = open( 'inp.txt','r' ).readlines()[0] a = a.rstrip( os.linesep ) should match '/cmd.com' and '\cmd.com' Regards, Gora From baiju.m.mail at gmail.com Sun Mar 20 18:30:51 2011 From: baiju.m.mail at gmail.com (Baiju M) Date: Sun, 20 Mar 2011 13:30:51 -0400 Subject: [BangPypers] GetPython3.net (Lightning talk at PyCON 2011, Atlanta) Message-ID: Hi All, Here is the video of my lightning talk at PyCON 2011, Atlanta. http://www.youtube.com/watch?v=UEMjz5A_M5o Thanks to all BangPypers who contributed to the site. Regards, Baiju M From agn82kv at gmail.com Mon Mar 21 11:16:27 2011 From: agn82kv at gmail.com (agnel varghese) Date: Mon, 21 Mar 2011 15:46:27 +0530 Subject: [BangPypers] ImportError: No module named win32com.shell In-Reply-To: References: Message-ID: Hi , I am trying to control a GUI based application through one python script .Getting below error while running it.Any ideas how to resolve it ImportError: No module named win32com.shell Thanks, Agnel. $ pythonw ssc.py scanlist -v Traceback (most recent call last): File "ssc.py", line 52, in import SSCConfig File "c:\ssc_auto\lib\SSCConfig.py" > > , line 103, in > import PathFinder > File "c:\ssc_auto\lib\PathFinder.py", line 11, in > from win32com.shell import shellcon, shell > ImportError: No module named win32com.shell > > From orsenthil at gmail.com Mon Mar 21 11:20:06 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Mon, 21 Mar 2011 18:20:06 +0800 Subject: [BangPypers] ImportError: No module named win32com.shell In-Reply-To: References: Message-ID: agnel varghese wrote: > .Getting below error while running it.Any ideas how to resolve it > ImportError: No module named win32com.shell Install http://sourceforge.net/projects/pywin32/ Or Python from ActiveState for your windows. -- Senthil From nitin.nitp at gmail.com Tue Mar 22 08:55:35 2011 From: nitin.nitp at gmail.com (Nitin Kumar) Date: Tue, 22 Mar 2011 13:25:35 +0530 Subject: [BangPypers] Regular Expressions In-Reply-To: References: Message-ID: use this way r'(/|\\)cmd\.com$' Its raw string stuff which i guess you are missing out. Nitin K On Wed, Mar 16, 2011 at 11:27 PM, Santhosh Edukulla < santhosh.edukulla at gmail.com> wrote: > Hi All, > > I have a list of regular expressions under a file(EX: inp.txt). Then, I > have > another file (EX: search.txt).The task is that i have to read the list of > regular expressions from the inp.txt and then search for the same in > "search.txt" and output the matched string. > > Iam using python *"re" *module to do this. > > This works except for few regular expressions below. > > *a = '(/|\\)cmd\.com$'* > > 1. > When i read the regular expression above from the file to a variable *(EX: > a)*. as below > EX: > *a=open('inp.txt','r').readlines()[0]* > > Iam getting a different regular expresson as *'(/|\\\\)cmd\.com$'* > > 2. Instead when i use *a.encode('string-escape')*, iam getting the output > for a as * '(/|\)cmd\.com$' *So when used like this, re.compile() statement > gives an error > > > Any idea to solve this. > > Thx, > Santhosh > > > Thx, > Santhosh > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- Nitin K From subhabangalore at gmail.com Tue Mar 22 13:34:51 2011 From: subhabangalore at gmail.com (Subhabrata Banerjee) Date: Tue, 22 Mar 2011 18:04:51 +0530 Subject: [BangPypers] Reference for BangPypers Digest, Vol 43, Issue 13 Message-ID: Hi, You can refer to Alan Gauld's Python Tutorial.(* http://www.freenetpages.co.uk/hp/alan.gauld/) * It has excellent tutorial on Regular Expression. Hope this would solve your problem. Best Regards, Subhabrata Banerjee On Tue, Mar 22, 2011 at 4:30 PM, wrote: > Send BangPypers mailing list submissions to > bangpypers at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.python.org/mailman/listinfo/bangpypers > or, via email, send a message with subject or body 'help' to > bangpypers-request at python.org > > You can reach the person managing the list at > bangpypers-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of BangPypers digest..." > > > Today's Topics: > > 1. Re: Regular Expressions (Nitin Kumar) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 22 Mar 2011 13:25:35 +0530 > From: Nitin Kumar > To: Bangalore Python Users Group - India > Subject: Re: [BangPypers] Regular Expressions > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1 > > use this way > > r'(/|\\)cmd\.com$' > > Its raw string stuff which i guess you are missing out. > > Nitin K > > On Wed, Mar 16, 2011 at 11:27 PM, Santhosh Edukulla < > santhosh.edukulla at gmail.com> wrote: > > > Hi All, > > > > I have a list of regular expressions under a file(EX: inp.txt). Then, I > > have > > another file (EX: search.txt).The task is that i have to read the list of > > regular expressions from the inp.txt and then search for the same in > > "search.txt" and output the matched string. > > > > Iam using python *"re" *module to do this. > > > > This works except for few regular expressions below. > > > > *a = '(/|\\)cmd\.com$'* > > > > 1. > > When i read the regular expression above from the file to a variable > *(EX: > > a)*. as below > > EX: > > *a=open('inp.txt','r').readlines()[0]* > > > > Iam getting a different regular expresson as *'(/|\\\\)cmd\.com$'* > > > > 2. Instead when i use *a.encode('string-escape')*, iam getting the output > > for a as * '(/|\)cmd\.com$' *So when used like this, re.compile() > statement > > gives an error > > > > > > Any idea to solve this. > > > > Thx, > > Santhosh > > > > > > Thx, > > Santhosh > > _______________________________________________ > > BangPypers mailing list > > BangPypers at python.org > > http://mail.python.org/mailman/listinfo/bangpypers > > > > > > -- > Nitin K > > > ------------------------------ > > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > > > End of BangPypers Digest, Vol 43, Issue 13 > ****************************************** > -- SUBHABRATA BANERJEE PDF(2007-2010)[IISc, Bangalore] Call:09873237945 From brijithp at gmail.com Tue Mar 22 16:09:50 2011 From: brijithp at gmail.com (brijithp at gmail.com) Date: Tue, 22 Mar 2011 20:39:50 +0530 Subject: [BangPypers] How to handle files efficiently in python Message-ID: Hi All, How can I print last five lines of a file using python. The file may contain thousands of lines each line may differ in length. * Regrads, Brijith P * * * From pradeep at btbytes.com Tue Mar 22 16:50:37 2011 From: pradeep at btbytes.com (Pradeep Gowda) Date: Tue, 22 Mar 2011 11:50:37 -0400 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: References: Message-ID: If you are on Linux/Unix, use the tail command $ tail -n 5 somefile.txt On Tue, Mar 22, 2011 at 11:09 AM, brijithp at gmail.com wrote: > Hi All, > ? ? ? ?How can I print last five lines of a file using python. The file may > contain thousands of lines each line may differ in length. > > > * > Regrads, > > Brijith P > * > * > * > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > From noufal at gmail.com Tue Mar 22 17:49:51 2011 From: noufal at gmail.com (Noufal Ibrahim) Date: Tue, 22 Mar 2011 09:49:51 -0700 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: (brijithp@gmail.com's message of "Tue, 22 Mar 2011 20:39:50 +0530") References: Message-ID: <87vczb2j80.fsf@gmail.com> On Tue, Mar 22 2011, brijithp at gmail.com wrote: > Hi All, > How can I print last five lines of a file using python. The file may > contain thousands of lines each line may differ in length. [...] You'd have to use some kind of heuristic (a.k.a. dirty hack). Stat the file to find it's size and use an "average" length of line to go a few lines back. Then count the number of newlines from there to the end. If it's 4, then you have your 5 lines. Otherwise, seek back a little more and repeat. -- From subhabangalore at gmail.com Tue Mar 22 18:08:39 2011 From: subhabangalore at gmail.com (Subhabrata Banerjee) Date: Tue, 22 Mar 2011 22:38:39 +0530 Subject: [BangPypers] Suggestions over BangPypers Digest, Vol 43, Issue 13 Message-ID: Sir, Over your question I sent a suggestion that you may kindly refer to Alan Gauld's python Tutorial in http://www.freenetpages.co.uk/hp/alan.gauld/. But the mail bounced. I sent the mail to bangpypers at python.org I have joined the mailing list quite new so I do not know exactly why it bounced. I felt you are looking for the answer so I gave you a copy. It has nothing to do other than the answer, so sorry for sending you a copy. But if you can kindly guide me how to answer in the forum I would be immensely helpful. Best Regards, Subhabrata Banerjee. On Tue, Mar 22, 2011 at 4:30 PM, wrote: > Send BangPypers mailing list submissions to > bangpypers at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.python.org/mailman/listinfo/bangpypers > or, via email, send a message with subject or body 'help' to > bangpypers-request at python.org > > You can reach the person managing the list at > bangpypers-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of BangPypers digest..." > > > Today's Topics: > > 1. Re: Regular Expressions (Nitin Kumar) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 22 Mar 2011 13:25:35 +0530 > From: Nitin Kumar > To: Bangalore Python Users Group - India > Subject: Re: [BangPypers] Regular Expressions > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1 > > use this way > > r'(/|\\)cmd\.com$' > > Its raw string stuff which i guess you are missing out. > > Nitin K > > On Wed, Mar 16, 2011 at 11:27 PM, Santhosh Edukulla < > santhosh.edukulla at gmail.com> wrote: > > > Hi All, > > > > I have a list of regular expressions under a file(EX: inp.txt). Then, I > > have > > another file (EX: search.txt).The task is that i have to read the list of > > regular expressions from the inp.txt and then search for the same in > > "search.txt" and output the matched string. > > > > Iam using python *"re" *module to do this. > > > > This works except for few regular expressions below. > > > > *a = '(/|\\)cmd\.com$'* > > > > 1. > > When i read the regular expression above from the file to a variable > *(EX: > > a)*. as below > > EX: > > *a=open('inp.txt','r').readlines()[0]* > > > > Iam getting a different regular expresson as *'(/|\\\\)cmd\.com$'* > > > > 2. Instead when i use *a.encode('string-escape')*, iam getting the output > > for a as * '(/|\)cmd\.com$' *So when used like this, re.compile() > statement > > gives an error > > > > > > Any idea to solve this. > > > > Thx, > > Santhosh > > > > > > Thx, > > Santhosh > > _______________________________________________ > > BangPypers mailing list > > BangPypers at python.org > > http://mail.python.org/mailman/listinfo/bangpypers > > > > > > -- > Nitin K > > > ------------------------------ > > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > > > End of BangPypers Digest, Vol 43, Issue 13 > ****************************************** > -- SUBHABRATA BANERJEE PDF(2007-2010)[IISc, Bangalore] Call:09873237945 From brijithp at gmail.com Wed Mar 23 05:45:35 2011 From: brijithp at gmail.com (brijithp at gmail.com) Date: Wed, 23 Mar 2011 10:15:35 +0530 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: <87vczb2j80.fsf@gmail.com> References: <87vczb2j80.fsf@gmail.com> Message-ID: Hi, I too answered this way to this question. I thought there may be some other efficient way to handle this situation. * Regrads, Brijith P **** * On 22 March 2011 22:19, Noufal Ibrahim wrote: > On Tue, Mar 22 2011, brijithp at gmail.com wrote: > > > Hi All, > > How can I print last five lines of a file using python. The file > may > > contain thousands of lines each line may differ in length. > > [...] > > You'd have to use some kind of heuristic (a.k.a. dirty hack). Stat the > file to find it's size and use an "average" length of line to go a few > lines back. Then count the number of newlines from there to the end. If > it's 4, then you have your 5 lines. Otherwise, seek back a little more > and repeat. > > -- > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > From gora at mimirtech.com Wed Mar 23 06:01:16 2011 From: gora at mimirtech.com (Gora Mohanty) Date: Wed, 23 Mar 2011 10:31:16 +0530 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: References: <87vczb2j80.fsf@gmail.com> Message-ID: On Wed, Mar 23, 2011 at 10:15 AM, brijithp at gmail.com wrote: > Hi, > > I too answered this way to this question. I thought there may be some other > efficient way to handle this situation. [...] Is this a question for students? Otherwise, what is the rationale of doing this in Python? Regards, Gora From brijithp at gmail.com Wed Mar 23 06:43:06 2011 From: brijithp at gmail.com (brijithp at gmail.com) Date: Wed, 23 Mar 2011 11:13:06 +0530 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: References: <87vczb2j80.fsf@gmail.com> Message-ID: * Regrads, Brijith P **NuCore Software Solutions Pvt Ltd* *Mobile: +91 8089703840* Url: *www.nucoreindia.com* Email: *brijith at nucoreindia.com* On 23 March 2011 10:31, Gora Mohanty wrote: > On Wed, Mar 23, 2011 at 10:15 AM, brijithp at gmail.com > wrote: > > Hi, > > > > I too answered this way to this question. I thought there may be some > other > > efficient way to handle this situation. > [...] > > Is this a question for students? Otherwise, what is the rationale > of doing this in Python? > This a question asked by netApp > > Regards, > Gora > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > From orsenthil at gmail.com Wed Mar 23 07:24:35 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Wed, 23 Mar 2011 14:24:35 +0800 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: References: <87vczb2j80.fsf@gmail.com> Message-ID: <20110323062435.GB2698@kevin> brijithp at gmail.com wrote: > This a question asked by netApp > At the end of the interview, you could have asked the interviewer himself as what response he was looking for. - subprocess based - use tail. I would go for this. - file IO based. Seek to end of file and do a seek in the reverse direction based on a factor matching the size, and process and display. ( I found a good discussing on StackOverflow after writing this). -- Senthil From orsenthil at gmail.com Wed Mar 23 07:27:38 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Wed, 23 Mar 2011 14:27:38 +0800 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: <20110323062435.GB2698@kevin> References: <87vczb2j80.fsf@gmail.com> <20110323062435.GB2698@kevin> Message-ID: On Wed, Mar 23, 2011 at 2:24 PM, Senthil Kumaran wrote: > - file IO based. Seek to end of file and do a seek in the reverse > ?direction based on a factor matching the size, and process and > ?display. ( I found a good discussing on StackOverflow after writing > ?this). http://stackoverflow.com/questions/136168/get-last-n-lines-of-a-file-with-python-similar-to-tail -- Senthil From gnuyoga at gmail.com Wed Mar 23 08:08:30 2011 From: gnuyoga at gmail.com (Sreekandh B) Date: Wed, 23 Mar 2011 12:38:30 +0530 Subject: [BangPypers] X-Post: Amazon Web Services User Group, Bangalore Message-ID: Guys, A quick posting about the new Amazon Web Services User Group in bangalore. (I know this is no way connected to python language directly but am sure some of u like me use AWS to host our python code ;)) Bunch of us met in feb and realized the need for this and we are excitied to announce our first meet (2nd of april). (since we are hosting at jaaga we are restricting it to 100 folks). If you know some friends who are not in the group but interested/working in ?WS then pls pass the link to them as well. We have an unconference session too. More here http://www.meetup.com/AmazonAWS-Bangalore Some of AWS senior technical evanglist will be there and planning some really cool demo as well. (Sree) From santrajan at gmail.com Wed Mar 23 08:31:31 2011 From: santrajan at gmail.com (Santosh Rajan) Date: Wed, 23 Mar 2011 13:01:31 +0530 Subject: [BangPypers] X-Post: Amazon Web Services User Group, Bangalore In-Reply-To: References: Message-ID: Incidentally, reddit.com, which hosts on amazon web services, crashed last week. You can read about it here. http://blog.reddit.com/2011/03/why-reddit-was-down-for-6-of-last-24.html The reddit guys don't look too happy with AWS. On Wed, Mar 23, 2011 at 12:38 PM, Sreekandh B wrote: > Guys, > > A quick posting about the new Amazon Web Services User Group in bangalore. > > (I know this is no way connected to python language directly but am sure > some of u like me use AWS to host our python code ;)) > > Bunch of us met in feb and realized the need for this and we are excitied > to announce our first meet (2nd of april). > > (since we are hosting at jaaga we are restricting it to 100 folks). > > If you know some friends who are not in the group but interested/working in > ?WS then pls pass the link to them as well. We have an unconference session > too. > > More here http://www.meetup.com/AmazonAWS-Bangalore > > Some of AWS senior technical evanglist will be there and planning some > really cool demo as well. > > (Sree) > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- http://about.me/santosh.rajan From abpillai at gmail.com Wed Mar 23 09:14:32 2011 From: abpillai at gmail.com (Anand Balachandran Pillai) Date: Wed, 23 Mar 2011 13:44:32 +0530 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: References: <87vczb2j80.fsf@gmail.com> <20110323062435.GB2698@kevin> Message-ID: http://code.google.com/p/pytailer/ It uses an implementation very similar to the popular answer on the stack over flow link. --Anand On Wed, Mar 23, 2011 at 11:57 AM, Senthil Kumaran wrote: > On Wed, Mar 23, 2011 at 2:24 PM, Senthil Kumaran > wrote: > > - file IO based. Seek to end of file and do a seek in the reverse > > direction based on a factor matching the size, and process and > > display. ( I found a good discussing on StackOverflow after writing > > this). > > > http://stackoverflow.com/questions/136168/get-last-n-lines-of-a-file-with-python-similar-to-tail > > -- > Senthil > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- --Anand From gnuyoga at gmail.com Wed Mar 23 09:24:00 2011 From: gnuyoga at gmail.com (Sreekandh B) Date: Wed, 23 Mar 2011 13:54:00 +0530 Subject: [BangPypers] X-Post: Amazon Web Services User Group, Bangalore In-Reply-To: References: Message-ID: > Incidentally, reddit.com, which hosts on amazon web services, crashed last > week. You can read about it here. > http://blog.reddit.com/2011/03/why-reddit-was-down-for-6-of-last-24.html Interesting read ! In my opinion if you are running production stuff in any datacenter you need to design your systems for failure. (Btw am no amazon employee and thoughts are just mine ;) ) (Sree) From gora at mimirtech.com Wed Mar 23 09:37:53 2011 From: gora at mimirtech.com (Gora Mohanty) Date: Wed, 23 Mar 2011 14:07:53 +0530 Subject: [BangPypers] X-Post: Amazon Web Services User Group, Bangalore In-Reply-To: References: Message-ID: On Wed, Mar 23, 2011 at 1:54 PM, Sreekandh B wrote: >> Incidentally, reddit.com, which hosts on amazon web services, crashed last >> week. You can read about it here. >> http://blog.reddit.com/2011/03/why-reddit-was-down-for-6-of-last-24.html > > Interesting read ! Very interesting read, indeed. > In my opinion if you are running production stuff in any datacenter you need > to design your systems for failure. True, but IMHO, Amazon hosting brings in issues that one might not have thought about from past experience. I am so glad that my buddy insisted that we use RAID for databases and for crucial data partitions, on Amazon. Regards, Gora From santrajan at gmail.com Wed Mar 23 09:44:00 2011 From: santrajan at gmail.com (Santosh Rajan) Date: Wed, 23 Mar 2011 14:14:00 +0530 Subject: [BangPypers] X-Post: Amazon Web Services User Group, Bangalore In-Reply-To: References: Message-ID: > > > In my opinion if you are running production stuff in any datacenter you > need > to design your systems for failure. > > There in lies the crux of the whole thing. If you are hosting on a cloud service, who should design for failure? The host (Amazon) or you the service user? My take away from this is that, if you are operating on medium scale, (thousands of users, just a ballpark), cloud services are ok. If you scale into large scale (millions of users), you are better off with dedicated server(s). Having said that, for medium or small scale, any decent hosting service should be good enough (need not be cloud hosting). In which case it begs the question, do we need cloud hosting services at all? -- http://about.me/santosh.rajan From gnuyoga at gmail.com Wed Mar 23 13:23:59 2011 From: gnuyoga at gmail.com (Sreekandh B) Date: Wed, 23 Mar 2011 17:53:59 +0530 Subject: [BangPypers] X-Post: Amazon Web Services User Group, Bangalore In-Reply-To: References: Message-ID: > There in lies the crux of the whole thing. If you are hosting on a cloud > service, who should design for failure? The host (Amazon) or you the service > user? > Yes I got the point. In this case using availability zones would have definetly helped, so its up to us to use that particular feature and have fall back mechanism ;) (Sree) From vsapre80 at gmail.com Wed Mar 23 21:55:04 2011 From: vsapre80 at gmail.com (Vishal) Date: Thu, 24 Mar 2011 02:25:04 +0530 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: References: <87vczb2j80.fsf@gmail.com> <20110323062435.GB2698@kevin> Message-ID: On Wed, Mar 23, 2011 at 11:57 AM, Senthil Kumaran wrote: > On Wed, Mar 23, 2011 at 2:24 PM, Senthil Kumaran > wrote: > > - file IO based. Seek to end of file and do a seek in the reverse > > direction based on a factor matching the size, and process and > > display. ( I found a good discussing on StackOverflow after writing > > this). > > > http://stackoverflow.com/questions/136168/get-last-n-lines-of-a-file-with-python-similar-to-tail > > -- > Senthil > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > if you could read the entire file in one go...(i.e. unless your file is more than 50MB)...how about the following? # n is the number of lines you want to read. l = open(filename).read().rsplit('\n', n+1) # following is to keep the memory requirement low. # but this is optional, if you only want to print the lines, and then end the python process. l[0] = None gc.collect() print '\n'.join(l[1:]) ... do other stuff if you cannot read the entire file in one go...then the stackoverflow answer above from S.Lott is best. Enjoy, Vishal -- Thanks and best regards, Vishal Sapre --- "So say...Day by day, in every way, I am getting better, better and better !!!" "A Strong and Positive attitude creates more miracles than anything else." "Life is 10% how you make it, and 90% how you take it" "Diamond is another piece of coal that did well under pressure? "May we do good and not evil. May we find forgiveness for ourself and forgive others. May we share freely, never taking more than we give." "????? ?????, ????? ????? (Benefit for most people, Happiness for most people.)" The difference between "*ordinary*" and "*extraordinary*" is that "extra". Its called by another name, "*preparation*". Extraordinary people prepare themselves for situations that ordinary people choose to ignore. From orsenthil at gmail.com Thu Mar 24 03:26:45 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 24 Mar 2011 10:26:45 +0800 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: References: <87vczb2j80.fsf@gmail.com> <20110323062435.GB2698@kevin> Message-ID: <20110324022645.GA3124@kevin> On Thu, Mar 24, 2011 at 02:25:04AM +0530, Vishal wrote: > if you could read the entire file in one go...(i.e. unless your file is more > than 50MB)...how about the following? >>> for line in reversed((open('filename').readlines()[-1:-n:-1])): ... print line Some comments: > # n is the number of lines you want to read. > l = open(filename).read().rsplit('\n', n+1) - readlines would be better. > # following is to keep the memory requirement low. > # but this is optional, if you only want to print the lines, and then end > the python process. > l[0] = None - Could not get why you are setting the first item to None. > gc.collect() This does not free anything. Where is something un-referenced for it to garbage collect? -- Senthil From pradeep at btbytes.com Thu Mar 24 04:29:04 2011 From: pradeep at btbytes.com (Pradeep Gowda) Date: Wed, 23 Mar 2011 23:29:04 -0400 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: <87vczb2j80.fsf@gmail.com> References: <87vczb2j80.fsf@gmail.com> Message-ID: On Tue, Mar 22, 2011 at 12:49 PM, Noufal Ibrahim wrote: > On Tue, Mar 22 2011, brijithp at gmail.com wrote: > >> Hi All, >> ? ? ? ? How can I print last five lines of a file using python. The file may >> contain thousands of lines each line may differ in length. > > [...] > > You'd have to use some kind of heuristic (a.k.a. dirty hack). Stat the > file to find it's size and use an "average" length of line to go a few > lines back. Then count the number of newlines from there to the end. If > it's 4, then you have your 5 lines. Otherwise, seek back a little more > and repeat. Negative file.seek does not work on Text files [1] [1] http://mail.python.org/pipermail/python-bugs-list/2008-August/057225.html From vsapre80 at gmail.com Thu Mar 24 06:33:10 2011 From: vsapre80 at gmail.com (Vishal) Date: Thu, 24 Mar 2011 11:03:10 +0530 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: <20110324022645.GA3124@kevin> References: <87vczb2j80.fsf@gmail.com> <20110323062435.GB2698@kevin> <20110324022645.GA3124@kevin> Message-ID: On Thu, Mar 24, 2011 at 7:56 AM, Senthil Kumaran wrote: > On Thu, Mar 24, 2011 at 02:25:04AM +0530, Vishal wrote: > > if you could read the entire file in one go...(i.e. unless your file is > more > > than 50MB)...how about the following? > > >>> for line in reversed((open('filename').readlines()[-1:-n:-1])): > ... print line > > Some comments: > > > # n is the number of lines you want to read. > > l = open(filename).read().rsplit('\n', n+1) > > - readlines would be better. > > > # following is to keep the memory requirement low. > > # but this is optional, if you only want to print the lines, and then end > > the python process. > > l[0] = None > > - Could not get why you are setting the first item to None. > > > gc.collect() > > This does not free anything. Where is something un-referenced for it > to garbage collect? > > -- > Senthil > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > setting l[0] to None, un-references the earlier string data associated with that name, which is then (force) collected by the collect() call. I have tried it multiple times, (on my windows box) and it works perfectly. In fact, I found it to be the only way to make sure memory consumption stays low when I have to deal with reading data columns from files that are in hundreds of megabytes. Would love to know of other deterministic ways of freeing memory in Python. -- Thanks and best regards, Vishal Sapre From vsapre80 at gmail.com Thu Mar 24 06:36:14 2011 From: vsapre80 at gmail.com (Vishal) Date: Thu, 24 Mar 2011 11:06:14 +0530 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: References: <87vczb2j80.fsf@gmail.com> <20110323062435.GB2698@kevin> <20110324022645.GA3124@kevin> Message-ID: On Thu, Mar 24, 2011 at 11:03 AM, Vishal wrote: > > > On Thu, Mar 24, 2011 at 7:56 AM, Senthil Kumaran wrote: > >> On Thu, Mar 24, 2011 at 02:25:04AM +0530, Vishal wrote: >> > if you could read the entire file in one go...(i.e. unless your file is >> more >> > than 50MB)...how about the following? >> >> >>> for line in reversed((open('filename').readlines()[-1:-n:-1])): >> ... print line >> >> Some comments: >> >> > # n is the number of lines you want to read. >> > l = open(filename).read().rsplit('\n', n+1) >> >> - readlines would be better. >> >> > # following is to keep the memory requirement low. >> > # but this is optional, if you only want to print the lines, and then >> end >> > the python process. >> > l[0] = None >> >> - Could not get why you are setting the first item to None. >> >> > gc.collect() >> >> This does not free anything. Where is something un-referenced for it >> to garbage collect? >> >> -- >> Senthil >> _______________________________________________ >> BangPypers mailing list >> BangPypers at python.org >> http://mail.python.org/mailman/listinfo/bangpypers >> > > setting l[0] to None, un-references the earlier string data associated with > that name, which is then (force) collected by the collect() call. > I have tried it multiple times, (on my windows box) and it works perfectly. > > In fact, I found it to be the only way to make sure memory consumption > stays low when I have to deal with reading data columns from files that are > in hundreds of megabytes. > > Would love to know of other deterministic ways of freeing memory in Python. > > > Ok, I should have explained a little bit more. the way rsplit() is used in the my code snippet, makes the last five lines to be sent to l[1:], where as all the other lines (from the entire file) to be present in l[0]. That is why I thought of deleting all that memory, in case you use this snippet as part of a larger program. Now, setting l[0] to None, un-references the earlier string data associated with that name, which is then (force) collected by the collect() call. I have tried it multiple times, (on my windows box) and it works perfectly. In fact, I found it to be the only way to make sure memory consumption stays low when I have to deal with reading data columns from files that are in hundreds of megabytes. Would love to know of other deterministic ways of freeing memory in Python. -- Thanks and best regards, Vishal Sapre From orsenthil at gmail.com Thu Mar 24 07:37:17 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 24 Mar 2011 14:37:17 +0800 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: References: <87vczb2j80.fsf@gmail.com> <20110323062435.GB2698@kevin> <20110324022645.GA3124@kevin> Message-ID: <20110324063716.GB3124@kevin> On Thu, Mar 24, 2011 at 11:06:14AM +0530, Vishal wrote: > setting l[0] to None, un-references the earlier string data associated with > that name, which is then (force) collected by the collect() call. Can you please elaborate this further? I don't see such behavior in this snippet. >>> l [10, 20, 30] >>> sys.getsizeof(l) 104 >>> l[0] = None >>> import gc >>> gc.collect() 0 >>> sys.getsizeof(l) 104 >>> l[0] = range(1000) >>> sys.getsizeof(l) 104 >>> l[0] = None >>> gc.collect() 0 >>> sys.getsizeof(l) 104 -- Senthil From vsapre80 at gmail.com Thu Mar 24 12:22:00 2011 From: vsapre80 at gmail.com (Vishal) Date: Thu, 24 Mar 2011 16:52:00 +0530 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: <20110324063716.GB3124@kevin> References: <87vczb2j80.fsf@gmail.com> <20110323062435.GB2698@kevin> <20110324022645.GA3124@kevin> <20110324063716.GB3124@kevin> Message-ID: On Thu, Mar 24, 2011 at 12:07 PM, Senthil Kumaran wrote: > On Thu, Mar 24, 2011 at 11:06:14AM +0530, Vishal wrote: > > setting l[0] to None, un-references the earlier string data associated > with > > that name, which is then (force) collected by the collect() call. > > Can you please elaborate this further? I don't see such behavior in > this snippet. > > >>> l > [10, 20, 30] > >>> sys.getsizeof(l) > 104 > >>> l[0] = None > >>> import gc > >>> gc.collect() > 0 > >>> sys.getsizeof(l) > 104 > >>> l[0] = range(1000) > >>> sys.getsizeof(l) > 104 > >>> l[0] = None > >>> gc.collect() > 0 > >>> sys.getsizeof(l) > 104 > > -- > Senthil > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > Use of sys.getsizeof() is misleading, specially for sequence objects. sys.getsizeof() does not give the correct size of the data that is referenced by the collection itself. So that would not be a good way to find size of the actual data. You should use something like this: http://code.activestate.com/recipes/546530/ . I saved this recipe by the name of PyObjSize which I have used below. >>> mb = 1024*1024 >>> s1 = 'a'*mb >>> s2 = 'b'*mb >>> s3 = 'c'*mb >>> sys.getsizeof(s1) 1048600 >>> sys.getsizeof(s2) 1048600 >>> sys.getsizeof(s3) 1048600 >>> l = [s1, s2, s3] >>> sys.getsizeof(l) 48 >>> import PyObjSize >>> PyObjSize.asizeof(l) 3145848 >>> l[0] = None >>> PyObjSize.asizeof(l) 2097256 >>> collect() 0 Return from collect() is the number of "unreachable objects" which, I think, means number of objects that could not be collected as part of this run. The doc also says that ints and floats may not be collected correctly. http://docs.python.org/library/gc.html You'd have to look at the process size, using the Windows Task Manager or may be top in Unix. Using these tools, I have found that, string data gets collected fine. -- Thanks and best regards, Vishal Sapre From jinsthomas at gmail.com Thu Mar 24 13:04:58 2011 From: jinsthomas at gmail.com (Jins Thomas) Date: Thu, 24 Mar 2011 17:34:58 +0530 Subject: [BangPypers] Dictionary in Python - A doubt Message-ID: Hi all, This is a very basic doubt. I was experimenting concept of dictionary in python3 My problem was to find the occurance of each word in a file f = open('hello.txt', 'r') count = {} for line in f: words = line.split() for i in words : if i in count: count[i] += 1 else: count[i] = 1 print (count) I was just comparing hash in Perl. In Perl 'if i in count: else:' statement is not required i could simply uses count{i} +=1 even if it exists or not exists. I was thinking why Python has put this restriction. Or is it something which i did wrongly. Also i'm looking for some sites similar to perlmonks.org in python. There in perlmonks we could see lots of small small articles (basic and advanced) which will give clear idea on the usage, style etc. Have you heard of anything of that sort. Thanks Jins Thomas From orsenthil at gmail.com Thu Mar 24 13:10:33 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 24 Mar 2011 20:10:33 +0800 Subject: [BangPypers] Dictionary in Python - A doubt In-Reply-To: References: Message-ID: <20110324121033.GB2801@kevin> On Thu, Mar 24, 2011 at 05:34:58PM +0530, Jins Thomas wrote: > > I was just comparing hash in Perl. In Perl 'if i in count: else:' statement > is not required i could simply uses count{i} +=1 even if it exists or not > exists. I was thinking why Python has put this restriction. Or is it > something which i did wrongly. That is not a restriction. That is consistent in understanding of how dictionary is supposed to behave. BTW for perl equivalent behavior, look at collections.defaultdict >>> s = 'mississippi' >>> d = defaultdict(int) >>> for k in s: ... d[k] += 1 ... >>> d.items() [('i', 4), ('p', 2), ('s', 4), ('m', 1)] -- Senthil From orsenthil at gmail.com Thu Mar 24 13:13:38 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 24 Mar 2011 20:13:38 +0800 Subject: [BangPypers] How to handle files efficiently in python In-Reply-To: References: <20110323062435.GB2698@kevin> <20110324022645.GA3124@kevin> <20110324063716.GB3124@kevin> Message-ID: <20110324121338.GC2801@kevin> Vishal wrote: > size of the actual data. You should use something like this: > http://code.activestate.com/recipes/546530/ . I saved this recipe by the > name of PyObjSize which I have used below. > > >>> import PyObjSize > >>> PyObjSize.asizeof(l) > 3145848 > >>> l[0] = None > >>> PyObjSize.asizeof(l) > 2097256 > >>> collect() > 0 Thanks! That helps in understanding it better. -- Senthil From b.ghose at gmail.com Thu Mar 24 13:16:20 2011 From: b.ghose at gmail.com (Baishampayan Ghose) Date: Thu, 24 Mar 2011 17:46:20 +0530 Subject: [BangPypers] Dictionary in Python - A doubt In-Reply-To: References: Message-ID: On Thu, Mar 24, 2011 at 5:34 PM, Jins Thomas wrote: > My problem was to find the occurance of each word in a file > > f = open('hello.txt', 'r') > count = {} > for line in f: > ? ?words = line.split() > ? ?for i in words : > ? ? ? ?if i in count: > ? ? ? ? ? ?count[i] += 1 > ? ? ? ?else: > ? ? ? ? ? ?count[i] = 1 > print (count) > > I was just comparing hash in Perl. In Perl 'if i in count: ?else:' statement > is not required i could simply uses count{i} +=1 even if it exists or not > exists. I was thinking why Python has put this restriction. Or is it > something which i did wrongly. Perl let's you do a += on a dictionary value because Perl is weakly typed(?) and Python is not. To tell Python to assume that the default value of your dictionary is an integer instead of anything else, you will have to use `defaultdict`[1] which is a subclass of the built-in Python dict. This is how you can use it - count = defaultdict(int) with open("hello.txt", "r") as f: for line in f: words = line.split() for i in words: count[i] += 1 print (count) Hope this helps. Regards, BG [1] http://docs.python.org/library/collections.html#collections.defaultdict -- Baishampayan Ghose b.ghose at gmail.com From vsapre80 at gmail.com Fri Mar 25 10:52:02 2011 From: vsapre80 at gmail.com (Vishal) Date: Fri, 25 Mar 2011 15:22:02 +0530 Subject: [BangPypers] chaining string operations.. Message-ID: Hello, I have a string that can be many megabytes worth memory size, and i want to perform simple string replacements. There is a certain sequence I have to follow...so I do the following: >>> type(s) >>> s1 = s.replace(' a', 'S').replace(' \t', '\n').replace('C#', '\n') Now I would like to chain these replacements into something like this: >>> s1 = s.replacements((' a', 'S'), (' \t', '\n'), ('C#', '\n'))) One could write this function in Python itself, but I want to push this down to the C level. Is there a known way of doing this currently in Python, apart from producing my own Cython/C extension module. Thanks and best regards, Vishal Sapre From orsenthil at gmail.com Fri Mar 25 11:01:08 2011 From: orsenthil at gmail.com (Senthil Kumaran) Date: Fri, 25 Mar 2011 18:01:08 +0800 Subject: [BangPypers] chaining string operations.. In-Reply-To: References: Message-ID: <20110325100108.GA8525@kevin> On Fri, Mar 25, 2011 at 03:22:02PM +0530, Vishal wrote: > I have a string that can be many megabytes worth memory size, and i > want to If you have got that string from a file like object, instead of loading the entire string in memory you could iterate through the file object to load one line at time. This will help provided the string is has line-separators. Otherwise you could store in a mock StringIO file like object and iterate, if the string is heavy to handle in memory. > > Now I would like to chain these replacements into something like this: > > >>> s1 = s.replacements((' a', 'S'), (' \t', '\n'), ('C#', '\n'))) > > Is there a known way of doing this currently in Python, apart from producing Here is one recipe to do multiple string replacement in python in a single shot. http://uthcode.sarovar.org/pycon2010/strings.html#making-multiple-replacements-in-a-string-in-a-single-pass -- Senthil From jinsthomas at gmail.com Fri Mar 25 11:53:09 2011 From: jinsthomas at gmail.com (Jins Thomas) Date: Fri, 25 Mar 2011 16:23:09 +0530 Subject: [BangPypers] Python equivalent for Pelmonks.org Message-ID: Hi all, Do we have some site which we can compare with perlmonks.org for Python. I have seen perlmonks.org is very helpful with lot's of Q&As , some small practical tutorials, thoughts on programming etc, code snippets etc. Thanks Jins Thomas From santrajan at gmail.com Fri Mar 25 12:10:59 2011 From: santrajan at gmail.com (Santosh Rajan) Date: Fri, 25 Mar 2011 16:40:59 +0530 Subject: [BangPypers] Python equivalent for Pelmonks.org In-Reply-To: References: Message-ID: I think the perl monks have already answered your question. http://www.perlmonks.org/?node_id=511582 On Fri, Mar 25, 2011 at 4:23 PM, Jins Thomas wrote: > Hi all, > > Do we have some site which we can compare with perlmonks.org for Python. I > have seen perlmonks.org is very helpful with lot's of Q&As , some small > practical tutorials, thoughts on programming etc, code snippets etc. > > > Thanks > Jins Thomas > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- http://about.me/santosh.rajan ?The *young man* knows the rules but the *old man* knows the exceptions?. *Oliver Wendell Holmes* From jinsthomas at gmail.com Fri Mar 25 12:50:21 2011 From: jinsthomas at gmail.com (Jins Thomas) Date: Fri, 25 Mar 2011 17:20:21 +0530 Subject: [BangPypers] Python equivalent for Pelmonks.org In-Reply-To: References: Message-ID: On Fri, Mar 25, 2011 at 4:40 PM, Santosh Rajan wrote: > I think the perl monks have already answered your question. > > http://www.perlmonks.org/?node_id=511582 > > Yes that's right. I had gone through that. But it didn't give any solid results. > On Fri, Mar 25, 2011 at 4:23 PM, Jins Thomas wrote: > > > Hi all, > > > > Do we have some site which we can compare with perlmonks.org for Python. > I > > have seen perlmonks.org is very helpful with lot's of Q&As , some small > > practical tutorials, thoughts on programming etc, code snippets etc. > > > > > > Thanks > > Jins Thomas > > _______________________________________________ > > BangPypers mailing list > > BangPypers at python.org > > http://mail.python.org/mailman/listinfo/bangpypers > > > > > > -- > http://about.me/santosh.rajan > > ?The *young man* knows the rules but the *old man* knows the > exceptions?. *Oliver > Wendell Holmes* > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > From vsapre80 at gmail.com Fri Mar 25 13:15:16 2011 From: vsapre80 at gmail.com (Vishal) Date: Fri, 25 Mar 2011 17:45:16 +0530 Subject: [BangPypers] chaining string operations.. In-Reply-To: <20110325100108.GA8525@kevin> References: <20110325100108.GA8525@kevin> Message-ID: On Fri, Mar 25, 2011 at 3:31 PM, Senthil Kumaran wrote: > On Fri, Mar 25, 2011 at 03:22:02PM +0530, Vishal wrote: > > > I have a string that can be many megabytes worth memory size, and i > > want to > > If you have got that string from a file like object, instead of > loading the entire string in memory you could iterate through the file > object to load one line at time. This will help provided the string is > has line-separators. > > Otherwise you could store in a mock StringIO file like object and > iterate, if the string is heavy to handle in memory. > > > > > Now I would like to chain these replacements into something like this: > > > > >>> s1 = s.replacements((' a', 'S'), (' \t', '\n'), ('C#', '\n'))) > > > > Is there a known way of doing this currently in Python, apart from > producing > > Here is one recipe to do multiple string replacement in python in a > single shot. > > > http://uthcode.sarovar.org/pycon2010/strings.html#making-multiple-replacements-in-a-string-in-a-single-pass > > -- > Senthil > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > Thanks for his one. This is what I was looking for. -- Thanks and best regards, Vishal Sapre From lawgon at gmail.com Sat Mar 26 09:32:33 2011 From: lawgon at gmail.com (Kenneth Gonsalves) Date: Sat, 26 Mar 2011 14:02:33 +0530 Subject: [BangPypers] Python equivalent for Pelmonks.org In-Reply-To: References: Message-ID: <1301128353.1960.7.camel@localhost> On Fri, 2011-03-25 at 17:20 +0530, Jins Thomas wrote: > On Fri, Mar 25, 2011 at 4:40 PM, Santosh Rajan > wrote: > > > I think the perl monks have already answered your question. > > > > http://www.perlmonks.org/?node_id=511582 > > > > > Yes that's right. I had gone through that. But it didn't give any > solid > results. well, django has djangosnippets.org - I do not know about the rest of python. (why don't you start one?) -- regards Kenneth Gonsalves http://lawgon.livejournal.com/ From rmathews at gmail.com Sun Mar 27 03:21:10 2011 From: rmathews at gmail.com (Roshan Mathews) Date: Sun, 27 Mar 2011 06:51:10 +0530 Subject: [BangPypers] Python equivalent for Pelmonks.org In-Reply-To: <1301128353.1960.7.camel@localhost> References: <1301128353.1960.7.camel@localhost> Message-ID: On Sat, Mar 26, 2011 at 14:02, Kenneth Gonsalves wrote: > well, django has djangosnippets.org - I do not know about the rest of > python. (why don't you start one?) > http://code.activestate.com/recipes/ should be the Python equivalent of djangosnippets. -- http://about.me/rosh From abpillai at gmail.com Mon Mar 28 05:43:43 2011 From: abpillai at gmail.com (Anand Balachandran Pillai) Date: Mon, 28 Mar 2011 09:13:43 +0530 Subject: [BangPypers] Fwd: Help with Python IDE's In-Reply-To: References: Message-ID: Hi Folks, Any suggestions for Cherian ? I am not myself aware of any Django specific IDE. Thanks, --Anand ---------- Forwarded message ---------- From: Cherian Thomas Date: Sat, Mar 26, 2011 at 7:49 PM Subject: Help with Python IDE's To: Anand Balachandran Pillai Hi, Can you suggest me an IDE for Django development? I looking to build an app at Zynga in extremely fast iterations. I am not very conversant with Python, though i have programmed proabably 8-10 times. Regards, Cherian -- --Anand From abpillai at gmail.com Mon Mar 28 05:45:59 2011 From: abpillai at gmail.com (Anand Balachandran Pillai) Date: Mon, 28 Mar 2011 09:15:59 +0530 Subject: [BangPypers] Python equivalent for Pelmonks.org In-Reply-To: References: Message-ID: On Fri, Mar 25, 2011 at 4:40 PM, Santosh Rajan wrote: > I think the perl monks have already answered your question. > > http://www.perlmonks.org/?node_id=511582 > Short answer is No. Long answer is "Python has no single centralized repository of knowledge which is an analogy to perlmonks.org". Still longer answer is "Python has no single centralized repository of knowledge which is an analogy to perlmonks.org. It is more distributed all over the place - in c.l.p, stackoverflow, blogs, forums such as these". --Anand > > On Fri, Mar 25, 2011 at 4:23 PM, Jins Thomas wrote: > > > Hi all, > > > > Do we have some site which we can compare with perlmonks.org for Python. > I > > have seen perlmonks.org is very helpful with lot's of Q&As , some small > > practical tutorials, thoughts on programming etc, code snippets etc. > > > > > > Thanks > > Jins Thomas > > _______________________________________________ > > BangPypers mailing list > > BangPypers at python.org > > http://mail.python.org/mailman/listinfo/bangpypers > > > > > > -- > http://about.me/santosh.rajan > > ?The *young man* knows the rules but the *old man* knows the > exceptions?. *Oliver > Wendell Holmes* > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- --Anand From gora at mimirtech.com Mon Mar 28 05:48:15 2011 From: gora at mimirtech.com (Gora Mohanty) Date: Mon, 28 Mar 2011 09:18:15 +0530 Subject: [BangPypers] Fwd: Help with Python IDE's In-Reply-To: References: Message-ID: On Mon, Mar 28, 2011 at 9:13 AM, Anand Balachandran Pillai wrote: > Hi Folks, > > ?Any suggestions for Cherian ? I am not myself aware of any Django specific > IDE. You could refer him to http://lmgtfy.com/?q=django+ide :-) Most any IDE that one is using, and that handles Python should work. Personally, I use emacs with various modes for Django development. Regards, Gora From ramdaz at gmail.com Mon Mar 28 05:57:44 2011 From: ramdaz at gmail.com (Ramdas S) Date: Mon, 28 Mar 2011 09:27:44 +0530 Subject: [BangPypers] Fwd: Help with Python IDE's In-Reply-To: References: Message-ID: On Mon, Mar 28, 2011 at 9:18 AM, Gora Mohanty wrote: > On Mon, Mar 28, 2011 at 9:13 AM, Anand Balachandran Pillai > wrote: > > Hi Folks, > > > > Any suggestions for Cherian ? I am not myself aware of any Django > specific > > IDE. > > Pycharm is the best IDE available, but its commercial> Otherwise you can use Eclipse + Pydev > You could refer him to http://lmgtfy.com/?q=django+ide > :-) > > Most any IDE that one is using, and that handles Python > should work. Personally, I use emacs with various modes > for Django development. > > Regards, > Gora > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- Ramdas S +91 9342 583 065 From navin.kabra at gmail.com Mon Mar 28 06:07:12 2011 From: navin.kabra at gmail.com (Navin Kabra) Date: Mon, 28 Mar 2011 09:37:12 +0530 Subject: [BangPypers] Fwd: Help with Python IDE's In-Reply-To: References: Message-ID: On Mon, Mar 28, 2011 at 9:18 AM, Gora Mohanty wrote: > Most any IDE that one is using, and that handles Python > should work. Personally, I use emacs with various modes > for Django development. > Can you share exactly which modes you're using for django developement. I use python-mode.el and while it works reasonably well for python, I haven't managed to make it work for django (it does not autocomplete for django). Also, any other tips/tricks you can share would help. Thanks. From santrajan at gmail.com Mon Mar 28 06:09:50 2011 From: santrajan at gmail.com (Santosh Rajan) Date: Mon, 28 Mar 2011 09:39:50 +0530 Subject: [BangPypers] Fwd: Help with Python IDE's In-Reply-To: References: Message-ID: If you are planning to work on Python in the long term, emacs is the way to go. If you are looking for something simple for the short term Komodo Edit is an excellent choice. Please note Komodo Edit is free, while Komodo IDE is not free. On Mon, Mar 28, 2011 at 9:13 AM, Anand Balachandran Pillai < abpillai at gmail.com> wrote: > Hi Folks, > > Any suggestions for Cherian ? I am not myself aware of any Django specific > IDE. > > Thanks, > > --Anand > > ---------- Forwarded message ---------- > From: Cherian Thomas > Date: Sat, Mar 26, 2011 at 7:49 PM > Subject: Help with Python IDE's > To: Anand Balachandran Pillai > > > Hi, > Can you suggest me an IDE for Django development? I looking to build an app > at Zynga in extremely fast iterations. > I am not very conversant with Python, though i have programmed proabably > 8-10 times. > > Regards, > Cherian > > > > > -- > --Anand > _______________________________________________ > BangPypers mailing list > BangPypers at python.org > http://mail.python.org/mailman/listinfo/bangpypers > -- http://about.me/santosh.rajan ?The *young man* knows the rules but the *old man* knows the exceptions?. *Oliver Wendell Holmes* From rmathews at gmail.com Mon Mar 28 06:52:01 2011 From: rmathews at gmail.com (Roshan Mathews) Date: Mon, 28 Mar 2011 10:22:01 +0530 Subject: [BangPypers] Fwd: Help with Python IDE's In-Reply-To: References: Message-ID: On Mon, Mar 28, 2011 at 09:37, Navin Kabra wrote: > On Mon, Mar 28, 2011 at 9:18 AM, Gora Mohanty wrote: >> Most any IDE that one is using, and that handles Python >> should work. Personally, I use emacs with various modes >> for Django development. > > Can you share exactly which modes you're using for django developement. > Yes, please give us more details. > I use python-mode.el and while it works reasonably well for python, I > haven't managed to make it work for django (it does not autocomplete for > django). Also, any other tips/tricks you can share would help. Thanks. > I use the default "python-mode" from python.el ... what is python-mode? And you mean "smart" auto-complete? I usually just go with dabbrev-expand (M-/). Maybe I've gotten too comfortable with my setup, never looked around for enhancements to Emacs' default python handling. Details of your setup would be appreciated. Roshan -- http://about.me/rosh From vsapre80 at gmail.com Mon Mar 28 09:09:51 2011 From: vsapre80 at gmail.com (Vishal) Date: Mon, 28 Mar 2011 12:39:51 +0530 Subject: [BangPypers] chaining string operations.. In-Reply-To: References: <20110325100108.GA8525@kevin> Message-ID: On Fri, Mar 25, 2011 at 5:45 PM, Vishal wrote: > On Fri, Mar 25, 2011 at 3:31 PM, Senthil Kumaran wrote: > >> On Fri, Mar 25, 2011 at 03:22:02PM +0530, Vishal wrote: >> >> > I have a string that can be many megabytes worth memory size, and i >> > want to >> >> If you have got that string from a file like object, instead of >> loading the entire string in memory you could iterate through the file >> object to load one line at time. This will help provided the string is >> has line-separators. >> >> Otherwise you could store in a mock StringIO file like object and >> iterate, if the string is heavy to handle in memory. >> >> > >> > Now I would like to chain these replacements into something like this: >> > >> > >>> s1 = s.replacements((' a', 'S'), (' \t', '\n'), ('C#', '\n'))) >> > >> > Is there a known way of doing this currently in Python, apart from >> producing >> >> Here is one recipe to do multiple string replacement in python in a >> single shot. >> >> >> http://uthcode.sarovar.org/pycon2010/strings.html#making-multiple-replacements-in-a-string-in-a-single-pass >> >> -- >> Senthil >> _______________________________________________ >> BangPypers mailing list >> BangPypers at python.org >> http://mail.python.org/mailman/listinfo/bangpypers >> > > Thanks for his one. This is what I was looking for. > > -- > Thanks and best regards, > Vishal Sapre > Hi, Just realized after using the above solution : http://uthcode.sarovar.org/pycon2010/strings.html#making-multiple-replacements-in-a-string-in-a-single-pass def multiple_replace(text, adict): rx = re.compile('|'.join(map(re.escape, adict))) def one_xlat(match): return adict[match.group(0)] return rx.sub(one_xlat, text) that *this one is not* what I was looking for. The use of '|' character in the regular expression, makes sure that the sequence of replacements is not preserved, which is exactly what I want to avoid. Is there another way to making (compiling) the regex such that match happens as per a given sequence, and not as per occurrence of pattern in the given string. -- Thanks and best regards, Vishal Sapre From amit.pureenergy at gmail.com Mon Mar 28 09:58:22 2011 From: amit.pureenergy at gmail.com (Amit Sethi) Date: Mon, 28 Mar 2011 13:28:22 +0530 Subject: [BangPypers] Fwd: Help with Python IDE's In-Reply-To: References: Message-ID: I think you might be looking for pydev for eclipse http://pydev.blogspot.com/2006/09/configuring-pydev-to-work-with-django.html On emacs you could use python-mode for autocomplete and https://github.com/gregnewman/django-mode if you want snippets, the later is based on yasnippet. From noufal at gmail.com Mon Mar 28 12:42:38 2011 From: noufal at gmail.com (Noufal Ibrahim) Date: Mon, 28 Mar 2011 16:12:38 +0530 Subject: [BangPypers] Fwd: Help with Python IDE's In-Reply-To: <87y63zcz9c.fsf@gmail.com> (Noufal Ibrahim's message of "Mon, 28 Mar 2011 16:01:27 +0530") References: <87y63zcz9c.fsf@gmail.com> Message-ID: <87ipv3cyqp.fsf@nibrahim.net.in> On Mon, Mar 28 2011, Noufal Ibrahim wrote: [...] > JetBrains (creators of the famous Java IDE IntelliJ which a lot of > good people I knwo swear by) created PyCharm[2] that specifically > boasts of good Django integration. I met the devs and they seem like > bright people. Another interesting proprietary product is the Python integration plugin for Microsoft Visual Studio. I saw a demo of it and was quite impressed especially with its introspection and code completion capabilities. It's Open Source (although the IDE itself is not). It doesn't however have any Django specific functionality. And also, it's Windows only. http://pytools.codeplex.com/ This SO thread discusses a bunch of other options that you might find useful. http://stackoverflow.com/questions/537689/python-ide-built-into-visual-studio-2008 My personal recommendation? Choose a single, cross platform, open source editor that you can script and customise (vim, emacs, eclipse etc.) and learn it well. Use it for all your work. The comfort you get with being a master at your work environment will more than compensate for all the bells and whistles you get with language specific IDEs. -- ~noufal http://nibrahim.net.in From noufal at gmail.com Mon Mar 28 12:31:27 2011 From: noufal at gmail.com (Noufal Ibrahim) Date: Mon, 28 Mar 2011 16:01:27 +0530 Subject: [BangPypers] Fwd: Help with Python IDE's In-Reply-To: (Anand Balachandran Pillai's message of "Mon, 28 Mar 2011 09:13:43 +0530") References: Message-ID: <87y63zcz9c.fsf@gmail.com> On Mon, Mar 28 2011, Anand Balachandran Pillai wrote: [...] > ---------- Forwarded message ---------- > From: Cherian Thomas [...] > Can you suggest me an IDE for Django development? I looking to build > an app at Zynga in extremely fast iterations. I am not very > conversant with Python, though i have programmed proabably 8-10 times. [...] If you're doing this for a commercial organisation, you should get them to buy a license for whatever IDE you're comfortable with using. Switching IDEs when you switch languages seems (atleast to me) to be a non productive way to go since there are a new batch of idioms to learn [1]. I use Emacs and nxhtml mode with mumamo to edit my templates when I do any Django work and that works fine for me. Apart from the templates, Django is (mostly) just Python so the stock Python mode works fine for me along with pylint and a wrapper to do on the fly semantic checking. JetBrains (creators of the famous Java IDE IntelliJ which a lot of good people I knwo swear by) created PyCharm[2] that specifically boasts of good Django integration. I met the devs and they seem like bright people. Footnotes: [1] Given that I use Emacs for everything except browsing and games, you should take this with a few bags of salt. [2] http://www.jetbrains.com/pycharm/ -- From lawgon at gmail.com Mon Mar 28 13:34:51 2011 From: lawgon at gmail.com (Kenneth Gonsalves) Date: Mon, 28 Mar 2011 17:04:51 +0530 Subject: [BangPypers] Fwd: Help with Python IDE's In-Reply-To: References: Message-ID: <1301312091.2254.1.camel@localhost> On Mon, 2011-03-28 at 09:13 +0530, Anand Balachandran Pillai wrote: > Any suggestions for Cherian ? I am not myself aware of any Django > specific > IDE. I know vim, emacs and textmate have django bindings. I use geany. -- regards Kenneth Gonsalves http://lawgon.livejournal.com/ From abpillai at gmail.com Wed Mar 30 07:51:11 2011 From: abpillai at gmail.com (Anand Balachandran Pillai) Date: Wed, 30 Mar 2011 11:21:11 +0530 Subject: [BangPypers] Gsoc - Mediawiki Parser in Python Message-ID: If anyone is interested, https://wiki.mozilla.org/Community:SummerOfCode11:MediaWikiParser -- --Anand From ashok at parliaments.info Thu Mar 31 09:04:22 2011 From: ashok at parliaments.info (Ashok Hariharan) Date: Thu, 31 Mar 2011 10:04:22 +0300 Subject: [BangPypers] [JOB] Technical Writers wanted Message-ID: Hello there -- We are a United Nations organization (UN/DESA) developing a Parliamentary Information System on the Zope3 platform (see http://www.bungeni.org and http://www.parliaments.info). We want to sub-contract technical writing work for the parliamentary information system being developed by us. The scope of work is to extend and expand on the current technical documentation (see http://code.google.com/p/bungeni-portal/wiki/TableOfContents?tm=6) into a complete "Developers Guide" and "Administrators guide" Interested individuals / parties may contact me offlist... thanks ! Ashok Hariharan From sateeshpyper at gmail.com Wed Mar 30 09:10:56 2011 From: sateeshpyper at gmail.com (Sateesh Kumar) Date: Wed, 30 Mar 2011 12:40:56 +0530 Subject: [BangPypers] Dictionary in Python - A doubt In-Reply-To: References: Message-ID: On Thu, Mar 24, 2011 at 5:34 PM, Jins Thomas wrote: > > I was just comparing hash in Perl. In Perl 'if i in count: else:' > statement > is not required i could simply uses count{i} +=1 even if it exists or not > exists. This is a Perl feature known as autovivification. http://en.wikipedia.org/wiki/Autovivification reg, sateesh