[Tutor] Re subprocess
jarod_v6 at libero.it
jarod_v6 at libero.it
Wed Sep 10 11:20:38 CEST 2014
If I follow the exmple I have this type of error:
File "./RNA_prova.py", line 73, in run
for line in p1.stdout():
TypeError: 'NoneType' object is not callable
This is the class I use:
def run(cmd,pi):
import subprocess
import time
import logging
logging.basicConfig(level=logging.DEBUG,format="%(asctime)s - %(name)s - %
(levelname)s - %(message)s")
#logging.debug(" Running pipelines: %s" % (cmd))
# setup logging
log_file = "None"
tou = "file"+"_.log.txt"
if log_file is not None:
logfh = open(tou, "w")
else:
logfh = None
print "####################################################"
p1 = subprocess.Popen(cmd,shell=True,stdout=logfh,stderr=logfh,cwd=pi)
#logging.info(" Running pipelines: %s" % (cmd))
for line in p1.stdout():
process(line)
p1.stdout.close()
if p.wait() != 0:
raise Exception
# end logging
if logfh is not None:
logfh.close()
return 0
How can be sure the process start finish before to proced with the other
comands?
thanks in advance for any help!
>----Messaggio originale----
>Da: tutor-request at python.org
>Data: 09/09/2014 17.05
>A: <tutor at python.org>
>Ogg: Tutor Digest, Vol 127, Issue 27
>
>Send Tutor mailing list submissions to
> tutor at python.org
>
>To subscribe or unsubscribe via the World Wide Web, visit
> https://mail.python.org/mailman/listinfo/tutor
>or, via email, send a message with subject or body 'help' to
> tutor-request at python.org
>
>You can reach the person managing the list at
> tutor-owner at python.org
>
>When replying, please edit your Subject line so it is more specific
>than "Re: Contents of Tutor digest..."
>
>
>Today's Topics:
>
> 1. Re: Good approach regarding classes attributes (Joel Goldstick)
> 2. Re: Understand subprocess poll (Peter Otten)
> 3. Re: Understand subprocess poll (jarod_v6 at libero.it)
> 4. Re: Good approach regarding classes attributes (Sydney Shall)
>
>
>----------------------------------------------------------------------
>
>Message: 1
>Date: Tue, 9 Sep 2014 10:05:09 -0400
>From: Joel Goldstick <joel.goldstick at gmail.com>
>Cc: "tutor at python.org" <tutor at python.org>
>Subject: Re: [Tutor] Good approach regarding classes attributes
>Message-ID:
> <CAPM-O+wovzN+RFDMT321GmqKwx9FP2uA1bXgQ_6FY69Q6iWwdQ at mail.gmail.com>
>Content-Type: text/plain; charset=UTF-8
>
>On Tue, Sep 9, 2014 at 10:02 AM, Sydney Shall <s.shall at virginmedia.com>
wrote:
>> On 09/09/2014 15:44, Peter Otten wrote:
>>
>> Sydney Shall wrote:
>>
>> On 08/09/2014 18:39, Alan Gauld wrote:
>>
>> On 08/09/14 15:17, Juan Christian wrote:
>>
>> One tiny tweak...
>>
>> class User():
>>
>> You don't need the parens after User. You don;t have any superclasses
>> so they do nothing. Python convention for an empty parent list is just
>> to leave the parens off:
>>
>> class User:
>>
>> A simple question from a newbie, in response to this surprise.
>> Is it not helpful to always put (object) as the parent, if the class is
>> not itself a sub-class?
>>
>> The answer differs between Python 2 and 3. In Python 3
>>
>> class C: # preferred in Python 3
>> pass
>>
>> and
>>
>> class C(object):
>> pass
>>
>> are the same, so there is no point adding the explicit object inheritance.
>>
>> In Python 2 however
>>
>> class C:
>> pass
>>
>> will create a "classic class" whereas
>>
>> class C(object): # preferred in Python 2
>> pass
>>
>> is a "newstyle class". The most notable difference between these is that
>> properties work correctly only with newstyle classes. Therefore making all
>> your classes "newstyle" is a good idea.
>>
>> And while I am writing, what does OP stand for in this list?
>>
>> Original Poster, as Leam says.
>>
>>
>> _______________________________________________
>> Tutor maillist - Tutor at python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>> Thanks Peter, most helpful.
>> I was taught with Python 2.7, so now I understand the advice.
>>
>>
>> --
>> Sydney Shall
>>
>> _______________________________________________
>> Tutor maillist - Tutor at python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>Please post in plain text
>
>--
>Joel Goldstick
>http://joelgoldstick.com
>
>
>------------------------------
>
>Message: 2
>Date: Tue, 09 Sep 2014 16:18:40 +0200
>From: Peter Otten <__peter__ at web.de>
>To: tutor at python.org
>Subject: Re: [Tutor] Understand subprocess poll
>Message-ID: <lun280$7j5$1 at ger.gmane.org>
>Content-Type: text/plain; charset="ISO-8859-1"
>
>Wolfgang Maier wrote:
>
>> On 09/09/2014 11:45 AM, Peter Otten wrote:
>>> jarod_v6 at libero.it wrote:
>>>
>>>> I want to use subprocess for run some programs But I need to be sure the
>>>> program end before continue with the other:
>>>>
>>>> subprocess.call("ls")
>>>> cmd1 = i
>>>> p1 = subprocess.Popen(cmd1,shell=True,stdout=subprocess.PIPE)
>>>>
>>>> while True:
>>>> if p1.poll() is None:
>>>> time.sleep(3)
>>>>
>>>> pass
>>>> if p1.poll()==0:
>>>> print '#'
>>>> break
>>>> if p1.poll() is not None and p1.poll() != 0:
>>>> raise Exception('Error building Alignment using star with hg19
>>>> database')
>>>
>>>> This are not working. How can I do?
>>>> thanks in advance for the precious help
>>>> bw,
>>>
>>> I don't understand why you would need this loop. Why don't you use
>>> subprocess.call() and be done?
>>>
>>
>> The OP is piping the process stdout so I assume he is going to read from
>> it in place of the pass in his example.
>> Since the subprocess is doing genome-wide sequence alignment (at least I
>> guess so from the exception string) there will be lots of output, which
>> would cause subprocess.call() to block.
>>
>> Assuming that the posted code was indented correctly and was otherwise
>> run as posted this could also be the answer to the original question:
>> you have to keep on consuming data from the pipe or its buffer is going
>> to fill up and block everyhing. With a simple pass statement you do not
>> achieve anything that you can't do with call.
>
>Ah, you're right.
>
>I still don't see where the need to poll arises, I'd expect something like
>
>p = subprocess.Popen(cmd, ..., stdout=PIPE)
>for line in p.stdout:
> process(line)
>
>p.stdout.close()
>if p.wait() != 0:
> raise Exception
>
>to work.
>
>
>
>------------------------------
>
>Message: 3
>Date: Tue, 9 Sep 2014 16:36:47 +0200 (CEST)
>From: "jarod_v6 at libero.it" <jarod_v6 at libero.it>
>To: <tutor at python.org>
>Subject: Re: [Tutor] Understand subprocess poll
>Message-ID:
> <28501232.4398921410273407217.JavaMail.httpd at webmail-07.iol.local>
>Content-Type: text/plain;charset="UTF-8"
>
>Thanks for yhe help!
>The comand run is this
>
>the class created are this:
>def run(cmd,pi):
> import subprocess
> import time
> import logging
>
> logging.basicConfig(level=logging.DEBUG,format="%(asctime)s - %(name)s - %
>(levelname)s - %(messag
>e)s")
>
> #logging.debug(" Running pipelines: %s" % (cmd))
> # setup logging
> log_file = "None"
> tou = "file"+"_.log.txt"
> if log_file is not None:
> logfh = open(tou, "w")
>
> else:
> logfh = None
> print "####################################################"
>
>
> p1 = subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE,stderr=logfh,
>cwd=pi)
>
> #logging.info(" Running pipelines: %s" % (cmd))
> while True:
> if p1.poll() is None:
> time.sleep(3)
> pass
> if p1.poll()==0:
> print 'Finish this step.'
> logging.info("###################: %s %s" % (cmd,time.ctime()))
> break
> if p1.poll() is not None and p1.poll() != 0:
> raise Exception('Not working please check the error')
> # end logging
> if logfh is not None:
> logfh.close()
>
> return 0
>
>
>step_2_out =["~/software/STAR_2.3.0e.Linux_x86_64_static/STAR --genomeDir
>/home/sbsuser/d
>atabases/Starhg19/GenomeDir/ --runMode alignReads --readFilesIn %s %s --
>runThreadN 12 --readFilesCommand zcat
> "%(dx,sn)]
>
>The problems is the script end the process but not allign all the data. If
I
>use the comand line the same code the process work.
>How can resolve this issue?
>
>
>
>
>>----Messaggio originale----
>>Da: tutor-request at python.org
>>Data: 09/09/2014 16.02
>>A: <tutor at python.org>
>>Ogg: Tutor Digest, Vol 127, Issue 26
>>
>>Send Tutor mailing list submissions to
>> tutor at python.org
>>
>>To subscribe or unsubscribe via the World Wide Web, visit
>> https://mail.python.org/mailman/listinfo/tutor
>>or, via email, send a message with subject or body 'help' to
>> tutor-request at python.org
>>
>>You can reach the person managing the list at
>> tutor-owner at python.org
>>
>>When replying, please edit your Subject line so it is more specific
>>than "Re: Contents of Tutor digest..."
>>
>>
>>Today's Topics:
>>
>> 1. Re: Good approach regarding classes attributes (Sydney Shall)
>> 2. Re: Good approach regarding classes attributes (leam hall)
>> 3. Re: Understand subprocess poll (Wolfgang Maier)
>> 4. Re: Good approach regarding classes attributes (Peter Otten)
>> 5. Re: Good approach regarding classes attributes (Juan Christian)
>> 6. Re: Good approach regarding classes attributes (Sydney Shall)
>>
>>
>>----------------------------------------------------------------------
>>
>>Message: 1
>>Date: Tue, 09 Sep 2014 15:09:32 +0200
>>From: Sydney Shall <s.shall at virginmedia.com>
>>To: tutor at python.org
>>Subject: Re: [Tutor] Good approach regarding classes attributes
>>Message-ID: <540EFC0C.3000202 at virginmedia.com>
>>Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>>
>>On 08/09/2014 18:39, Alan Gauld wrote:
>>> On 08/09/14 15:17, Juan Christian wrote:
>>>
>>> One tiny tweak...
>>>
>>>> class User():
>>>
>>> You don't need the parens after User. You don;t have any superclasses
>>> so they do nothing. Python convention for an empty parent list is just
>>> to leave the parens off:
>>>
>>> class User:
>>>
>>A simple question from a newbie, in response to this surprise.
>>Is it not helpful to always put (object) as the parent, if the class is
>>not itself a sub-class?
>>And while I am writing, what does OP stand for in this list?
>>
>>
>>--
>>Sydney Shall
>>-------------- next part --------------
>>An HTML attachment was scrubbed...
>>URL: <http://mail.python.
>org/pipermail/tutor/attachments/20140909/ebcdb792/attachment-0001.html>
>>
>>------------------------------
>>
>>Message: 2
>>Date: Tue, 9 Sep 2014 09:14:39 -0400
>>From: leam hall <leamhall at gmail.com>
>>To: tutor <tutor at python.org>
>>Subject: Re: [Tutor] Good approach regarding classes attributes
>>Message-ID:
>> <CACv9p5qq=PSmxPaYTZ3iCrgnWb3B8aWBV1X_WAHYfC=OA6+THA at mail.gmail.com>
>>Content-Type: text/plain; charset=UTF-8
>>
>>On Tue, Sep 9, 2014 at 9:09 AM, Sydney Shall <s.shall at virginmedia.com>
wrote:
>>
>>> And while I am writing, what does OP stand for in this list?
>>
>>"Original Poster". So I understand. Won't answer the Python question
>>since I'm a newbie here myself.
>>
>>--
>>Mind on a Mission
>>
>>
>>------------------------------
>>
>>Message: 3
>>Date: Tue, 09 Sep 2014 15:05:11 +0200
>>From: Wolfgang Maier <wolfgang.maier at biologie.uni-freiburg.de>
>>To: tutor at python.org
>>Subject: Re: [Tutor] Understand subprocess poll
>>Message-ID: <540EFB07.5050901 at biologie.uni-freiburg.de>
>>Content-Type: text/plain; charset=utf-8; format=flowed
>>
>>On 09/09/2014 11:45 AM, Peter Otten wrote:
>>> jarod_v6 at libero.it wrote:
>>>
>>>> I want to use subprocess for run some programs But I need to be sure the
>>>> program end before continue with the other:
>>>>
>>>> subprocess.call("ls")
>>>> cmd1 = i
>>>> p1 = subprocess.Popen(cmd1,shell=True,stdout=subprocess.PIPE)
>>>>
>>>> while True:
>>>> if p1.poll() is None:
>>>> time.sleep(3)
>>>>
>>>> pass
>>>> if p1.poll()==0:
>>>> print '#'
>>>> break
>>>> if p1.poll() is not None and p1.poll() != 0:
>>>> raise Exception('Error building Alignment using star with hg19
>>>> database')
>>>
>>>> This are not working. How can I do?
>>>> thanks in advance for the precious help
>>>> bw,
>>>
>>> I don't understand why you would need this loop. Why don't you use
>>> subprocess.call() and be done?
>>>
>>
>>The OP is piping the process stdout so I assume he is going to read from
>>it in place of the pass in his example.
>>Since the subprocess is doing genome-wide sequence alignment (at least I
>>guess so from the exception string) there will be lots of output, which
>>would cause subprocess.call() to block.
>>
>>Assuming that the posted code was indented correctly and was otherwise
>>run as posted this could also be the answer to the original question:
>>you have to keep on consuming data from the pipe or its buffer is going
>>to fill up and block everyhing. With a simple pass statement you do not
>>achieve anything that you can't do with call.
>>
>>Wolfgang
>>
>>
>>
>>------------------------------
>>
>>Message: 4
>>Date: Tue, 09 Sep 2014 15:44:22 +0200
>>From: Peter Otten <__peter__ at web.de>
>>To: tutor at python.org
>>Subject: Re: [Tutor] Good approach regarding classes attributes
>>Message-ID: <lun07o$e8n$1 at ger.gmane.org>
>>Content-Type: text/plain; charset="ISO-8859-1"
>>
>>Sydney Shall wrote:
>>
>>> On 08/09/2014 18:39, Alan Gauld wrote:
>>>> On 08/09/14 15:17, Juan Christian wrote:
>>>>
>>>> One tiny tweak...
>>>>
>>>>> class User():
>>>>
>>>> You don't need the parens after User. You don;t have any superclasses
>>>> so they do nothing. Python convention for an empty parent list is just
>>>> to leave the parens off:
>>>>
>>>> class User:
>>>>
>>> A simple question from a newbie, in response to this surprise.
>>> Is it not helpful to always put (object) as the parent, if the class is
>>> not itself a sub-class?
>>
>>The answer differs between Python 2 and 3. In Python 3
>>
>>class C: # preferred in Python 3
>> pass
>>
>>and
>>
>>class C(object):
>> pass
>>
>>are the same, so there is no point adding the explicit object inheritance.
>>
>>In Python 2 however
>>
>>class C:
>> pass
>>
>>will create a "classic class" whereas
>>
>>class C(object): # preferred in Python 2
>> pass
>>
>>is a "newstyle class". The most notable difference between these is that
>>properties work correctly only with newstyle classes. Therefore making all
>>your classes "newstyle" is a good idea.
>>
>>> And while I am writing, what does OP stand for in this list?
>>
>>Original Poster, as Leam says.
>>
>>
>>
>>
>>------------------------------
>>
>>Message: 5
>>Date: Tue, 9 Sep 2014 10:54:22 -0300
>>From: Juan Christian <juan0christian at gmail.com>
>>Cc: "tutor at python.org" <tutor at python.org>
>>Subject: Re: [Tutor] Good approach regarding classes attributes
>>Message-ID:
>> <CAAp0bGvBVZD3OGSv4dE7q4J4YYSd1WKrXFbSovNrg-3xyFGWKA at mail.gmail.com>
>>Content-Type: text/plain; charset="utf-8"
>>
>>On Mon, Sep 8, 2014 at 5:58 AM, Peter Otten <__peter__ at web.de <javascript:;
>>
>>wrote:
>>
>>>
>>> PS: This is not about being pythonic, but it might be more convenient for
>>> client code if you use datetime objects instead of timestamps:
>>>
>>> >>> import datetime
>>> >>> last_logoff = datetime.datetime.utcfromtimestamp(1410065399)
>>> >>> print(last_logoff)
>>> 2014-09-07 04:49:59
>>>
>>
>>Yes, I'll do it for sure, the API response is indeed returned that way to
>>make things easier.
>>-------------- next part --------------
>>An HTML attachment was scrubbed...
>>URL: <http://mail.python.
>org/pipermail/tutor/attachments/20140909/aacbe49f/attachment-0001.html>
>>
>>------------------------------
>>
>>Message: 6
>>Date: Tue, 09 Sep 2014 16:02:01 +0200
>>From: Sydney Shall <s.shall at virginmedia.com>
>>To: tutor at python.org
>>Subject: Re: [Tutor] Good approach regarding classes attributes
>>Message-ID: <540F0859.5070802 at virginmedia.com>
>>Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>>
>>On 09/09/2014 15:44, Peter Otten wrote:
>>> Sydney Shall wrote:
>>>
>>>> On 08/09/2014 18:39, Alan Gauld wrote:
>>>>> On 08/09/14 15:17, Juan Christian wrote:
>>>>>
>>>>> One tiny tweak...
>>>>>
>>>>>> class User():
>>>>> You don't need the parens after User. You don;t have any superclasses
>>>>> so they do nothing. Python convention for an empty parent list is just
>>>>> to leave the parens off:
>>>>>
>>>>> class User:
>>>>>
>>>> A simple question from a newbie, in response to this surprise.
>>>> Is it not helpful to always put (object) as the parent, if the class is
>>>> not itself a sub-class?
>>> The answer differs between Python 2 and 3. In Python 3
>>>
>>> class C: # preferred in Python 3
>>> pass
>>>
>>> and
>>>
>>> class C(object):
>>> pass
>>>
>>> are the same, so there is no point adding the explicit object inheritance.
>>>
>>> In Python 2 however
>>>
>>> class C:
>>> pass
>>>
>>> will create a "classic class" whereas
>>>
>>> class C(object): # preferred in Python 2
>>> pass
>>>
>>> is a "newstyle class". The most notable difference between these is that
>>> properties work correctly only with newstyle classes. Therefore making all
>>> your classes "newstyle" is a good idea.
>>>
>>>> And while I am writing, what does OP stand for in this list?
>>> Original Poster, as Leam says.
>>>
>>>
>>> _______________________________________________
>>> Tutor maillist - Tutor at python.org
>>> To unsubscribe or change subscription options:
>>> https://mail.python.org/mailman/listinfo/tutor
>>>
>>Thanks Peter, most helpful.
>>I was taught with Python 2.7, so now I understand the advice.
>>
>>
>>--
>>Sydney Shall
>>-------------- next part --------------
>>An HTML attachment was scrubbed...
>>URL: <http://mail.python.
>org/pipermail/tutor/attachments/20140909/502cb8e0/attachment.html>
>>
>>------------------------------
>>
>>Subject: Digest Footer
>>
>>_______________________________________________
>>Tutor maillist - Tutor at python.org
>>https://mail.python.org/mailman/listinfo/tutor
>>
>>
>>------------------------------
>>
>>End of Tutor Digest, Vol 127, Issue 26
>>**************************************
>>
>
>
>
>
>------------------------------
>
>Message: 4
>Date: Tue, 09 Sep 2014 17:05:46 +0200
>From: Sydney Shall <s.shall at virginmedia.com>
>To: tutor at python.org
>Subject: Re: [Tutor] Good approach regarding classes attributes
>Message-ID: <540F174A.1000709 at virginmedia.com>
>Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>
>On 09/09/2014 16:05, Joel Goldstick wrote:
>> On Tue, Sep 9, 2014 at 10:02 AM, Sydney Shall <s.shall at virginmedia.com>
wrote:
>>> On 09/09/2014 15:44, Peter Otten wrote:
>>>
>>> Sydney Shall wrote:
>>>
>>> On 08/09/2014 18:39, Alan Gauld wrote:
>>>
>>> On 08/09/14 15:17, Juan Christian wrote:
>>>
>>> One tiny tweak...
>>>
>>> class User():
>>>
>>> You don't need the parens after User. You don;t have any superclasses
>>> so they do nothing. Python convention for an empty parent list is just
>>> to leave the parens off:
>>>
>>> class User:
>>>
>>> A simple question from a newbie, in response to this surprise.
>>> Is it not helpful to always put (object) as the parent, if the class is
>>> not itself a sub-class?
>>>
>>> The answer differs between Python 2 and 3. In Python 3
>>>
>>> class C: # preferred in Python 3
>>> pass
>>>
>>> and
>>>
>>> class C(object):
>>> pass
>>>
>>> are the same, so there is no point adding the explicit object inheritance.
>>>
>>> In Python 2 however
>>>
>>> class C:
>>> pass
>>>
>>> will create a "classic class" whereas
>>>
>>> class C(object): # preferred in Python 2
>>> pass
>>>
>>> is a "newstyle class". The most notable difference between these is that
>>> properties work correctly only with newstyle classes. Therefore making all
>>> your classes "newstyle" is a good idea.
>>>
>>> And while I am writing, what does OP stand for in this list?
>>>
>>> Original Poster, as Leam says.
>>>
>>>
>>> _______________________________________________
>>> Tutor maillist - Tutor at python.org
>>> To unsubscribe or change subscription options:
>>> https://mail.python.org/mailman/listinfo/tutor
>>>
>>> Thanks Peter, most helpful.
>>> I was taught with Python 2.7, so now I understand the advice.
>>>
>>>
>>> --
>>> Sydney Shall
>>>
>>> _______________________________________________
>>> Tutor maillist - Tutor at python.org
>>> To unsubscribe or change subscription options:
>>> https://mail.python.org/mailman/listinfo/tutor
>>>
>> Please post in plain text
>>
>My apologies. I thought I was. I will immediately change it.
>
>--
>Sydney Shall
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: <http://mail.python.
org/pipermail/tutor/attachments/20140909/93d19a3d/attachment.html>
>
>------------------------------
>
>Subject: Digest Footer
>
>_______________________________________________
>Tutor maillist - Tutor at python.org
>https://mail.python.org/mailman/listinfo/tutor
>
>
>------------------------------
>
>End of Tutor Digest, Vol 127, Issue 27
>**************************************
>
More information about the Tutor
mailing list