Mailman 3 June 2005 - Python-Dev

subprocess.call() and stdin
by Stuart Bishop June 22, 2005

June 22, 2005

Redirecting to python-dev for discussion. When I invoke subprocess.call(), I often want to ensure that the subprocess' stdin is closed. This ensures it will die if the subprocess attempts to read from stdin rather than block. This could be done if the subprocess.call() helper closes the input if stdin=subprocess.PIPE, which may be the only sane way to handle this argument (I can't think of any use cases for spawning a subprocess with an input stream that nothing can write too). It could also be done by adding a subprocess.CLOSED constant, which if passed to Popen causes a new closed file descriptor to be given to the subprocess. -------- Original Message -------- Bugs item #1220113, was opened at 2005-06-14 15:04 Message generated for change (Comment added) made by zenzen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1220113&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Resolution: Rejected Priority: 5 Submitted By: Stuart Bishop (zenzen) Assigned to: Peter Åstrand (astrand) Summary: subprocess call() helper should close stdin if PIPE Initial Comment: The following code snippet should die instead of hang. >>> from subprocess import call, PIPE >>> rv = call(['/usr/bin/bzip2', '-c'], stdout=PIPE, stdin=PIPE) It makes no sense not to close stdin if it is PIPE because the stream cannot be accessed. The use case for this is ensuring a subprocess that detects if it is connected to a terminal or not runs in 'batch' mode, and that it will die instead of hang if it unexpectidly attempts to read from stdin. Workaround is to use Popen instead. ---------------------------------------------------------------------- >Comment By: Stuart Bishop (zenzen) Date: 2005-06-22 16:12 Message: Logged In: YES user_id=46639 I can't think of any uses cases for wanting to create an inaccessible pipe and give it to the child. Wanting to pass a closed file handle is common. It is needed when calling a program that behaves differently if its stdin is a terminal or not. Or when you simply would prefer the subprocess to die if it attempts to read from its stdin rather than block. Using Popen instead of call is s simpler workaround than creating and closing a file descriptor and passing it in. Perhaps what is needed is a new constant, subprocess.CLOSED which creates a new file descriptor and closes it? This would be useful for Popen too, allowing call() to remain a think and trivially documented wrapper? ---------------------------------------------------------------------- Comment By: Peter Åstrand (astrand) Date: 2005-06-22 02:08 Message: Logged In: YES user_id=344921 >It makes no sense not to close stdin if it is PIPE >because the stream cannot be accessed True, but what if you actually *want* to create an inaccessible pipe, and give it to the child? Currently, the call() wrapper is *very* short and simple. I think this is very good. For example, it allows us to document it in two sentences. You get what you ask for: If you use call() with strange arguments, you'll get a somewhat strange behavíour. I see no point in introducing lots of sanity checks in the wrapper functions. >The use case for this is ensuring a subprocess that >detects if it is connected to a terminal or not runs in >batch' mode, and that it will die instead of hang if >it unexpectidly attempts to read from stdin I'm not sure I understand what you want, but if you want to have stdin connected to a closed file descriptor, just pass one: >>> from subprocess import call, PIPE >>> rv = call(['/usr/bin/bzip2', '-c'], stdout=PIPE, stdin=4711) (Of course, you should verify that 4711 is unused.) If you don't agree with me, post to python-dev for discussion. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1220113&group_… -- Stuart Bishop <stuart(a)stuartbishop.net> http://www.stuartbishop.net/

1 0

Recommend accepting PEP 312 -- Simple Implicit Lambda
by Raymond Hettinger June 22, 2005

June 22, 2005

This PEP is an excellent example of improving readability and usability by omitting a keyword and simplifying syntax. It neither provides nor takes away functionality; instead, it is a bit of a beautification effort. Essentially it creates a lighter-weight, more usable syntax for specifying deferred function arguments. Like decorators, this helps out programmers who understand and use this technique, and it is harmless and invisible for the rest. It is especially handy in the absence of an if-then-else expression. The PEP is backwards compatible. Recommend accepting just the basic PEP which only targets simple, obvious cases. The discussed extensions are unattractive and should be skipped. Raymond

10 26

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser June 22, 2005

June 22, 2005

Patch / Bug Summary ___________________ Patches : 338 open ( +0) / 2866 closed ( +5) / 3204 total ( +5) Bugs : 914 open ( +5) / 5060 closed (+13) / 5974 total (+18) RFE : 188 open ( +0) / 170 closed ( +0) / 358 total ( +0) New / Reopened Patches ______________________ update the binhex module for Mach-O (2005-06-14) http://python.org/sf/1220874 opened by Bob Ippolito ftplib storbinary callback (2005-06-15) http://python.org/sf/1221598 opened by Phil Schwartz C++ compilation support for distutils (2005-06-17) http://python.org/sf/1222585 opened by Antti Honkela PEP 342/343 Generator enhancements (2005-06-18) http://python.org/sf/1223381 opened by Phillip J. Eby HP-UX ia64 64-bit Python executable (2005-06-21) http://python.org/sf/1225212 opened by Peter Kropf Patches Closed ______________ _csv.c leaks when dialect creation fails (2005-06-14) http://python.org/sf/1220242 closed by montanaro test_curses bugs (2004-08-09) http://python.org/sf/1005892 closed by akuchling no html file for modulefinder (2005-04-10) http://python.org/sf/1180012 closed by birkenfeld Restore GC support to set objects (2005-05-11) http://python.org/sf/1200018 closed by rhettinger change recall in IDLE shell to not overwrite current command (2005-05-06) http://python.org/sf/1196917 closed by kbk New / Reopened Bugs ___________________ buffer() object broken. (2005-06-15) CLOSED http://python.org/sf/1221424 opened by Ray Schumacher Incorrect documentation for InPlace Number operators (2005-06-15) CLOSED http://python.org/sf/1221477 opened by Daniel Stutzbach float() not accurate (2005-06-16) CLOSED http://python.org/sf/1222098 opened by Brian Dols Bad optparse help wrapping with multiple paragraphs (2005-06-16) http://python.org/sf/1222235 opened by Barry A. Warsaw Typo in section 5.3 of tutorial (2005-06-17) CLOSED http://python.org/sf/1222459 opened by uman tk + setlocale problems... (2005-06-17) http://python.org/sf/1222721 opened by Alex A. Naanou SimpleXMLRPCServer does not set FD_CLOEXEC (2005-06-17) http://python.org/sf/1222790 opened by Winfried Harbecke it's -> its (2005-06-17) CLOSED http://python.org/sf/1222928 opened by Ed Swierk 2.4.1 .msi file won't install to different disk (2005-06-17) CLOSED http://python.org/sf/1222978 opened by Ray Trent race in os.makedirs() (2005-06-18) http://python.org/sf/1223238 opened by Mattias Engdegård subprocess.py abuse of errno (2005-06-20) http://python.org/sf/1223937 reopened by astrand subprocess.py abuse of errno (2005-06-20) http://python.org/sf/1223937 opened by Oren Tirosh error locale.getlocale() with LANGUAGE=eu_ES (2005-06-20) http://python.org/sf/1223976 opened by Zunbeltz Izaola Len too large with national characters (2005-06-20) CLOSED http://python.org/sf/1224047 opened by Henrik Winther Jensen int/long unification and hex() (2005-06-20) http://python.org/sf/1224347 opened by Josiah Carlson tokenize module does not detect inconsistent dedents (2005-06-21) CLOSED http://python.org/sf/1224621 opened by Danny Yoo rsplit introduces spurious CR (2005-06-21) http://python.org/sf/1224672 opened by Richard Prosser Line endings problem with Python 2.4.1 cause imports to fail (2005-06-21) http://python.org/sf/1225059 opened by Nicolas Lehuen inspect.isclass() fails with custom __getattr__ (2005-06-21) http://python.org/sf/1225107 opened by Jeremy Kloth Bugs Closed ___________ float issue for NaN type in .pyc file (2004-12-07) http://python.org/sf/1080440 closed by mwh buffer() object broken. (2005-06-15) http://python.org/sf/1221424 closed by nascheme Incorrect documentation for InPlace Number operators (2005-06-15) http://python.org/sf/1221477 closed by rhettinger Replace MSVC memory allocator with ptmalloc2 (2005-06-07) http://python.org/sf/1216562 closed by loewis Sub threads execute in restricted mode (2005-03-15) http://python.org/sf/1163563 closed by mwh float() not accurate (2005-06-16) http://python.org/sf/1222098 closed by nascheme Typo in section 5.3 of tutorial (2005-06-17) http://python.org/sf/1222459 closed by rhettinger it's -> its (2005-06-17) http://python.org/sf/1222928 closed by nascheme 2.4.1 .msi file won't install to different disk (2005-06-17) http://python.org/sf/1222978 closed by hacksoncode spurious blank page in dist.pdf (2005-05-27) http://python.org/sf/1209560 closed by birkenfeld subprocess.py abuse of errno (2005-06-20) http://python.org/sf/1223937 closed by astrand Len too large with national characters (2005-06-20) http://python.org/sf/1224047 closed by mwh tokenize module does not detect inconsistent dedents (2005-06-21) http://python.org/sf/1224621 closed by rhettinger subprocess call() helper should close stdin if PIPE (2005-06-14) http://python.org/sf/1220113 closed by astrand

1 0

Propose updating PEP 284 -- Integer for-loops
by Raymond Hettinger June 21, 2005

June 21, 2005

I recommend that the proposed syntax be altered to be more parallel with the existing for-loop syntax to make it more parsable for both humans and for the compiler. Like existing for-statements, the target expression should immediately follow the 'for' keyword. Since this is known to be a range assignment, only an expression is needed, not a full expression list. Immediately following should be a token to distinguish the new and old syntaxes. Putting that distinction early in the statement prepares readers (who scan left-to-right) for what follows. IOW, augment the existing syntax: for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite] with an alternative syntax in the form: for_stmt: 'for' expr 'tok' rangespec ':' suite ['else' ':' suite] Given that form, the PEP authors can choose the best options for 'tok'. Basically, anything will do as long as it is not 'in'. Likewise, they can choose any rangespec format. Within that framework, there are many possibilities: for i between 2 < i <= 10: ... for i over 2 < i <= 10: ... # chained comparison style for i over [2:11]: ... # Slice style for i = 3 to 10: ... # Basic style The rangespecs with comparison operators offer more flexibility in terms of open/closed intervals. In contrast, the slice notation version and the Basic versions can admit a step argument. The key changes are stating the target variable first and then using a specific token to distinguish between the two forms. Also, I recommend tightening the PEP's motivation. There are only two advantages, encoding and readability. The former is only a minor gain because all it saves is a function call, an O(1) savings in an O(n) context. The latter is where the real benefits lay. The PEP authors should also put to rest the issues section: * Decide once and for all on the simplest approach of immediately evaluating the whole rangespec prior to execution. This best parallels the range() approach and it is the least surprising. * Note that new proposal works equally well with list comps and genexps. * Decide to always return an iterator rather than a list as there is never an advantage to doing otherwise. * If you go for a chained comparison style rangespec, then drop the issue of a step argument. If the slice or Basic style rangespecs are chosen, then there is no reason not to allow a step argument. * Competition with PEP 276 is no longer an issue. * Simply disallow floating point bounds. We've already got deprecation warnings in place for Py2.5. There is no need to exacerbate the problem. Alternately, simplify the issue by declaring that the values will be handled as if by xrange(). The above recommendations should get the PEP ready for judgement day. Good luck. Raymond

6 6

Re: [Python-Dev] Explicitly declaring expected exceptions for ablock
by Michael Chermside June 21, 2005

June 21, 2005

Dmitry Dvoinikov writes: > The reason for that being self-tests with lots and lots of > little code snippets like this: > > try: > c().foo() > except TypeError: > pass Paul Du Boise already responded explaining that PEP 343 probably handles the task you want. I just wanted to mention that you may need to reconsider the task. The above snippet is almost certainly incorrect. I suspect that you wanted either: try: c().foo() fail('Should have raised TypeError') except TypeError: pass # expected or perhaps this: try: c().foo() except TypeError: fail('Should not have raised TypeError') There ARE situations when you want to allow an exception (but not necessarily expect it) and do nothing when it occurs, but I don't find them all that common, and I certainly don't find them arising in unit tests. -- Michael Chermside

2 2

Re: [Python-Dev] Explicitly declaring expected exceptions for a block
by Dmitry Dvoinikov June 21, 2005

June 21, 2005

> If you're trying to write tests, perhaps a better use-case would be > something like: > with required_exception(SomeError): > do something that should cause SomeError Yes, you are right, that'd be a better and more flexible way, thank you. Sincerely, Dmitry Dvoinikov http://www.targeted.org/ --- Original message follows --- > On 6/20/05, Dmitry Dvoinikov <dmitry(a)targeted.org> wrote: >> Excuse me if I couldn't find that in the existing PEPs, but >> wouldn't that be useful to have a construct that explicitly >> tells that we know an exception of specific type could happen >> within a block, like: >> ignore TypeError: >> do stuff >> [else: >> do other stuff] > If I understand PEP 343 correctly, it allows for easy implementation > of part of your request. It doesn't implement the else: clause, but > you don't give a use case for it either. > class ignored_exceptions(object): > def __init__(self, *exceptions): > self.exceptions = exceptions > def __enter__(self): > return None > def __exit__(self, type, value, traceback): > try: > raise type, value, traceback > except self.exceptions: > pass > with ignored_exceptions(SomeError): > do stuff > I don't see the use, but it's possible. >> The reason for that being self-tests with lots and lots of >> little code snippets like this: > If you're trying to write tests, perhaps a better use-case would be > something like: > with required_exception(SomeError): > do something that should cause SomeError > paul

1 0

Recommend accepting PEP 312 -- Simple Implicit Lambda
by Jim Jewett June 20, 2005

June 20, 2005

>>> lambda x,y: x+y*y >>> lambda x,y: y**2+x >>> are essentialy the same functions with different implementations [1]. >> Except that they are not. Think of __pow__, think of __add__ and __radd__. > You know the difference between the concept of a function and it's > infinitely many representations? That's why formal definitions exist. To clear up the archives, the problem is that Kay and Reinhold were talking about different things when they said "function". In arithmetic (and most sane extensions), those two lines are essentially the same function written different ways. In python those two lines represent two different functions, because x and y might not be (bound to) sane numbers. The variables could be bound to objects that redefine the addition, multiplication, and exponentiation operators. So y*y might not be the same as y**2, and x+y might not be the same as y+x. -jJ

1 0

Multiple interpreters not compatible with current thread module
by Jeremy Maxfield June 20, 2005

June 20, 2005

The current threadmodule.c does not seem to correctly support multiple (sub) interpreters. This became apparent when using jep - (a Java/Python bridge) and also seems to cause problems with mod_python. The incompatibility began in Python version 2.3.5 and has been traced to changes to the 2.4 threadmodule.c that were backported to the 2.3 delivery. A bug report was raised on 2005-03-15 (http://sourceforge.net/tracker/index.php?func=detail&aid=1163563&group_id=5…) which covers the problem in more detail. I've just submitted a patch (I hope it's correctly formatted) for threadmodule.c (http://sourceforge.net/tracker/index.php?func=detail&aid=1203393&group_id=5…) adapted from the pre-2.3.5 threadmodule.c (replacing the PyGILState_XXX calls with those from the earlier thread module). The patch works correctly but it will probably re-introduce the problem that the change for threadmodule.c version 2.59 fixed.("Fix for [ 1010677 ] thread Module Breaks PyGILState_Ensure(),and a test case.When booting a new thread, use the PyGILState API to manage the GIL."). The documentation (http://docs.python.org/api/threads.html) states "Note that the PyGILState_*() functions assume there is only one global interpreter (created automatically by Py_Initialize()). Python still supports the creation of additional interpreters (using Py_NewInterpreter()), but mixing multiple interpreters and the PyGILState_*() API is unsupported. ", so it looks like that using the PyGilState_XXX functions in the core threadmodule.c means the Py_NewInterpreter() call (i.e. multiple interpreters) is no longer supported when threads are involved. This problem is preventing us upgrading to Python 2.4 so we'd obviously like to see a resolution the next Python release if that were possible... Cheers, Max

2 2

Re: [Python-Dev] Multiple expression eval in compound if statement?
by Michael Chermside June 20, 2005

June 20, 2005

Gerrit Holl writes: > What would happen if... Raymond replies: > Every molecule in your body would simultaneously implode at the speed of > light. So you're saying it triggers C-language "undefined behavior"? -- Michael Chermside

1 0

Re: [Python-Dev] misplaced PEP
by Nick Jacobson June 20, 2005

June 20, 2005

Well, it's fixed now. Thanks to whomever took care of it. --- Nick Jacobson <nicksjacobson(a)yahoo.com> wrote: > At the www.python.org/peps page, PEP 281 is > erroneously listed in the "Finished PEPs (done, > implemented in CVS)" section. > > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

1 0