Mailman 3 May 2012 - Python-ideas

Re: [Python-ideas] shutil.run no security thread
by anatoly techtonik 23 May '12

23 May '12

Ok, let's separately discuss shutil.run() added value without touching security at all (subj changed). Is it ok? Is it nice idea? Would it be included in stdlib in an ideal world where security implications doesn't matter? -- anatoly t. On Tue, May 22, 2012 at 11:30 PM, Mike Meyer <mwm(a)mired.org> wrote: > On Tue, 22 May 2012 18:39:16 +0300 > anatoly techtonik <techtonik(a)gmail.com> wrote: > >> Therefore, inspired by Fabric API, I've finally found the solution - >> shutil.run() function: >> https://bitbucket.org/techtonik/shutil-run/src >> >> run(command, combine_stderr=True): >> >> Run command through a system shell, return output string with >> additional properties: >> >> output.succeeded - result of the operation True/False >> output.return_code - specific return code >> output.stderr - stderr contents if combine_stderr=False >> >> `combine_stderr` if set, makes stderr merged into output string, >> otherwise it will be available as `output.stderr` attribute. > [...] >> That's the most intuitive way I found so far. Objective advantages: >> >> 1. Better than >> subprocess.call(cmd, shell=true) >> subprocess.check_call(cmd, shell=true) >> subprocess.check_output(cmd, shell=True) >> because it is just >> shutil.run(cmd) >> i.e. short, simple and _easy to remember_ > > -2 > > Unless there's some way to turn off shell processing (better yet, have > no shell processing be the default, and require that it be turned on), > it can't be used securely with tainted strings, so it should *not* be > used with tainted strings, which means it's pretty much useless in any > environment where security matters. With everything being networked, > there may no longer be any such environments. > >> 3. shutil.run() is predictable and consistent - its arguments are not >> dependent on each other, their combination doesn't change the function >> behavior over and over requiring you iterate over the documentation >> and warnings again and again > > As proposed, it certainly provides a predictable and consistent > vulnerability to code injection attacks. > >> 4. shutil.run() is the correct next level API over subprocess base >> level. subprocess executes external process - that is its role, but >> automatic ability to execute external process inside another external >> process (shell) looks like a hack to me. Practical, but still a hack. > > It's only correct if you are in an environment where you don't care > about security. If you care about security, you can't use it. If we're > going to add yet another system() replacement, let's at least try and > make it secure. > > <mike > -- > Mike Meyer <mwm(a)mired.org> http://www.mired.org/ > Independent Software developer/SCM consultant, email for more information. > > O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

3 2

processing subprocess output line-by-line (Was: shutil.run)
by anatoly techtonik 23 May '12

23 May '12

On Wed, May 23, 2012 at 10:09 AM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > On Wed, May 23, 2012 at 4:49 PM, Devin Jeanpierre > <jeanpierreda(a)gmail.com> wrote: >> On Tue, May 22, 2012 at 5:41 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: >>> Having dealt with long running child processes lately, I can also say that >>> producing output line-by-line would be on my personal list of requirements. >> >> You can do that with subprocess, right? Just have to be sure to close >> stdin/stderr and read p.stdout with readline() repeatedly... > > Yep, subprocess is a swiss army knife - you can do pretty much > anything with it. That's the complaint, though - *because* it's so > configurable, even the existing convenience APIs aren't always that > convenient for simple operations. It is quite likely that there are use cases where subprocess fails, \ because they require async control. http://bugs.python.org/issue14872 And line-by-line recipe is here: http://stackoverflow.com/questions/5582933/need-to-avoid-subprocess-deadloc… -- anatoly t.

1 0

Important fixes not getting to releases
by anatoly techtonik 23 May '12

23 May '12

I know why important usability features are not getting into releases - they are taken care too late in release cycle and it all starts with bug tracker which imposes the workflow where usability bugs are always an enhancement. http://bugs.python.org/issue14872 -- anatoly t.

1 0

Run attached Python tests in browser
by anatoly techtonik 23 May '12

23 May '12

I am not sure if it belongs here, to python-dev or infrastructure. Lately I've been looking at http://repl.it/ and found it to be pretty convenient to code stuff that otherwise require a Python editor to be installed. So, I thought that it might be actually convenient to use for automatically testing patches in Python bugtracker without going through the hassle to download, patch and run everything locally. Of course, not everything will work, but at least some parts of it could be. -- anatoly t.

1 0

weakrefs
by Ethan Furman 19 May '12

19 May '12

From the manual [8.11]: > A weak reference to an object is not enough to keep the object alive: > when the only remaining references to a referent are weak references, > garbage collection is free to destroy the referent and reuse its > memory for something else. This leads to a difference in behaviour between CPython and the other implementations: CPython will (currently) immediately destroy any objects that only have weak references to them with the result that trying to access said object will require making a new one; other implementations (at least PyPy, and presumably the others that don't use ref-count gc's) can "reach into the grave" and pull back objects that don't have any strong references left. I would like to have the guarantees for weakrefs strengthened such that any weakref'ed object that has no strong references left will return None instead of the object, even if the object has not yet been garbage collected. Without this stronger guarantee programs that are relying on weakrefs to disappear when strong refs are gone end up relying on the gc method instead, with the result that the program behaves differently on different implementations. ~Ethan~

7 8

get method for sets?
by Mike Meyer 17 May '12

17 May '12

Is there some reason that there isn't a straightforward way to get an element from a set without removing it? Everything I find either requires multiple statements or converting the set to another data type. It seems that some kind of get method would be useful. The argument that "getting an arbitrary element from a set isn't useful" is refuted by 1) the existence of the pop method, which does just that, and 2) the fact that I (and a number of other people) have run into such a need. My search for such a reason kept finding people asking how to get an element instead. Of course, my key words (set and get) are heavily overloaded. thanks, <mike -- Mike Meyer <mwm(a)mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

19 40

Add `future_builtins` as an alias for `builtins`
by Sven Marnach 17 May '12

17 May '12

With the reintroduction of u"Unicode literals", Python 3.3 will remove one of the major stumbling stones for supporting Python 2.x and 3.3 within the same code base. Another rather trivial stumbling stone could be removed by adding the alias `future_builtins` for the `builtins` module. Currently, you need to use a try/except block, which isn't too bad, but I think it would be nicer if a line like from future_builtins import map continues to work, just like __future__ imports continue to work. I think the above actually *is* a kind of __future__ report which just happens to be in a regular module because it doesn't need any special compiler support. I know a few module names changed and some modules have been reorganised to packages, so you will still need try/except blocks for other imports. However, I think `future_builtins` is special because it's sole raison d'être is forward-compatibility and becuase of the analogy with `__future__`. Cheers, Sven

7 7

input function: built-in space between string and user-input
by Suriaprakash.Mariappan＠smsc.com 17 May '12

17 May '12

print function: built-in space between string and variable: The below python code, length = 5 print('Length is', length) gives an output of Length is 5 Even though we have not specified a space between 'Length is' and the variable length, Python puts it for us so that we get a clean nice output and the program is much more readable this way (since we don't need to worry about spacing in the strings we use for output). This is surely an example of how Python makes life easy for the programmer. input function: built-in space between string and user-input: However, the below python code, guess = int(input('Enter an integer')) gives an output of Enter an integer7 [Note: Assume 7 is entered by the user.] Suggestion: Similar to the printf function, for the input function also, it will be nice to have the Python put a space between string and user-input, so that the output in the above case will be more readable as below. Enter an integer 7 Thanks and Regards, Suriaprakash M, Principal Engineer - Software, Standard Microsystems India Pvt. Ltd., Module 1, 4th Floor, Block A, SP Infocity, #40, MGR Salai, Perungudi, Chennai - 600 096, Tamil Nadu, INDIA. Email: Suriaprakash.Mariappan(a)smsc.com Mobile :+919381453832 Skype ID: msuriaprakash

4 4

Re: [Python-ideas] [Python-Dev] sys.implementation
by Eric Snow 15 May '12

15 May '12

At this point I'm pretty comfortable with where PEP 421 is at. Before asking for pronouncement, I'd like to know if anyone has any outstanding concerns that should be addressed first. The only (relatively) substantial point of debate has been the type for sys.implementation. The PEP now limits the specification of the type to the minimum (Big-Endian vs. Little...er...attribute-access vs mapping). If anyone objects to the decision there to go with attribute-access, please make your case. >From my point of the view either one would be fine for what we need and attribute-access is more representative of the fixed namespace. Unless there is a really good reason to use a mapping, I'd like to stick with that. Thanks! -eric

1 0

Unhelpful error message from sorted
by Michael Foord 15 May '12

15 May '12

Hello all, It seems to me that the following error message, whilst technically correct, is unhelpful: >>> sorted([3, 2, 1], reverse=None) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: an integer is required Worth creating an issue for? Michael -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

6 6