re-sending with subject :)
On Mon, Dec 12, 2011 at 2:44 PM, Eric Snow
Guido posted this on Google+:
IEEE/ISO are working on a draft document about Python vulunerabilities: http://grouper.ieee.org/groups/plv/DocLog/300-399/360-thru-379/22-WG23-N-037... (in the context of a larger effort to classify vulnerabilities in all languages: ISO/IEC TR 24772:2010, available from ISO at no cost at: http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html (its link is near the bottom of the web page).
Will this document have a broad use, such that we should make sure it is accurate (to avoid any future confusion)? I skimmed through and found that it covers a lot of ground, not necessarily about vulnerabilities, with some inaccuracies but not a ton that I noticed. If it doesn't matter then no big deal. Just thought I'd bring it up.
-eric
IEEE/ISO are working on a draft document about Python vulunerabilities: http://grouper.ieee.org/groups/plv/DocLog/300-399/360-thru-379/22-WG23-N-037... (in the context of a larger effort to classify vulnerabilities in all languages: ISO/IEC TR 24772:2010, available from ISO at no cost at: http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html (its link is near the bottom of the web page).
Random comments. I didn't read everything. -- "Vulnerability descriptions for the language Python Standards and terminology based on the 3.x standard only." (...) "Automatic conversion also occurs when an integer becomes too large to fit within the constraints of the large integer specified in the language (typically C) used to create the Python interpreter. On a 32‐bit machine this would be the range ‐2^30 to 2^30‐1. When an integer becomes too large to fit into that range it is converted to an extended precision integer of arbitrary length." (...) "otherwise, if either argument is a floating point number, the other is converted to floating otherwise, if either argument is a long integer, the other is converted to long integer;" 10 and 2**1024 have the same type (int) in Python 3. I don't really understand what "extended precision" means. There are no more "long" integers. -- "Python.16 Wrap‐around Error [XYY]" (...) "... exception handling for floating point operations cannot be assumed to catch this type of error because they are not standardized in the underlying C language." Can you give me an example of such problem? If there is really an issue, can we configure the FPU to catch such error? pyfpe.h has PyFPE_START_PROTECT and PyFPE_END_PROTECT macros, but they do nothing by default. You can to enable this protection using ./configure --with-fpectl. -- "if(y > 0):print(x)" Even if this example is valid, it is surprising to see parenthesis around the condition in Python. "if y > 0: print(x)" or even "if y > 0: print(x)" would be better. -- "Python also encourages structured programming by not introducing any of the following constructs which could easily lead to unstructured code: - Labels and branching statements such as GO TO; - Case, GO TO DEPENDING, EVALUATE, switch and other statements that branch dependent on a variable’s value; and - ALTER which changes GO TO label to branch to a different label." You have to modify the language (and so build your own interpreter) to add a "goto" instruction to Python. Or do you mean that someone may want to implement something like goto using exceptions for example? -- "When sorting a list using the sort() method, attempting to inspect or mutate the content of the list will result in undefined behaviour." Oh... I never imagined such "use case". Let's try: $ ./python Python 3.3.0a0 (default:3ad7d01acbf4+, Dec 12 2011, 21:07:55)
def hack(x): ... mylist.append(10) ... return ... mylist=[1] mylist.sort(key=hack) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: list modified during sort
Same behaviour with Python 2.7 and 3.2: so the Python behaviour is defined, you get a ValueError. Are there other ways to inspect or mutate a list while sorting it? -- "The sequence of keys in a dictionary is undefined because the hashing function used to index the keys is unspecified therefore different implementations are likely to yield different sequences." Exact. You might mention that collections.OrderedDict has a defined behaviour: it lists keys (and values) in the insertion order. -- "Mixing tabs and spaces to indent is defined differently for UNIX and non‐UNIX platforms;" You can use the -tt command line option to raise an IndentationError (a block can still be indented using spaces and tabs). Victor
2011/12/12 Victor Stinner
"When sorting a list using the sort() method, attempting to inspect or mutate the content of the list will result in undefined behaviour."
But is this even true? in listobject.c::listsort(), since 2002, /* The list is temporarily made empty, so that mutations performed * by comparison functions can't affect the slice of memory we're * sorting (allowing mutations during sorting is a core-dump * factory, since ob_item may change). */ So behaviour is not undefined at all... maybe this report is only based on note #10 of the documentation: http://docs.python.org/library/stdtypes.html#mutable-sequence-types and only considers python 2.2 or older... -- Amaury Forgeot d'Arc
Hi,
On Tue, Dec 13, 2011 at 11:37, Amaury Forgeot d'Arc
"When sorting a list using the sort() method, attempting to inspect or mutate the content of the list will result in undefined behaviour."
(...) So behaviour is not undefined at all...
No, the behavior _is_ undefined. The comment you cited says that it cannot crash the Python interpreter; additionally, it makes a best-effort attempt at catching such accesses and raising ValueError. But I think I can build a strange-looking example where you mutate a list during sorting and don't get a ValueError (although admittedly it needs a lot of hacking to do that nowadays, e.g. multiple threads). A bientôt, Armin.
2011/12/13 Armin Rigo
No, the behavior _is_ undefined. The comment you cited says that it cannot crash the Python interpreter; additionally, it makes a best-effort attempt at catching such accesses and raising ValueError. But I think I can build a strange-looking example where you mutate a list during sorting and don't get a ValueError (although admittedly it needs a lot of hacking to do that nowadays, e.g. multiple threads).
I'm interested to see how! The current implementation installs an empty array in the list, and the initial array is only held by a local variable in listsort(). even gc.get_referrers() can return the empty list... -- Amaury Forgeot d'Arc
participants (4)
-
Amaury Forgeot d'Arc -
Armin Rigo -
Eric Snow -
Victor Stinner