Python-Dev August 2012

python-dev@python.org

81 participants
76 discussions

Re: [Python-Dev] cpython (3.2): open() / TextIOWrapper doc: make it explicit than newline='\n' doesn't

by Serhiy Storchaka

On 04.08.12 02:27, victor.stinner wrote: > http://hg.python.org/cpython/rev/243ad1a6f638 > changeset: 78403:243ad1a6f638 > branch: 3.2 > parent: 78400:f19bea7bbee7 > user: Victor Stinner <victor.stinner(a)gmail.com> > date: Sat Aug 04 01:18:56 2012 +0200 > summary: > open() / TextIOWrapper doc: make it explicit than newline='\n' doesn't > translate newlines on output. > + " newline is '' or '\n', no translation takes place. If newline is any\n" Non-escaped "\n".

7 years, 11 months

Re: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): #11062: Fix adding a message from file to Babyl mailbox

by Andrew Svetlov

Looks like it is the source of buildbot fail on Windows box. On Wed, Aug 15, 2012 at 2:42 PM, petri.lehtinen <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/7c8c6b905a18 > changeset: 78586:7c8c6b905a18 > parent: 78583:8d90fde35cc6 > parent: 78585:cbc1dc8cda06 > user: Petri Lehtinen <petri(a)digip.org> > date: Wed Aug 15 14:36:14 2012 +0300 > summary: > #11062: Fix adding a message from file to Babyl mailbox > > files: > Lib/mailbox.py | 2 +- > Lib/test/test_mailbox.py | 18 ++++++------------ > Misc/NEWS | 2 ++ > 3 files changed, 9 insertions(+), 13 deletions(-) > > > diff --git a/Lib/mailbox.py b/Lib/mailbox.py > --- a/Lib/mailbox.py > +++ b/Lib/mailbox.py > @@ -1440,9 +1440,9 @@ > line = line[:-1] + b'\n' > self._file.write(line.replace(b'\n', linesep)) > if line == b'\n' or not line: > - self._file.write(b'*** EOOH ***' + linesep) > if first_pass: > first_pass = False > + self._file.write(b'*** EOOH ***' + linesep) > message.seek(original_pos) > else: > break > diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py > --- a/Lib/test/test_mailbox.py > +++ b/Lib/test/test_mailbox.py > @@ -152,20 +152,16 @@ > f.write(_bytes_sample_message) > f.seek(0) > key = self._box.add(f) > - # See issue 11062 > - if not isinstance(self._box, mailbox.Babyl): > - self.assertEqual(self._box.get_bytes(key).split(b'\n'), > - _bytes_sample_message.split(b'\n')) > + self.assertEqual(self._box.get_bytes(key).split(b'\n'), > + _bytes_sample_message.split(b'\n')) > > def test_add_binary_nonascii_file(self): > with tempfile.TemporaryFile('wb+') as f: > f.write(self._non_latin_bin_msg) > f.seek(0) > key = self._box.add(f) > - # See issue 11062 > - if not isinstance(self._box, mailbox.Babyl): > - self.assertEqual(self._box.get_bytes(key).split(b'\n'), > - self._non_latin_bin_msg.split(b'\n')) > + self.assertEqual(self._box.get_bytes(key).split(b'\n'), > + self._non_latin_bin_msg.split(b'\n')) > > def test_add_text_file_warns(self): > with tempfile.TemporaryFile('w+') as f: > @@ -173,10 +169,8 @@ > f.seek(0) > with self.assertWarns(DeprecationWarning): > key = self._box.add(f) > - # See issue 11062 > - if not isinstance(self._box, mailbox.Babyl): > - self.assertEqual(self._box.get_bytes(key).split(b'\n'), > - _bytes_sample_message.split(b'\n')) > + self.assertEqual(self._box.get_bytes(key).split(b'\n'), > + _bytes_sample_message.split(b'\n')) > > def test_add_StringIO_warns(self): > with self.assertWarns(DeprecationWarning): > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -13,6 +13,8 @@ > Library > ------- > > +- Issue #11062: Fix adding a message from file to Babyl mailbox. > + > - Issue #15646: Prevent equivalent of a fork bomb when using > multiprocessing on Windows without the "if __name__ == '__main__'" > idiom. > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > -- Thanks, Andrew Svetlov

7 years, 12 months

Installation on Macs

by Raymond Hettinger

On Mountain Lion, the default security settings only allow installation of applications downloaded from the Mac App Stored and "identified developers". We need to either become an "identified developer" or include some instructions on how to change the security settings (System Preference -- General -- Unlock --Select the "Anywhere" radio button -- Install Python -- Restore the original settings -- and Relock). Changing the security settings isn't appealing because 1) it weakens the user's security 2) it involves multiple steps and 3) the user will see an unsettling warnings along the way. Another unrelated issue is that the instructions for updating Tcl/Tk are problematic. In the past few months, I've witnessed hundreds of people unsuccessfully trying follow the instructions and having an immediate unpleasant out-of-the-box experience when IDLE crashes. I suggest that we stop being so indirect about the chain of footnotes and links leading to a Tcl/Tk download. I would like to see a direct Tcl/Tk updater link side-by-side with our Python installer link at http://www.python.org/download/releases/2.7.3/ Someone did add a note the the IDLE startup screen to the effect of: "WARNING: The version of Tcl/Tk (8.5.9) in use may be unstable. Visit http://www.python.org/download/mac/tcltk/ for current information." In some ways this is progress. In others, it falls short. If IDLE crashes, you can't see the message. If you have installed the ActiveTCL 8.5.12 update, you still see the warning eventhough it isn't necessary. Also, I don't link that the referenced page is so complex and that it is full unsettling warnings, important notices, do-not-use advice, mentions of instability, etc. I would like to see our download page have something more simple, affirmative, positively worded and direct. For example: * Mac OS X 64-bit/32-bit Installer (3.2.3) for Mac OS X 10.6 and 10.7 [2] (sig). To run IDLE or Tkinter, you need to update Tcl/Tk to ActiveTcl 8.5.12 <http://www.activestate.com/activetcl/downloads> . That saves you from having to click a links down to a footnote at the bottom of the page <http://www.python.org/download/releases/2.7.3/#id6 > that sends you to <http://www.python.org/download/mac/tcltk> which is another page full of tables, warnings,etc that leads you to the Apple 8.5.9 section <http://www.python.org/download/mac/tcltk/#apple-8-5-9> which is a dead-end because there are still known issues with 8.5.9, leaving you with the ActiveTCL section <http://www.python.org/download/mac/tcltk/#activetcl-8-5-12> which has a paragraph of text obscuring the link you actually needed: http://www.activestate.com/activetcl/downloads . I applaud that some effort was made to document a solution; however, in practice the daisy chain of footnotes, tables, and links has proven unworkable for most of the engineers I've been working with. Raymond

7 years, 12 months

PEPs build system

by Yury Selivanov

Hi, There seems to be a problem with PEPs build process again. As far as I see - PEP 362 and PEP 398 are out of sync with what is in the repo. Thanks, - Yury

7 years, 12 months

AST optimizer implemented in Python

by Victor Stinner

Hi, I started to implement an AST optimizer in Python. It's easy to create a new AST tree, so I'm surprised that I didn't find any existing project. https://bitbucket.org/haypo/misc/src/tip/python/ast_optimizer.py To test its peephole optimizations (by checking manually its final bytecode), I wrote a patch for Python to disable Python internal peephole optimizer (on bytecode): https://bitbucket.org/haypo/misc/src/tip/python/compile_disable_peephole.pa… -- There is BytecodeAssembler [1], but it seems to be specialized on bytecode. There are (at least?) 3 different issues to implement an AST optimizer, but in C, not in Python: http://bugs.python.org/issue1346238 http://bugs.python.org/issue10399 http://bugs.python.org/issue11549 -- My proof-of-concept only implements very basic optimizations like 1+1 => 2 or "abcdef"[:3] => "abc", but it should easy to extend it to do more interesting optimization like function inlining. To allow more optimization, the optimizer permits to declare variables as constant. For example, sys.hexversion is replaced by its value by the optimizer, and checks like "sys.hexversion > 0x3000000" are done at compile time. This feature can be used to drop completly debug code at compilation, without the need of a preprocessor. For example, calls to logging.debug() can simply be dropped if the log level is hardcoded to ERROR (40). Other idea to improve this optimizer: - move invariant out of loops. Example: "x=[]; for i in range(10): x.append(i)" => "x=[]; x_append=x.append; for i in range(10): x_append(i)". Require to infer the type of variables. - unroll loops - inline small functions - etc. To be able to use such optimizer on a whole Python project, a new function (ex: sys.setastoptimizer()) can be added to CPython to call the optimizer between the compilation to AST and the compilation to bytecode. So the optimizer would be optionnal and it avoids bootstrap issues. [1] http://pypi.python.org/pypi/BytecodeAssembler Victor

7 years, 12 months

Re: [Python-Dev] [Python-checkins] cpython: Review of signature docs.

by Andrew Svetlov

Thank you for review. On Tue, Aug 14, 2012 at 10:45 AM, georg.brandl <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/e1e7d628c0b9 > changeset: 78560:e1e7d628c0b9 > user: Georg Brandl <georg(a)python.org> > date: Tue Aug 14 09:45:28 2012 +0200 > summary: > Review of signature docs. > > files: > Doc/library/inspect.rst | 127 +++++++++++++-------------- > 1 files changed, 62 insertions(+), 65 deletions(-) > > > diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst > --- a/Doc/library/inspect.rst > +++ b/Doc/library/inspect.rst > @@ -397,25 +397,18 @@ > > .. _inspect-signature-object: > > -Introspecting callables with Signature Object > ---------------------------------------------- > - > -Signature object represents the call signature of a callable object and its > -return annotation. To get a Signature object use the :func:`signature` > -function. > - > +Introspecting callables with the Signature object > +------------------------------------------------- > > .. versionadded:: 3.3 > > -.. seealso:: > - > - :pep:`362` - Function Signature Object. > - The detailed specification, implementation details and examples. > - > +The Signature object represents the call signature of a callable object and its > +return annotation. To retrieve a Signature object, use the :func:`signature` > +function. > > .. function:: signature(callable) > > - Returns a :class:`Signature` object for the given ``callable``:: > + Return a :class:`Signature` object for the given ``callable``:: > > >>> from inspect import signature > >>> def foo(a, *, b:int, **kwargs): > @@ -432,24 +425,24 @@ > >>> sig.parameters['b'].annotation > <class 'int'> > > - Accepts a wide range of python callables, from plain functions and classes > - to :func:`functools.partial` objects. > + Accepts a wide range of python callables, from plain functions and classes to > + :func:`functools.partial` objects. > > .. note:: > > - Some callables may not be introspectable in certain implementations > - of Python. For example, in CPython, built-in functions defined in C > - provide no metadata about their arguments. > + Some callables may not be introspectable in certain implementations of > + Python. For example, in CPython, built-in functions defined in C provide > + no metadata about their arguments. > > > .. class:: Signature > > - A Signature object represents the call signature of a function and its > - return annotation. For each parameter accepted by the function it > - stores a :class:`Parameter` object in its :attr:`parameters` collection. > + A Signature object represents the call signature of a function and its return > + annotation. For each parameter accepted by the function it stores a > + :class:`Parameter` object in its :attr:`parameters` collection. > > - Signature objects are *immutable*. Use :meth:`Signature.replace` to make > - a modified copy. > + Signature objects are *immutable*. Use :meth:`Signature.replace` to make a > + modified copy. > > .. attribute:: Signature.empty > > @@ -462,30 +455,29 @@ > > .. attribute:: Signature.return_annotation > > - The "return" annotation for the callable. If the callable has > - no "return" annotation, this attribute is set to > - :attr:`Signature.empty`. > + The "return" annotation for the callable. If the callable has no "return" > + annotation, this attribute is set to :attr:`Signature.empty`. > > .. method:: Signature.bind(*args, **kwargs) > > - Creates a mapping from positional and keyword arguments to parameters. > - Returns :class:`BoundArguments` if ``*args`` and ``**kwargs`` match > - the signature, or raises a :exc:`TypeError`. > + Create a mapping from positional and keyword arguments to parameters. > + Returns :class:`BoundArguments` if ``*args`` and ``**kwargs`` match the > + signature, or raises a :exc:`TypeError`. > > .. method:: Signature.bind_partial(*args, **kwargs) > > - Works the same way as :meth:`Signature.bind`, but allows the > - omission of some required arguments (mimics :func:`functools.partial` > - behavior.) Returns :class:`BoundArguments`, or raises a :exc:`TypeError` > - if the passed arguments do not match the signature. > + Works the same way as :meth:`Signature.bind`, but allows the omission of > + some required arguments (mimics :func:`functools.partial` behavior.) > + Returns :class:`BoundArguments`, or raises a :exc:`TypeError` if the > + passed arguments do not match the signature. > > .. method:: Signature.replace([parameters], *, [return_annotation]) > > - Creates a new Signature instance based on the instance replace was > - invoked on. It is possible to pass different ``parameters`` and/or > - ``return_annotation`` to override the corresponding properties of > - the base signature. To remove return_annotation from the copied > - Signature, pass in :attr:`Signature.empty`. > + Create a new Signature instance based on the instance replace was invoked > + on. It is possible to pass different ``parameters`` and/or > + ``return_annotation`` to override the corresponding properties of the base > + signature. To remove return_annotation from the copied Signature, pass in > + :attr:`Signature.empty`. > > :: > > @@ -497,38 +489,36 @@ > "(a, b) -> 'new return anno'" > > > - > .. class:: Parameter > > - Parameter objects are *immutable*. Instead of modifying a Parameter object, > + Parameter objects are *immutable*. Instead of modifying a Parameter object, > you can use :meth:`Parameter.replace` to create a modified copy. > > .. attribute:: Parameter.empty > > - A special class-level marker to specify absence of default > - values and annotations. > + A special class-level marker to specify absence of default values and > + annotations. > > .. attribute:: Parameter.name > > - The name of the parameter as a string. Must be a valid python identifier > - name (with the exception of ``POSITIONAL_ONLY`` parameters, which can > - have it set to ``None``.) > + The name of the parameter as a string. Must be a valid python identifier > + name (with the exception of ``POSITIONAL_ONLY`` parameters, which can have > + it set to ``None``). > > .. attribute:: Parameter.default > > - The default value for the parameter. If the parameter has no default > + The default value for the parameter. If the parameter has no default > value, this attribute is set to :attr:`Parameter.empty`. > > .. attribute:: Parameter.annotation > > - The annotation for the parameter. If the parameter has no annotation, > + The annotation for the parameter. If the parameter has no annotation, > this attribute is set to :attr:`Parameter.empty`. > > .. attribute:: Parameter.kind > > - Describes how argument values are bound to the parameter. > - Possible values (accessible via :class:`Parameter`, like > - ``Parameter.KEYWORD_ONLY``): > + Describes how argument values are bound to the parameter. Possible values > + (accessible via :class:`Parameter`, like ``Parameter.KEYWORD_ONLY``): > > +------------------------+----------------------------------------------+ > | Name | Meaning | > @@ -577,10 +567,10 @@ > > .. method:: Parameter.replace(*, [name], [kind], [default], [annotation]) > > - Creates a new Parameter instance based on the instance replaced was > - invoked on. To override a :class:`Parameter` attribute, pass the > - corresponding argument. To remove a default value or/and an annotation > - from a Parameter, pass :attr:`Parameter.empty`. > + Create a new Parameter instance based on the instance replaced was invoked > + on. To override a :class:`Parameter` attribute, pass the corresponding > + argument. To remove a default value or/and an annotation from a > + Parameter, pass :attr:`Parameter.empty`. > > :: > > @@ -604,18 +594,18 @@ > .. attribute:: BoundArguments.arguments > > An ordered, mutable mapping (:class:`collections.OrderedDict`) of > - parameters' names to arguments' values. Contains only explicitly > - bound arguments. Changes in :attr:`arguments` will reflect in > - :attr:`args` and :attr:`kwargs`. > + parameters' names to arguments' values. Contains only explicitly bound > + arguments. Changes in :attr:`arguments` will reflect in :attr:`args` and > + :attr:`kwargs`. > > - Should be used in conjunction with :attr:`Signature.parameters` for > - any arguments processing purposes. > + Should be used in conjunction with :attr:`Signature.parameters` for any > + argument processing purposes. > > .. note:: > > Arguments for which :meth:`Signature.bind` or > :meth:`Signature.bind_partial` relied on a default value are skipped. > - However, if needed, it's easy to include them > + However, if needed, it is easy to include them. > > :: > > @@ -638,15 +628,16 @@ > > .. attribute:: BoundArguments.args > > - Tuple of positional arguments values. Dynamically computed > - from the :attr:`arguments` attribute. > + A tuple of positional arguments values. Dynamically computed from the > + :attr:`arguments` attribute. > > .. attribute:: BoundArguments.kwargs > > - Dict of keyword arguments values. Dynamically computed > - from the :attr:`arguments` attribute. > + A dict of keyword arguments values. Dynamically computed from the > + :attr:`arguments` attribute. > > - :attr:`args` and :attr:`kwargs` properties can be used to invoke functions:: > + The :attr:`args` and :attr:`kwargs` properties can be used to invoke > + functions:: > > def test(a, *, b): > ... > @@ -656,6 +647,12 @@ > test(*ba.args, **ba.kwargs) > > > +.. seealso:: > + > + :pep:`362` - Function Signature Object. > + The detailed specification, implementation details and examples. > + > + > .. _inspect-classes-functions: > > Classes and functions > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > -- Thanks, Andrew Svetlov

7 years, 12 months

Re: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Merge 3.2

by Brian Curtin

On Mon, Aug 13, 2012 at 5:13 PM, brian.curtin <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/256bfee696c5 > changeset: 78552:256bfee696c5 > parent: 78549:edcbf3edf701 > parent: 78551:fcad4566910b > user: Brian Curtin <brian(a)python.org> > date: Mon Aug 13 17:12:02 2012 -0500 > summary: > Merge 3.2 > > files: > Lib/test/support.py | 68 +++++++- > Misc/NEWS | 265 ++++++++++++++++++++++++++++++++ this Misc/NEWS disaster didn't appear in the diff. Fixing now...

7 years, 12 months

Re: [Python-Dev] [Python-checkins] cpython: Push importlib ABC hierarchy chart.

by Brett Cannon

For documentation, this appears to be for http://bugs.python.org/issue15628 . On Mon, Aug 13, 2012 at 3:19 PM, andrew.svetlov <python-checkins(a)python.org>wrote: > http://hg.python.org/cpython/rev/1c8a6df94602 > changeset: 78547:1c8a6df94602 > user: Andrew Svetlov <andrew.svetlov(a)gmail.com> > date: Mon Aug 13 22:19:01 2012 +0300 > summary: > Push importlib ABC hierarchy chart. > > files: > Doc/library/importlib.rst | 15 +++++++++++++++ > 1 files changed, 15 insertions(+), 0 deletions(-) > > > diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst > --- a/Doc/library/importlib.rst > +++ b/Doc/library/importlib.rst > @@ -121,6 +121,21 @@ > used by :keyword:`import`. Some subclasses of the core abstract base > classes > are also provided to help in implementing the core ABCs. > > +ABC hierarchy:: > + > + object > + +-- Finder > + | +-- MetaPathFinder > + | +-- PathEntryFinder > + +-- Loader > + +-- ResourceLoader --------+ > + +-- InspectLoader | > + +-- ExecutionLoader --+ > + +-- FileLoader > + +-- SourceLoader > + +-- PyLoader > + +-- PyPycLoader > + > > .. class:: Finder > > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > >

7 years, 12 months

python 2.7 + https + urlopen = ?

by Python Urlopen

I am a python 2.7.x user and am hoping to reach with this email some python developers who would be sympathetic to this scenario (And I understand that you might not, which is perfectly fine -- I've already requested one developer not to reply ) : How would you feel, if you issued : import urllib urlopen("""https://server.domain.com""").read() and the command got you data from some other URL without telling you! You use firefox, and the site is different than the data you got! Same with chrome. Safari. Even IE ! Cheated? (Well I was mad -- after IE worked). Then, you dig a little and say, hey there are bugs in networks/code, lets try the other tools that are available on python 2.x, who uses urlopen from urllib in 2012. There are tons, right? urllib2, urllib, urllib3, requests, twisted.getPage, ... None of them worked! Wow. Then you wonder, whats going on. You poke one of the server administrator, and he sends you the logs, and you see the problem. The keyword being "SNI". Now you start googling. First read about SNI perhaps. Here is a 2 line summary: SNI is a server side "feature" that extends SSL and TLS protocols to let you talk to a https server which is on an IP that serves multiple certificates for multiple https servers. SNI was first used in 2004 and OpenSSL started support in 2006. In 2007, it was backported to OpenSSL 0.9.x. In 2009 there was a bug filed with python-devs for fixing this in 2.6. The feature enhancement (or "bug fix") eventually happened -- for 3.2+. (http://en.wikipedia.org/wiki/Server_Name_Indication) Then you google more and you land up on this page: http://bugs.python.org/issue5639 which shows you that 2.6 has a patch. Then you wonder, why wasn't it included in 2.7 -- and you read -- AP : "No, Python 2 only receives bug fixes.". You instantly hate the guy. Sorry AP, nothing personal, but please do not reply to this post. I think I know what your reply will be. After a lot of pain, I got myself out of this trouble, and my code now works correctly on 2.7.x (thanks to Jean-Paul Calderone's pyopenssl). But do "you" think this is a "feature" and not a "bug"? -- And do you think debating on this, killing time on the debate, and letting all python 2.x users suffer sooner or later is right --. Something as basic as urlopen! Thanks for your time and I wish good luck to most python users.

7 years, 12 months

What's New in Python 3.3

by Raymond Hettinger

Hello all, I'll soon be starting the edits of Whatsnew for 3.3. When I did this for 3.2, it took over 150 hours of work to research all the changes. This time there are many more changes, so my previous process won't work (reviewing every "new in 3.3" entry in the docs, every entry in the voluminous Misc/NEWS file, etc). You can help out by checking-in draft entries for your favorite new features. That way, I can avoid the time consuming curation step and focus on the text, organization, and examples. I appreciate your help, Raymond

7 years, 12 months

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev August 2012